METANOMICS: LEARNING IN 3D IMMERSIVE ENVIRONMENTS IN TRAINING AND EDUCATION FEBRUARY 10, 2010 ANNOUNCER: Metanomics is brought to you by Remedy

Communications and Dusan Writer's Metaverse. ROBERT BLOOMFIELD: Hi. I'm Robert Bloomfield, professor at Cornell University's Johnson Graduate School of Management. Today we continue exploring Virtual Worlds in the larger sphere of social media, culture, enterprise and policy. Naturally, our discussion about Virtual Worlds takes place in a Virtual World. So join us. This is Metanomics. ANNOUNCER: Metanomics is filmed today in front of a live audience at our studios in Second Life. We are pleased to broadcast weekly to our event partners and to welcome discussion. We use ChatBridge technology to allow viewers to comment during the show. Metanomics is sponsored by the Johnson Graduate School of Management at Cornell University and Immersive Workspaces. Welcome. This is Metanomics. ROBERT BLOOMFIELD: Hello, and welcome to Metanomics. Today we are joined by Tony O'Driscoll and Karl Kapp, authors of the brand new book Learning in 3d: Adding a New Dimension to Enterprise Learning and Collaboration. Karl is a professor of instructional technology at Bloomsburg University, while Tony is a professor of practice at Duke University's Fuqua School of Business and actually a many-time participant on Metanomics. So, Tony, welcome back. Karl, welcome to Metanomics. TONY O'DRISCOLL: Good to be here, Rob. KARL KAPP: Thanks. Great to be here. ROBERT BLOOMFIELD: Yes, So I understand the two of you are pretty busy these weeks with a book tour, as you promote Learning in 3D, and I know you both have blogs, and you Twitter, and you've totally embraced new media. So here's my first question to you: Why write a traditional book? TONY O'DRISCOLL: Karl, you go ahead. KARL KAPP: Sure. So one of the things that, while we're in this transition to new media, books still carry a lot of weight and I think will always carry a lot of weight. But you look around the internet, there's a lot of different things going on with Virtual Worlds and 3D learning, it wasn't in one place. I talked to somebody the other day, and they said, "You know having a book makes it real and allows people, who maybe aren't as immersed in the 3D environment or in the social-networking environment or new media, to really get their head around and understand what's all involved. And what we tried to do with the book was bring together all of the disparate things going on, in terms of the 3D learning space, put it all in one place and allow either an instructional designer or a corporate executive or somebody to use that information contained in the book, to further their understanding of the world. And then, of course, we encourage them to jump in because you can't understand 3D unless you're in 3D. TONY O'DRISCOLL: Yeah, and I would say, while Karl and I may be listed as authors on the book, one of the things, as you well know in having been at the 3DTLC Conference is that this really was a group

effort when we reached out to a broad array of immersive internet kind of lead adopters, if you will, so it's really a compendium of early adopters and what their insights are. And, as Karl said, for some reason, we're still in the age where a book brings legitimacy to a particular discipline, and that was one of our hopes here was that that's what we would be doing in actually creating a book for the community, so to speak. ROBERT BLOOMFIELD: Yeah, that makes a lot of sense. Hopefully, some of the people here, who are listening, I know we have a lot of Metanomics viewers who are in the process of making pitches to their superiors, the people who handle the budgets and so on. So I think it is going to be useful to have. A hard cover, I assume? TONY O'DRISCOLL: Yeah. ROBERT BLOOMFIELD: At first anyway. So, yeah, that'll make a good gift to your boss, if you want to lend this endeavor more credibility. So hopefully, you'll have a leather-bound, gilt-edged version that the really old corporations can use. TONY O'DRISCOLL: Yeah, the parchment version is coming out soon. ROBERT BLOOMFIELD: Now one of the more entertaining stories actually isn't about the future at all, it's about the past. You tell the story of a family that takes their child to see a recreation of a sixteenth century Virginia settlement, and there's this little girl who finds all these things so unfamiliar. She has no idea what a blacksmith shop is. She's never seen anything like it. The loom is something that, again, is not part of her experience. A mill, and so on. But then you have this part I'll just read quickly from this, "As they waited for Mommy and Conner to come back from the restroom, Megan and her dad chatted about how things have really changed for the better over the last four centuries. Then they all headed toward a larger building over by the chapel. This time Megan didn't need to ask a single question, "Mommy, Daddy, don't tell me. Don't tell me. I know what this is. It's a classroom." And you go on to say that, "this visit is a testament to the fact that today's corporate schoolhouse continues to be captivated by the classroom-centric model. While traditional industrial processes in manufacturing, transportation, textiles, apparel and food processing have been transformed to become unrecognizable, Megan was instantly able to recognize the classroom." So why is it, in your views, and you probably have different views on this, but why is it there's been so little change in models of education? TONY O'DRISCOLL: Yeah. I'll take that one. I think that we've just been so schooled in what school is, and those people who are outside the world where Karl and I live, which is studying instructional design and all the different types of even philosophies that are out there, in terms of what's the best way to educate adults, our mental model is school. So executives who are writing checks to invest in the development of their people, when they're writing the check for quote/unquote "training and development," the prevailing mental model is the corporate schoolhouse. And the prevailing budgeting mechanism is butts in seats and how many people can we train. And I'm kind of a fan of Alvin Toffler. He has this quote, "Now that we are moving from factory work to anytime-anyplace work, we really do need an anytime-anyplace educational parallel." And I think Karl and I are in agreement that this environment, the virtually immersive environments really truly do provide, for the first time, an actual anytime-anyplace educational parallel. Karl? KARL KAPP: I agree. I think what happened is, when we moved--I do a slide show sometimes, and I show a journeyman and apprentice where the work happened right where the learning happened. The two working hand in hand. But then, as we went into the eighteenth century, that wasn't a scalable model because we [AUDIO GLITCH] the journeyman-apprentice approach, we would have one-to-one [AUDIO GLITCH]. But now I think, with Virtual Worlds, is--oops, that's my land line there. Tony, I'll turn it back to you, but, with Virtual Worlds, I think [AUDIO GLITCH]

TONY O'DRISCOLL: Yeah, I think that's a key point. So one of the things we talk about in being in the educational sphere is the notion of a teachable moment, Rob. And, in a world where work is becoming increasingly complex and non-routinized and most of us becoming more knowledge workers, the teachable moments emerge from the workplace, where it's like, "Oh, my god, I don't know how to do what I have to do to get my work done." And so, in the past, in the master-apprentice model, we really had the situation in the context presenting the problems and the master and the apprentice being able to learn in the context of doing the work. When we had the printing press and we were able to decouple knowledge from activity, clearly we could reach more people, but we kind of decontextualized the content down to a point where it becomes devoid of context. And what Karl and I firmly believe is, we're now back to perhaps being able to, through a digital format, move back more towards the master-apprentice model, which I think most people would agree is a great way to learn. We just weren't able to scale it in the past as we might be able to now by leveraging platforms such as the one we're in today. ROBERT BLOOMFIELD: So, Tony, to what extent is Learning in 3D and what you present there applicable primarily to business education, to corporate training, as opposed to all of the other education in primary and secondary schools, for example? TONY O'DRISCOLL: Yeah. And this comes down to the different types of knowledge, and we do talk about this a little bit, but the book itself, as you well know, Rob, on the front end is kind of descriptive and clearly this audience doesn't need to know what a Virtual World is or what an avatar is. But some in the educational community and the enterprise community do, so we're descriptive about that. But then we get into, on the back end, a little bit more wonky, if I could use that term in terms of instructional design, where there are kind of conceptual things to be known: What is gravity? And what's the capital of Kuala Lumpur and those type of things, declarative knowledge as it's called. Clearly there are things that students need to know to move and progress through grades. And a lot of those things are not about decision-making or synthesis or making sense of things; they're actually just about memorization and what are my times tables. I was going through the times tables with my son last night. ROBERT BLOOMFIELD: I remember those days. TONY O'DRISCOLL: Yeah. Would a Virtual World help with those? Well, probably not. But I think that the Virtual World can be an integrating space where you move beyond that kind of lower-level declarative knowledge and into decision-making and synthesis and trying to make sense of disparate datasets with a group of people so that you can, I don't know, place an investment as a firm. We call that, or at least I tend--Peter Senge coined the phrase, I think, "generative learning," where you have to actually come up with knowledge that hasn't been come up with before. In the past, what we used to say is, let's say we have to innovate a new product, we're going to invest the money to fly people because you can't help but having people get together if they want to co-create. And I firmly believe, having been knocking around these environments for a long time, that you can truly get a lot of co-creative juice flowing in a virtual immersive environment such as this. So for baseline curriculum, perhaps not as relevant, if you will, but the more you go up into the higher order of cognitive and a decision-making comparison, all those type of activities and synthesis coming to the application of the knowledge, I really think this platform, meaning a virtual immersive environment, not necessarily just Second Life, but any of them, can truly be a game-changer. Pun intended. Karl? ROBERT BLOOMFIELD: I don't know if we have Karl back. KARL KAPP: Yeah, here I am. Yes, I agree with that. I think the ability to learn anytime and immerse yourself in an environment, and actually there's a lot of research that says there's still a lot of [AUDIO GLITCH] times tables memorization and things like that, but there's also research that seems to indicate that you may learn and retain [AUDIO GLITCH] narrative [AUDIO GLITCH] ROBERT BLOOMFIELD: I don't know if it's just me, Karl, but I'm actually getting a bit of garbling on your

voice, and so I'd like to ask, if I could, if JenzZa or Jeanette can work with Karl privately and see if we can improve his voice, let's do that. KARL KAPP: Okay. ROBERT BLOOMFIELD: And, just for the moment, I will move on with Tony. Actually, I just want to say there's just an incredible amount of chat going on in the stream, and there are a couple things I'd like to pick up on. There are two terms that I am seeing that I know from business contexts and people are applying here to education. One of them is from toBe Destiny, refers to demand-side pedagogy. I've never heard that term before, but I know, certainly in business, we talk about demand-side and supply-side models and demand-side marketing and production is usually what we think of as being the way to go, that we're not making stuff and then pushing it on customers who might or might not want it. We identify what people want, and they basically pull the stuff out of us. I don't know. Are you familiar with this term in education? TONY O'DRISCOLL: The term itself I haven't run across, but the kind of underlying premise, I think, certainly. On the front end of the book, we kind of present an argument that says technology, as you mentioned, that's the kind of the story behind the [Manteo?], the family that goes to Manteo's. Technology has fundamentally reshaped business from the ground up, and then the question is: Can learning be far behind? And one of the things that we get into then is what we call the seven scary problems that have to do with learning. And the first problem is what we call the autonomous-learner problem. So the shorthand for that one, Rob, is sometimes instructions in context are better than instruction out of context. So in other words, in a world or a mediascape that we live in today, learning is competing with search, learning is competing with Twitter. One framing of learning can be, "How much knowledge can I pour into Abbott's head or Beyer's head?" And another framing is, "How quickly can I tune my network to the challenge or opportunity in hand?" Now if you think about the second one, not the first one for a second, most schooling is built around how much did Beyers consume, how much did he retain, and could he regurgitate it on the test. And for the lower-level declarative knowledge makes complete sense. But more and more what we're seeing is, on the demand side if people encounter a need, basically a learning gap, they have autonomy in their ability to avail themselves of the wealth of digital resources that they have at their disposal. And the way I say this to learning professionals in their enterprise is, "What do your employees hit more, the internal search engine or your learning management system?" And they become really quiet because, truly, I think certainly in the last five to seven years, learners have a lot more autonomy in being able to serve themselves when a knowledge gap appears within their work context. ROBERT BLOOMFIELD: Another item from the backchat is from Mystical Demina. This is another business term "just-in-time learning." So I'm familiar with just-in-time inventory. But Mystical says, "Although just-in-time learning is the result, isn't the real change similar to what happened to commerce, in that the physical location restraint was removed, which lets us use economies of scale and have a more decentralized learning environment?" How much further does it go in education? If all we had was the ability to do distance education, I take it that would not be merely what you are looking for in 3D learning. TONY O'DRISCOLL: Yeah. Karl and I joke that, if all we get out [AUDIO GLITCH] scary problems we talk about, which is routinization, a common, common thing that we see with the application of technology is that we take a radically new technology to automate the past, bad assumptions and all. So in a way, that joke about the family and the classroom is, here comes the internet, one of the most revolutionary collaborative platforms known to humankind, and we end up applying it to automate the classroom context as opposed to allowing it to be more of a peer-to-peer environment. So back in the days of knowledge management, KM1.0, if you will, which I had the good fortune of working with Larry Prusak at the time. Larry said to me, "Knowledge management is actually an

oxymoron. It can't be managed. It has to be enabled and curated and propagated. And I really believe that now--we used to talk about just in time, just enough, just for me, just the right context. Think about Terminator, the movie, the person's moving through and getting the information in context, all of that can be done inside of these environments now. But I don't want to come across as someone who says that means that the classroom in the school house go away. We just believe that we actually have a much broader palette to work with now. So absolutely the notion of just-in-time learning, our apprentice model, as Abbott was talking about earlier, can be integrated into this environment. But also, as we point out in the book, that requires that we think differently about the instructional design approaches and the archetypes and the macro structures as we're integrating this environment into an overall learning architecture. Abbot, are you back? KARL KAPP: [NO AUDIBLE RESPONSE] ROBERT BLOOMFIELD: I got a message indicating we should give Karl another try. Karl, do be sure to click your talk button or unmute your mike. When you're ready, just pipe up. I would like to point that, very much in line with what you're saying about routinization, Tony, that Exosius, who's watching on the web, is saying that basically the first attempt by just about every professor, when they see the web, is, "Oh, well, I will lecture, record it, and then broadcast it on the web." That's pretty much what a lot of us see as technology's opportunity for us in education. Actually, I'd like to build on that, just before turning to some of the specifics in your book. I have one more high-level question. I feel like people have been promising that technology will revolutionize education. I've heard this decades ago. Even when we talk about Virtual Worlds, the Virtual World platform Active Worlds has been around for, I'm not sure, I think more than 15 years at this point, or maybe not quite that long, but for a while. So my question is: Is it reasonable to think that the next five years are going to be different and that we really will see some revolutionary changes in education that put this technology to use? TONY O'DRISCOLL: Im kind of on the fence on that one, Rob, to be honest with you, because I think, if you look at the demographic, and I was just reading some stuff this morning about just how quickly the one below the net generation, if you will, is assimilating technology. So, to them, technology is just an appliance. It's like air, and they use it to do stuff. Randy Hinrichs, who's featured prominently in our book and who I think is a wonderful thinker in this space, kind of said, "The Virtual World is not about being there, it's about doing there." And I think that the next generation coming through really sees all of these myriad technologies and integrates them in a mashup at the moment of need, as they see fit. So in one way I think we've kind of got a generational pressure that's going to demand that the work of enterprise be a little bit more Web 2.0-ish collaborative mashupish, so to speak. On the other side though, when you start getting into educational policy and how we can actually leverage technology to do things differently but still need the kind of evaluation criteria. I keep going back. I just had a student-teacher meeting about my son with his teacher, and it's like there are a set of criterion through which every student needs to be assessed and evaluated. And that probably won't change for a while. So I could see it permeating around the edges, but, in terms of a full-scale revolution where learning moves from evaluating how much knowledge is poured into someone's head and being able to train to the test for third, fourth, fifth grade versus the kind of hive mind of these people on the outside. One of the kind of interesting things when we were looking at students and people playing World of Warcraft when I was back at IBM was the idea that, on Facebook or MySpace, mostly on Facebook, students were getting together, and they were colluding to get the work done, so if somebody was good at history, well, you go write the history paper, and then we'll just change it a little bit and submit it. And, as a professor, I'm going, "Oh, my gosh! That's sacrilege. You're cheating." And one of the kids flippantly said, "Well, cheat and teach are anagrams. We're getting the work done, aren't we?" ROBERT BLOOMFIELD: I hope none of my students are listening.

TONY O'DRISCOLL: Right. Well, you know as a professor and as Karl does too, we get all of these tools that can allow us to filter to see whether or not there's been plagiarism and so on. ROBERT BLOOMFIELD: Mm-hmm. Yeah. TONY O'DRISCOLL: I guess what I'm saying is, from a societal perspective, I see the next generation down using the more networked approach to problem solving and learning being more prevalent. But we still have educational institutions that have to hold each individual to certain standards. So that's not going to go away. And nor should it, to my way of thinking. ROBERT BLOOMFIELD: I'd like to turn to some of the specifics in your book, but before I do I would just like to react to a couple things that I'm seeing in the chat. One is Joel Foner asking whether teachers are taught anything other than how to lecture. And I would say those of you who are going to be coughing up big bucks to send your kids to college, you should know that most college faculty are never taught how to teach in any way, which is, of course, a problem that we struggle with. Naturally it's different in the lower levels. Let's see. It looks like the chatterers have it pretty much under control; lots of interesting stuff going on there. So while they chew over those issues, let's move on to your book, and, in particular, you have a number of case studies where Virtual Worlds are used in novel ways to accomplish educational goals. And I'm hoping you can walk through some of those, and, in particular, give us the context and identify what you call the archetypes that are being employed in these particular cases. I don't know if you want to start by defining what an archetype is or maybe just jump into one of my favorite cases, which is the Ernst & Young auditing exercise. TONY O'DRISCOLL: Now why did I know you'd like that one? ROBERT BLOOMFIELD: Im predictable. TONY O'DRISCOLL: Yeah. So let me jump into that, and then, hopefully, Abbott can come in and share some of his insights on other cases. This was some work that was done, again, Ran Hinrichs, from 2b3d, was partnered with me on this, and Mike Hamilton, who's the very visionary chief learning officer for Ernst & Young. And it was a challenge, Rob, and you're probably familiar with this: On average, a new hire coming into E&Y will take about 130 hours of formal learning in the first year. That's not an insignificant amount of education. Right? And then, from the time of hire, let's say they get out of MBA school or they get out of finance, accounting undergrad, to the first inventory observation can be as little as four months. So they don't want to come right after school gets out; they want to go to Europe and travel around, and then they want to start work. So what Mike Hamilton was curious about was whether or not situating people inside the actual context of an inventory observation might be a more efficient way of transferring the inventory observation knowledge. So the way they used to do it was, they would have all of the kind of vernacular, what's work in process and finished goods and what things sit in inventory and how we tag, right, the kind of procedural elements of doing the inventory observation. And then they do a case study about, "Hey, we're going into Grandma's cookie factory, and here's the situation. What would you do?" And the instructor, as you just mentioned, would facilitate a conversation about what would you do or what wouldn't you do. So we were fortunate here, in that we could actually have a control group that went through that process. But the other thing we did was, we then created--we, Randy did--a complete cookie factory in Second Life, and, in that context, those students were taken through the same conceptual content in terms of definitions and so on and so forth. But they instead [AUDIO GLITCH] the other cohort was asked to go into the cookie factory, meet with the plant line manager, try and go through their inventory observation and reconcile account. So why 3D? Mike's hunch that inventory observations are situational, contextual in nature. And a lot of things pops up, like the plant manager saying, "I've got to get home. It's my anniversary. Are you done

yet?" Or, a truck backing up and saying, "Hey, this is warranty returns. How shall we tag it?" Or, walking over into the corner of the building, and there's been some water damage, and you just don't know what to do with that particular situation, in terms of how to tag that inventory. Or you move something on a shelf, and, behind it, you see some older equipment that hasn't been accounted for. All of these situational things are things that really matter to doing a good inventory observation, but they don't necessarily come through in a case situation. Right? So unanticipated situations on the ground require quick decisions, and that's part of what Ernst & Young wanted their people to understand. So we put them through that whole thing, and two things we found out was the participants who went through the 3D learning environment learned and retained as much as their instructor-led training counterparts. What that means is, on a knowledge test of what is finished goods and work in process and all that, their results were comparable on retention. The cost and time requirement for the 3D learning environment was a lot less because it was you went in, and you ran through it yourself. You didn't have to have a facilitator there facilitating the conversation. And here was a really interesting finding, Rob: The participants who went through the 3D learning environment context felt a lot less confident in their ability to perform an inventory observation than their instructor-led training counterparts. So what does that mean? That means, if I come in and I ace the test and I can have a meaningful conversation about the case, I feel very confident that I can go in and do an inventory observation, right, which is not what you want. You want the people feeling a little edge. Whereas, the folks who went through the virtual context felt, "You know, there's more to it than meets the eye with an inventory observation so I'd better go in with eyes wide open." So that's the story on kind of Ernst & Young's challenge in bringing folks in, trying to get them up to speed and understanding more of the context, if you will. Go ahead. ROBERT BLOOMFIELD: I was just going to say I thought that was very interesting on the confidence because you know it cuts both ways. On the one hand, we get concerned when our students are not confident that they know what it is that they're supposed to be doing. On the other hand, certainly, when we're sending people out into complex auditing settings, that's probably a more accurate representation of their true state of knowledge, which is that they've only skimmed the surface. I see there's a comment from toBe Destiny, saying, "What I've been dealing with is the stream of military types from a military university, where the individuals have learned to pretend that they know, when, in fact, they've not learned and have not been exposed to the actual content they were supposed to have mastered. So it sounds like what you're saying is, just from sort of a cost-benefit perspective, we can use Virtual Worlds to put people in very complex situations, realistically complex situations, and use a lot less instructor time in these self-directed activities. TONY O'DRISCOLL: And that word "mastery" is key because I think mastery, in our world of pedagogy and instructional design means demonstrated ability to perform. So I'll go back to Randy's beautiful one-liner: It's not about being there, it's about doing there. So the inventory observation context that was set up was essentially a platform for demonstrating mastery over the conceptual content of "do I know what work in process is, or do I know what finished goods is, or do I know what to do with a particular situation." But now you have, rather than waiting for the new hires to go out and maybe muck it up in a client context, you have an environment against which they can develop that muscle of mastery by applying the declarative knowledge in a real context. ROBERT BLOOMFIELD: Let's put this particular Ernst & Young training session in an academic or theoretical context. I believe you two, in this book, are the ones who have defined these words--archetypes, sensibility, and so on--that you use to describe the benefits of learning in 3D. So could you tell us what those terms mean and how they help us understand this particular exercise better?

TONY O'DRISCOLL: Sure. Let me defer to Abbot. Can you go for it, Abbot? ROBERT BLOOMFIELD: Hi, Karl. KARL KAPP: Yes. Hi, how are you? Sorry about that, guys. ROBERT BLOOMFIELD: No, it happens. KARL KAPP: Somebody in the chat was talking about the digital divide and the computer haves and haves-not so I guess I was the have-not. But what we wanted to do with the book was define common terms. So we found that, if you got six people in a room and asked them the definition of Virtual World learning, you'd get seven or eight different answers. So we thought one of the things that needed to be done was to create a common lexicon so that we could talk about the topic. So we start with the sensibilities, which are what makes a 3D learning environment unique. So one of them, of course, is sense of self. One is death of distance. Then we say, "Okay. If you have these sensibilities, how do you design instruction to leverage those sensibilities?" And we came up with archetypes. And the reason for that is because I think we both had experiences where we went into virtual classrooms; a virtual instructor showed us virtual PowerPoints. And we said to ourselves, "Wouldn't it be better to be immersed in some type of a learning environment?" And then, on top of that, we had something called macro structures, which are just broad categories into which the archetypes and sensibilities can fit into. And then, finally, we had principles for designing instruction. And, as Tony said before, one of the main goals of what we wanted to do was both provide a foundational theoretical approach, but also some practical, "Roll up your sleeves. Here's how you would create instruction for immersive learning." And the other thing I want to point out in one of the chats, neither Tony nor I nor any of the contributors believe 3D learning is the only method of instruction. It's another tool in the toolbox. But, if we don't use it correctly, it will always be relegated to the back because people won't understand how to use it properly. And, if you take a CEO, for example, and say, "Yeah, 3D worlds are great. Look at this," and you bring him into an office or a classroom, he might say, "Uh-uh. What's the big deal?" Or, "I've got WebEx." But, if you bring him into a factory and say, "Okay. Let's look at all the compliance violations. Let's look at streamlining manufacturing processes by moving this piece of machinery over here." And those types of things, using the archetypes, the sensibilities, the macro structures and principles will make learning, in this space, very effective. ROBERT BLOOMFIELD: Okay. Great. And, Karl, I guess you've been covering closely the Penn State Initiative. Can you talk a little bit about that case? KARL KAPP: Sure. Penn State's really fascinating. They have somebody onboard at Penn State College, who's their social networking expert and works with alums and others. The person that I worked with her, but another individual Mary Ann Mengel(?), at one of the branch campuses, and she worked there with a faculty member to create a greenhouse, solar panels, tankless hot-water heaters. And she actually did a comparison of traditional instruction and the instruction with the virtual home and found that learning occurred in both venues a little bit more in the 2D environment. But she brought a really kind of an interesting element to the research problem, which is, a lot of times we compare the 2D virtual classroom instruction or classroom instruction to 3D learning, but we only use 2D assessments. So for example, she said, "Okay. The students will go to this house. They'll look at it. They'll explore all the learning that occurs in the home, and then, in order to compare it with other instruction, we'll give the learners a multiple-choice question and find out who scores better." I think that's the wrong approach. I think what we need to think about is, if we're doing learning in 3D, we really need to think about our assessments: Are we doing assessments that are congruent to what we're trying to teach? So for example, in that particular case, it might have been better to take the learners to an actual house that has

solar panels and has tankless hot-water heaters and ask the students questions in that context. So this class actually used the 3D virtual space to expand the class beyond what could be covered during the normal class hours and allowed students to explore that. And the students' comments were very interesting. They really liked the idea that, in a Virtual World--and they used Second Life for this example--they could fly because then they could go to the roof and look at the solar panels. Then they could teleport to the basement, look at the tankless hot-water heater. So they liked the idea that they could get the context of how everything fit into the system as opposed to the 2D instruction, which gave you the information and images, pictures and text, but it didn't give you the context of how the system fit together. ROBERT BLOOMFIELD: Okay. Very interesting. Any observations, Tony, on that project? TONY O'DRISCOLL: No, absolutely. I was just tracking on the chat there, and a couple of people called out: Kenny Hubble in Second Life; Ken Hudson, from Loyalist College, and the research he's done. And I think he's been on the show, if I'm not mistaken. ROBERT BLOOMFIELD: Actually, no. TONY O'DRISCOLL: Oh, Okay. Well, let me just give a little context here. This was post-9/11 with border patrol crossing from Canada into the United States, and, clearly, that's something that has an element of protocol and procedure and questioning, but it also has an element of intuition as to whether or not, "Well, maybe I want to search this car. Maybe I don't." So Kenny and his team at Loyalist College were tasked again developing an environment to train border crossing, border patrol, border crossing agents. One, in the procedure of asking the right questions, but secondly, in making a value judgment as to whether or not I'm going to hold up traffic and test out this car. So it was mentioned in the chat, and we can follow up, but Ken has really done nice work to demonstrate that those who went through this experiential context, their demonstrated ability to perform and their retention was much stronger. And I'm completely with Karl on this. Don't get an academic started on any area that theyve study a lot. One of the things I think with evaluation that's an issue is that we don't take it far enough, going back to that comment earlier in the chat about mastery. Mastery is about the demonstrated ability to perform. Right? So, if I'm going to have heart surgery, I want to be sure of the doctors demonstrated ability to perform. Ill look at the sheepskin on the wall, and it'll say whatever, University of (insert best school here for heart surgery), but I also want to know, "How many have you done successfully?" ROBERT BLOOMFIELD: That would be Cornell. TONY O'DRISCOLL: That would be Cornell. There you go. And, "What is your success rate?" and so on and so forth, in your ability to perform. However if you look at most of the evaluation that we do, particularly in the K through 12 back to your earlier question, it's proved to me that you know. And Jeff Pfeffer and Bob Sutton wrote a really good book out of Stanford a couple years ago called The Knowing-Doing Gap. I affectionately refer to it as my mother's problems. She's smoking a cigarette. She's saying, "These things will kill me," so it's not a knowledge problem. It's a behavior problem. The doing doesn't follow the knowing. And I do believe that, in these environments, because they are experiential in nature, we have more of an opportunity to practice the doing piece. And that's where mastery really comes. It's the synthesis of the knowledge with a situation, where the aha moment really happens and synapses get rewired, in my opinion. ROBERT BLOOMFIELD: I see a lot going on in the chat here on assessment. Some people are picking up on an earlier comment that one of you made about 3D versus 2D assessment. It's all going by so quickly, I don't remember who exactly said it, but one of the questions was: Is a paper and pencil test of learning from a 3D environment invalid or a bad fit? KARL KAPP: Well, I think we want to go back to what we're measuring. For example, in the case of a

solar house, if you just want people to be aware of something, then I guess a paper and pencil test is good. I always say, "Look at the show Are You Smarter Than a Fifth Grader?" People say, "Wow! That shows how smart fifth graders are or how much adults don't know." To me, that shows how much trivia is taught in fifth grade, that people don't need. So I think the problem with assessment is that we need assessment to be authentic to the task we want the learner to perform. So for example, if we want a learner--because we always say, we teach math because we want people to be engineers, but they don't do any engineering in a school setting. They learn about engineering. They study engineering. They regurgitate engineering information. They don't do any engineering. So what we have the opportunity to do with virtual 3D environments is allow some engineering to occur. And, of course, you've got to have the fundamental declarative knowledge. We need to assess that, and we need to see that, but, if we're really going to be competitive as individuals, organizations, etcetera, we really need to move the assessments more toward authentically what happens. We know from research that one of the things that separates experts from novices is their ability to recall experiences and then leverage those experiences in a new learning situation. And Virtual Worlds can give us those vicarious experiences and move us ahead. So for example, I was reading an article the other day in Wired, that said the football players today are far more intelligent, more strategy-savvy than ever before, and the reason is Madden NFL Football. These players have literally played thousands of hours of football, something actually that's physically impossible if you want your body to last for any length of time. So when they go out on the field, they've seen every defense. They've run every play. And that's what Virtual Worlds allow us to do. So yes, we have to assess whether or not they have the basic knowledge, but what it allows us to do is assess that higher-level thinking, which heretofore was very difficult to assess in a traditional academic environment. ROBERT BLOOMFIELD: Well, I'll say Madden Football taught me what a Cover 2 defense is, but I'm afraid it also taught me: throw a long pass on every down. TONY O'DRISCOLL: Rob, if I could just weigh in on this one because-ROBERT BLOOMFIELD: Sure. Sure. TONY O'DRISCOLL: --in our world is kind of, it's at the nub of what we're talking about here. Jerome Groopman wrote a book called How Doctors Think, and it was very popular. And essentially what he was doing is, it occurred to him--he's done multiple surgeries, and he's been very successful--that, if he just externalized what he was saying to himself when he was doing this, that other people might benefit from that experience. Right? ROBERT BLOOMFIELD: He did sort of like a verbal protocol? He just talked--yeah, okay. TONY O'DRISCOLL: Yeah. Yeah, not unlike what Karl was saying, is, "Oh, this is just like that one I had 20 years ago, and that was the first time," and so on and so forth. ROBERT BLOOMFIELD: And can I just get that name again of that author? TONY O'DRISCOLL: Jerome Groopman, G R O O P M A N, I believe. Don't quote me on that though. So just look at med school for a second, theres kind of two schools of thought in med school. One is, you spend the first two years getting all the declarative knowledge down: the endocrine system and the limbic system and the knee bone's connected to the thigh bone, and the venal system, and the arterial system. So you're basically just learning all the stuff. Right? And then after your second year, you start going into the field, and Beyers shows up and says, "I got a sore throat." And you've got to parse out, "Is that just a sore throat? Is it a cancerous node? Is it lymph node? What is it?" Right? So one model is learn all the content and then go to apply. The other model is the first day you show up at school somebody said, "My knee really hurts." And you've got to rifle through the content to figure out whether or not that's like, "Oh, my gosh! We really need to do

10

surgery or take an aspirin and see me in the morning." Right? Two very different ways. One is kind of a problem-based learning approach; the content ships with the freight. The other one is, let's make sure you've demonstrated mastery over the content, and then we'll let you apply it. So which one do you think, over time, would yield better retention and knowledge of being a doctor and, more importantly, less mortality, just intuitively? Abbott? KARL KAPP: Hello. Well, I think you know what my answer would be. TONY O'DRISCOLL: Yeah. So the idea here is that one of the things and my belief, now this is just Tony speaking. I'm not talking about research, I'm talking about Tony, is that motivation ships with the freight in the second model because it's like, "Man! I don't want to screw up here. They say they got a sore throat, but, if it's something much more serious--." So you're really motivated to kind of crawl through all that content to make sure that this person has an accurate diagnosis, right? So one of the questions underneath your question about the paper and pencil tests is, I think, if you're going to design instruction that's a problem-centered way, well, you better make sure that every instance of a particular diagnosis covers everything, like you better make sure that, if that's the model you're on, you don't miss something like the venal system. In other words, all of your problem-centered learning needs to add up to covering all of the content, right? But then the question becomes if you do do a problem-centered way of designing instruction, does the kind of conceptual knowledge ship with the freight? My believe is that, if it's designed properly and we hope that some of the design principles we share in this book will help folks design that properly, some of that kind of declarative knowledge, the conceptual stuff, will ship with the freight through application. So I actually believe that problem-based learning could have a highly accelerative effect on picking up declarative and conceptual knowledge along the way. ROBERT BLOOMFIELD: When I summarize the lessons that I can pull from Learning in 3D, your book, how much of it is because of the affordances of the technology, and how much of it is actually just a push for problem-based learning? And, as you said, doing, not knowing. TONY O'DRISCOLL: Karl, do you want to take that or want me to go? KARL KAPP: Yeah, well, let me take a stab, and then you can, Tony. I'm not sure that we have a straight percentage. There are definitely affordances. So for example, we talk about the avatar persona. And research has shown that people really invest in their avatar, and some research has actually shown that--and we were talking about this a little bit yesterday--that first-person view in a game is good for procedural knowledge, but third-person view is good for conceptual knowledge and actually driving behavior change for leadership skills and things like that. So I think we're just now on the edge of research exploring which type of view, which type of affordances really accelerate because they're in the 3D Virtual World. But we definitely know that people invest characteristics in their avatars, that make it effective for learning. And there was even a study that showed that people that imagine themselves voting in third person were more likely to vote than people who imagine themselves voting in first person. So there's definitely some idea behind that. But also, with new technology, if we're smart about it and avoid the routinization trap that Tony talks about often is that we need to design differently, and traditional technology doesn't allow us to do that. We can't, in a classroom, go on a field trip. We can't go to the factory. We're completely in a different context than where the work occurs. The 3D environment though allows us to be in that context, and that's what helps us accelerate learning. And the only other point I want to say here--it's from the backchannel chat--is that the future's not going to be in either/or. The future's going to be a blend. We're going to have a blend of 3D learning experiences, with traditional type of instruction, with classroom, with social media. So the idea as educators is for us to figure out what is the right blend, and how do we deliver that blend intelligently to the learners. Tony, I'll let you take a stab at it. TONY O'DRISCOLL: Yeah. Problem-based learning and I think another case that we have in the book

11

that you did cover was Maryland traffic incident, Rob. ROBERT BLOOMFIELD: Yeah. We have discussed that on Metanomics before. TONY O'DRISCOLL: So the challenge on that one is economic. Right? Or a big military exercise is like here in North Carolina, they've got one I think called Robin Sage. And this is not an insignificant economic outlay. You're squirting blood on people, and you're having to try and put people into the situation. So I think there's a strong understanding, particularly in large operations or military context, that a problem-based approach can yield accelerative learning, but the economics of it are really high. So to your question of, is this problem-based learning in sheep's clothing or whatever, I think it's true. If we can--really now, I think these platform allow us the opportunity to explore more of the possibilities of problem-based learning can bring just by virtue of the fact that we can do it much more economically here than we can trying to hire actors and keep a highway hostage for a day while people are run through the paces. And then the question, not dissimilar to what we saw with Ian Wise'(?) approach, or, as Karl was alluding to, with the virtual greenhouse, is about how high fidelity you need to be. And Ted Castronova's talked about this, I think, on your show, that cognitively whether we're in a virtual or physical world, the mind doesn't necessarily discriminate. The suspension of disbelief happens relatively quickly once we're immersed into a catchy and kind of immersive situation. So you suspend disbelief, and you become part of it, and your avatar persona takes over. And so, for that reason, I do believe that this is the first time immersive environments can provide us with an opportunity to truly test the limits of what problem-based learning can do. ROBERT BLOOMFIELD: Interesting. I see we're basically at the top of the hour so I think that may have to be the last word on the subject. Karl, any one last thought you'd like to leave us with? KARL KAPP: Well, the only thing I'd like to leave us with is that I think we're right on the tip of the potential of Virtual Worlds, and I just want to say, when people say, "Oh, they'll never catch on," or, "I don't think that they're going to work," I always remember teaching. I literally had classes where we had to teach people that a blue underlined word was a link, and when you clicked on that link, another window actually opened. And sometimes it opened behind the window that you were in. Now we take that all for granted, and I think, in a number of years, learning in a 3D Virtual World will be just like that: we'll take it for granted. ROBERT BLOOMFIELD: Well, as my mother would say, "From your mouth to God's ear." Thank you, Karl Kapp, Tony O'Driscoll, for joining us on Metanomics to talk about your new book, which I believe is available on Amazon. Is that right? TONY O'DRISCOLL: That's right. ROBERT BLOOMFIELD: For people who are looking for it. KARL KAPP: It is. Yes. ROBERT BLOOMFIELD: It is called Learning in 3D: Adding a New Dimension to Enterprise Learning and Collaboration. Thanks so much for joining us. KARL KAPP: Thank you. TONY O'DRISCOLL: Thank you. Really enjoyed it, Rob. Appreciate it. ROBERT BLOOMFIELD: So did I. And, as usual, we will close our show with a brief attempt by me to connect some dots. It's been very interesting listening to this conversation with Karl Kapp and Tony O'Driscoll as they talk about their book Learning in 3D because I am in the process--this is my

12

heavy teaching term--and so not only am I dedicating myself to teaching and thinking a lot about teaching, but I'm teaching a topic called Managerial Accounting. It has given me, I think, some useful insights into understanding the challenges of revolutionizing teaching with technology. So the job of a managerial accountant is to link an organization's strategic goals to a set of metrics that can be used to guide and evaluate products, processes and, of course, people. And once we have these goals and we've linked them to the metrics, then the managerial accountants can work with managers to identify initiatives that are going to improve the metrics and thereby help the organization achieve its goals. So that's managerial accounting in a nutshell. From this perspective, I'll start by observing that the strategic goals in the classroom haven't really changed much in hundreds or perhaps thousands of years. Basically, we want people to know more and to be able to do more. And we can flirt with variants of those basic goals. I think a lot of discussion in the U.S. on the difference between knowing information and knowing how to find it or knowing how to analyze it. But I'm not sure that really changes the classic "know and do more" model. And then, on the other side, when we look at metrics, I would say that the methods of evaluating performance haven't changed very much. Certainly, in business school, what we call deliverables--exams, essays, presentations, problem sets--are pretty much what they've always been. And I would argue that this is because the basic goals of education haven't changed. So we're not going to change our metrics unless we have a substantially different and better measurement technology, or--and then this is going to be my big focus--or we identify new initiatives. I see this is where I actually have a lot of sympathy with Karl and Tony's push for problem-based learning. Because, when I think certainly about business education, it's very much we are training people to put them in problematic situations, and we want them to be able to handle them. So yes, there is know more but, really, do more is the key to our success in business education, helping people do more. Where I see the true promise of learning in 3D is through simulating environments, in which we can test what our students can do. Let me just go back once more to this construction of what managerial accounting is, where we are tying strategic goals to metrics and then coming up with initiatives that will improve the metrics and therefore indicate we're getting closer to our strategic goals. That's what people do in business, and that is also the learning process. And so successful simulation will be when we can put someone in a setting where the goals and the technology for developing metrics do a good job of matching what the students will face in the field. That is the big challenge that educators have, which is constructing a faithful high-fidelity environment. The technology, as it always seems to be, is the easy part. The hard part is putting it to good use. So I know, just looking at the chat, there are a lot of people out there who are working on doing exactly that. I wish you the best of luck. If you're making progress, please let me know about it, and we can talk about it on a future episode of Metanomics. We'll be taking a couple weeks off, but coming back in I believe it will be the first week or so of March. So enjoy your snow. And I will see you in not too long. Rob Bloomfield, Beyers Sellers, signing off for a couple weeks. Bye bye. Document: cor1078.doc Transcribed by: http://www.hiredhand.com http://www.hiredhandtranscription.org

13