Moderated Industry Panel

Sebastian Thrun
-Udacity Many years ago Dan Goldin of NASA declared the age of better, faster, and cheaper. I only have good news for everyone in the room. This is going to be the age of better, faster, and cheaper for higher education. I think we are the advent of discovering new technology and new pedagogies that make higher education more accessible more affordable and spread higher education to many more people than ever before. Udacity we started our journey into higher education last fall with launch of a Stanford class (introduction to artificial intelligence) that you might have heard of...and we attracted 160,000 students and we were all over the news mostly for the cheaper part. Because apparently we found a way to educate students for a dollar a student. But what was largely not noticed is the opportunity to be better. We are just following the footsteps of many other individuals who have done this much longer. If you deprive your learning opportunities from the constraints of the physical classrooms from the circumstances that come with it and the logistics from changing the higher education. What can you achieve? And the answer is almost everything changes. Students can go on their own speed as Salman Kahn has shown us. We can have adaptive technology from Carnegie Melon with a system that responds to students and gives them different paths. We can even go mobile if you can fill little gaps during the day with learning and we can redesign the basic learning. Udacity number 1 mission is to develop pedagogies necessary to bring higher education for the 21st century. We do huge number of data studies to understand how to improve student output and retention. We find for example that the lecture based lecture in physical classrooms might be perfect for physics classrooms.does not work online the average person who watches Youtube videos watches them for 9 seconds. And the average person watching a lecture often doesnt spend that much time focusing on lecture whether the lecture if 5 hours long. What works in online much better is students engagement. I want to quote Simon who said that the only way the students learn is by doing and thinking and the teachers role is facilitate learning and thinking. Students rarely learn listeningor they never learn by listening. The challenge for us is to take this new medium and really bring it to a mode where students do something and learn by doing. And if you look at the broad spectrum of online technology with what happens. It doesnt really take long time to point to video games. And most of us look down on video games. Weve also played them. I know there are people in this room who play angry birds. Some people do. Some people dont admit it. Angry birds is an wonderful learning environment because you get drawn in, you solve the physics problems but the big problem is that it stops at angry birdsif the angry birds was good enough to get into the masters students in physics. It would be an amazing experience and you could do this at scale. Udacity classes are organized as student exercises. We are among the MOOC providers that have the largest numbers of exercises. But its not the numbers that are important but the idea of curating entire curriculum around student exercises and student doings. Taking from the existing structures and the wisdom and the learning objectives but moving from teacher-centric perspective to student-centric where student learning is the center of attention. Thats actually been challenging for

us. Working with professors have been hard. Most professors like to profess. And in doing so, we actually rejected a number of classes. We also find that theres a huge interest in industry to get into higher education. As we know that higher education is moving at a slower pace compared to the industry moves. We have been funded by a whole bunch of corporations that make the classes with us and theres a number of classes launching soon. On topics to be not covered in academiaif you look at the way the technology turns over but it will be 5-10 years in computer science if you look at the way colleges turn overits much more difficult because (inaudible) would be to hiring new faculty member because if we tenure them they are gonna be with us for 30 years so the national turn over rate for colleges is about 30 years(?) Industries its like 5-10 years. So theres a dispatch between in how the world changes and how colleges are able to keep up. Therefore in computer scienceit would be hard to find courses that teach technologies that are useful today such as ios and all the wonderful things that they do. So the industries jumped in and funded us to build these classes. Better, faster, cheaper.I want to say few final words to better. We all know that between beginning and the end we get about 8 and 15%...some classes even have 3% completion rate. And in doing research, we investigated research, how can we entice students to stay in the MOOC and one thing that we find is that the answer to the MOOC is solution for everything is just wrong. The MOOC is not a device. Its a component of technology and its often belittled. Weve been trying to build internal communities of peers, cohorts, with mentors and automatic email system with various different modalities. And find that the most important way to bring up retention is to have you mentor us little (?). You would help as a MOOC professional not as just peers. I think theres a future in which MOOCs can blend the efficiency of online medium and personalization with the important role of instructor are going forward. Lets talk about better, faster, and cheaper. Lets talk about cheaper. Because it is the most controversial topic that most people are afraid of. First of all, I think cheaper is a good news. Having a more effective learning mechanism, reaching more people, having a better way to reach those people and a lower cost for producing them is a good news for everybody. It means we can extend our markets. We can continue the education. Education should be a lifelong endeavor not just 4-6 year and never. But also comes of course all kinds of concerns. Now, I think we need to do a bit of research in terms of cost structure for higher education and rethink the kind of cost structures are adequate for going forward. Because it more and more preclude people who are taking and making it hard for people to really build a career selling with enormous college debt. I think its good to look at the service we are providing at higher education and brining MOOC as a component for cost saving strategy. We would ask what services are necessary for students to achieve the learning goals. And none of us have the answers. But in the existing classes, the level of services are often not that great. If you are an instructorI talked to numerous instructors and you divide the time the communal time and the personal time you give back to the students in terms of advising and gradingyou can be lucky as a student for 3 credits class to get 3 hours of personal time. Many people laugh and many say I spend 10 min/student per class and the rest I give to my TAs. Charging $1,000-$4,000 for that to me is gonna be a question going forward. You are gonna think this is a negative piece of news because well our institution has become dependent on this other sources of revenue. But the truth is that this is not

a fixed market. I think theres an enormous market specifically in California. You can look at the college readiness. I hear that in CSU system, the majority students go to the remedial education because they are not college ready. So theres marketable high school students. You can look at the huge numbers of waitlists for students. Theres about 472,000 waitlists in community colleges in CA. You can look at the population after the graduation. I dont think our education needs to go away after we graduate. My water company, my cable companyI probably would be happy to serve them for the rest of my life (?) but not so my university. Why not? Because I think theres a huge market that are going to those people. And as we change the delivery methods. When movies were invented, the total spending per person for entertaining went down and the total spending per person for education can go up, not down if it provides meaningful lifelong education. And I think thats all about. Better, because we can invent new pedagogies. We are vastly better in many cases than existing classrooms. Faster, because we can make education much faster adept to the pace of the society. And cheaper, cheaper to students in so many ways.

Daphne Koller
-Coursera So, Sebastians talk share many elements with mine which is not surprising because we originated from the same eco-system and the same grand experiment that the Stanford started in the year of 2011, which opened up three graduate level classes in the computer science for everyone around the world for free. And within a matter of weeks, each one of those classes had registration of 100,000 students approximately or more as in the case of the artificial intelligence class that Sebastian offered. So that grand experiment just to give you the notion of scales involved...one of the other of those classes is the machine learning class thats taught by my colleague and the co-founder, Andrew Ng. Its one of the largest Stanford classes. When we made the slidewhen it was taught at Stanford it was taught every year to 400 students. Now that has gone up considerably I think partly because of the popularity of the online class. Now its 600 studentsthe one that was taught last time. When Andrew taught that class online, there were 100,000 students with that class. For Andrew to reach that same size audience, Andrew would have to teach at Stanford for 250 years. And that ability to reach at a cost of effectively 0 dollars marginal cost per student. An audience of this size I think is the tremendous opportunity that we started to deliver we as a community in 2012. So just tell you where we are today and I will tell you a little bit about some of the elements and some of the history of this project so currently we have a little over 2.2 million students who have enrolled in 212 courses from these 33 very fine institutions that we were very fortunate to work with. Its been striking I think how many institutions across the entire world now have embraced this new direction that we are seeing with...now online is no longer a thing that these fringe institutions do occasionally. This is something that every institution realizes that they have to have a clear strategy and you can see here. Institutions like Stanford, PrincetonCal Tech, Columbia, Vanderbilt, Brown, University of Washington, Georgia Tech and many others including 8 international institutions. And I can tell you that that number is only growingthat is the number

of institutions that reach out to us and to other organizations like us is growing every day. And I think this is a transformative moment in higher education. In 10 years higher education is not going to look anything like how it look like today or when I went to school. So what was the turning point this transformative moment and I think Sebastian eluded to that already is the realization that online education is not and should not be an attempt to replicate in digital mediumthat same experience that we are used to providing for students in face-to-face because if we do that we lose out a lot of what is special in face-to-face. The ability to look people in the eye and get the sense of what they are thinking and making connection with them.we lose on that but we dont gain enough if we dont leverage on the strength of online medium. So if you decide that you are going to compromise by moving on some aspects and gain on others you end up in a position where you have unique strengths in online medium. So specifically one of the things that one can do in this online medium is leverage a much deeper, more interactive interaction between the students and the material and weve already heard about that and Candice has been the leader in that act of learning framework and also its something that hasnt come up very much in the discussion is so far the interaction between the students and their peers. When you are sitting in a 300 person auditorium you are looking at the instructor and its totally not interactive processyou might as well be in the room by yourself looking at the instructor. Whereas when you are doingwhen you have this online community, you are pausing the video and you are sending a question to a friend, you can ask questions in the forum, other students respondit becomes a collaborative learning experience. And this is one of those things that really I think distinguishes this latest generation of MOOCs. We are leveraging social network in a new way and in many cases that makes sense because digital social network is relatively new phenomenon of the 21st century. I think that is really critical aspect of student experience and the success of the technology and why students get engaged and enjoy doing it and I will come back to these points in few minutes. So our work on this actually originated from StanfordI want to go back to the prehistory before the 2011 experiment. The work on this actually rose from two separate threads that Stanford has been engaged in for the last few years and you can say that I did little and Andrew did more of the other. The first of these is the open course style model which is lets take Stanford content and put it out there for the world to benefit from and thats what people typically associates with the MOOCs. But actually a lot of technology and pedagogy and the insight that went into the design of this new generation of MOOCs came from a lot of the Stanford experiment on flipped classroom teaching where the idea wasthis was very much supported by Stanford administration including President John Hennessy. Lets take lecturing out of the classroom. Lecturing is not a place where weits not the way we want to teach our students in this day and age. Its a waste of time for me to come in to a lecture with 200 people and give the same lecture that Ive been giving for 15 years telling the same jokes at the same timeits just not a great experience for meits not a great experience for the students. Why not, instead come and talk to the students in class? Have a dialogue and have them talk to each other and have them to active things in the classroom so that they engage with each other and with course material thats much deeper and more meaningful ways and very many people have talked about the benefits of that kind of active learning in the classroom where its people like the Nobel prize winning physicists Carl Wieman who talks about improvement that corresponds to doubling the performance on matric such as engagement, retention, but also the plain old test performance down the line. Or

people like Eric Mazur from Harvard all of whom have demonstrated the benefits of active learning in the classrooms as providing much better learning outcomes to students across every imaginable matric. Now for us this effort always has been the synthesis of these two different threads and I think one of the aspect was the bit of surprise for us is the extent to which these are synergistic with each other. So initially we noticed that it was very much like the same technology, the same infrastructure, the same pedagogy for these two different threads. The same content can be used for these two different kinds of threads. But what we didnt realize is the extent to which by doing both of these together you could actually appeal all constituencies at the university by offering the university and its faculty both an opportunity to reach a worldwide audience and a huge impact to the benefit of humanity. While at the same time also using that to push faculty away from their current local optimum of professing in the classroom and this is something that has been traditionally hard to do. I tried to flip classes at Stanford for three or four years and failed to convince my colleagues that this is a good idea because you tend to go to the colleagues that are best teachers. And they say why should I change the way I teach? Im already the best teaching ratings in my department I should just continue doing what Im doing already. And what we found is by appealing to their wish to have a much better impact then they are forced to prepare their course in new way. And they realize the benefit of that and then they flip the classroom. I was one of those faculty. I can tell you a little bit of my experience in that because it was an interesting, personal experiment. I flipped my course at Stanford for the first time about 5 years ago and like many Stanford classes I had always been televised because Stanford has this learning program for people outside of the university at local industry. And by doing that Stanford courses that were televised typically have attendance of about 30% of enrollment this is pretty typical. So if I have a class of 100 people, 30% will show up by week 3 and this is completely standard. When I flipped my classroom and I said Im basically going to do this completely optional, active learning exercises in the classrooms. They are gonna watch all the content out of classeverything that you need to pass the exam is going to be in those video modulescoming to class is entirely optional. Furthermore, coming to class is entirely televised because distance learning students still need access to the content thats the Stanford rules. So, my colleagues told me you got to be kidding meyou have the content thats optional and televisedyou are gonna have attendance of about 3 students in your classroom by the end of week 3. My attendance more than doubled. Peopleinstead of having 30% of my students come to class I had 70% of students come to class because they saw the benefit of being in their in the classroom in a way interacts meaningfully with me and meaningfully with their peers. So I think we are seeing that in a lot of the courses that are now coming uppretty much all the courses that are taught in Coursera are now used to flip a classroom in the corresponding institution. And we are seeing that as the beginning of the transformation to a much different way of teaching at all of these institutions. So having said that I want to speak little bit about the questions that we were asked to answer in the instructions that we were given. The first is the issue of cost reduction. Im going to start with cost reduction and Im gonna talk about quality which is even more interesting as a second point. So where do we see savings in cost reduction? Well, there are some obvious ones. Its clearly reduction of effort in course preparation. Does it really makes sense to have thousands of instructors

around the country each and every one in every single semester preparing a calculus 1 and English 1, Business 1, or time [can be] spent better on something else. A second place for cost savings is increasing class size. Now you say that most faculty and administrators say that oh no increasing class size thats going to lose our rating thats going to decrease our ranking in US, world news report. Well, I think that one of the things weve learned is that increasing class size is not necessarily a worse experience. So, one of the aspect of having a large online community is at large its actually good. So, in the recent weeks, we have drawn a correlation between the response time on the discussion forumhow long does it take for students to get their answerstheir questions get answered by their peers on the discussion forum? And you know what? Its not surprising but thats directly inversely correlated with the number of active students in the classroom. The larger the class, the more quickly you get your questions answered and higher the quality of that answer. So in some sense, larger is perhaps better as opposed to worse which we should explore much more deeply. A third aspect of cost reduction is of course increased capacity of course thats been discussed before. This is a particular problem in California. One thing that has been discussed only little bit although President Qaoumi referred to it in his comment is the notion of increased completion. One of the most costly aspect of university education to both institutions but even more so to individuals is the fact that people do not complete their degrees. And I think we can increase completion rates by having students come in better prepared, but having students come in knowing what they want to do because they can browse online different courses and different level of selectivity, different disciplines and find the match thats good for them The last thing I want to talk about is quality and I think that has come up in a number of cases so Im not going to talk about that very much. Id like to refer to a seminar paper by Benjamin Bloomalmost 30 years ago there was a paper Bloom wrote on two sigma problem and how do you get two standard deviations of performance by having personalized attention by human tutor given to students and how we can afford do that as a society. Well, one of the things that we can do is by giving students personalized attention combining direct interaction with computer with adaptive learning and direct interaction with their peers. In a way, it might not be as quite as good as attention from human tutor but maybe it can come close and thats a really exciting challenge for us to engage in. When you put in the blended learning or flip classroom model you give students a lot of ways to interact in full ways and get personalized attention from their faculty who now are not quite as busy doing content preparation as well as endless grading as they were before and can actually identify whats going on with particular students and trying help them along. So, the last point that I would like to make is that one of the nice aspects of model that we have is that its a collaboration across a large number of very high quality institutions and that collaborative relationship is what we see a lot in terms of faculty exchanging information with faculty across the disciplines and other institutional boundaries. And by doing that I think we can learn much faster how to do pedagogy in this new online environment. In ways in which it would be much more challengingI think that each of us have to figure out on our own.

Steve Klingler
-Western Governors University

(first part not recorded)developed a new model and they sponsored the foundation of that model and that became Western Governors University. We operate entirely based on tuition. We are private, non-profitwe are competency based institution, our tuition is very affordable by mission6,000 dollars a year and it hasnt changed for five years. And its our objectives to hold that current tuition level as long as we can. Thats important to our students because our tuition is based on time not the number of credit hoursstudents who accelerate have built in cost savings. The faster they go the more they apply themselves in school the more they can save. And constantly the less time they occupy in institution then. We just had a couple of great milestones achievements. We now have 40,000 full-time enrolled studentswe only have full-time students and we just graduated our 20,000 graduates so those are big accomplishments for us. We are growing at a sustained 30-35% year over year growth rate. We will probably back that down a little bit. The biggest challenge to growing up fast is scaling faculty. And we are really proud of how weve done so far. Interestingly about a tenth of our students as well as a 10th of our graduates are Californians so we have a presence in the state currently. We have the students and the faculty in all 50 states and this was mentioned earlier we have state charters in Indiana, Washington and Texas where through legislation or governors decree we operate as a member of state university system still without any state assistance, financial assistance. But we are regarded as a member of state charter education community. Our average student is olderthey are 37 years oldour average time to graduation is about 3 years 28 months just under 3 years actually. We serve nontraditional students primarily. They are older, they have prior college experience, its very common that they transfer credit from multiple institutions when they come to WGEU. They are working adults they have family responsibilities in addition to their school. Many of them are rural. Many are first generation college students. Many are low income. So we are mission-driven in keeping costs low and improving the quality and access of that education. Of course we are accredited nationally, regionally. We have the appropriate accreditation for teachers college, information security, nursing, health informatics so forth so thats really important for us to be a credible member both in academia and industry.

There were 3 questions posed to us. Im not going to speak specifically to those questions. I wont presume to know the answers for California but I will speak generally question and describe what works for us. And maybe there may be ideas in there that would work for you as well. First of all, we dont think its a question of online versus classroom. Several others have already spoken to this already today. Todays students live in the connected world. They interact differently, they research differentlythey are constantly connected the idea of online is a little silly when they are all carrying around a connected device 24/7. It is just a part of their existence

now. And most of online education today is still really classroom education delivered over the wirethats not idealDaphne spoke to that specifically in her remarks. And we will hear more about that Im sure. But the opportunity to exploit online mediumthe technology to provide more authentic learning experience and a more authentic assessment experience we think is really exciting. And it does drive the cost down. It does allow us to structure things differently. So we think there are actuallythis forum is exploring available by taking advantage of technology. For western governors university, competency based education works like this. First of all, our programs were originally specified the governors based on workforce needsunmet workforces in the states. And we diligently look at this continually. We dont offer degrees that dont have market demand for those degrees. The requirements for graduation are determined by advisory boards that are led by leaders from academia and industry. So we know that our students are prepared for the workforce. There are no electives. Students can come and burn their financial resources and their grants, their loans, their time on electives that dont add value. We prescribe every course in that degree program thats required to complete and where the electives would have been most institutions we have specified those through consultation with these industry leaders. So our students are very well prepared for the workforce. As a point of validation on that, we have a 97% employment with our nursing graduates, we have some great examples where they are highly sought after because they are prepared to work in there in the hospital. And they require a lot less time on boarding to learn skills that arent part of the normal nursing program. The competencies are well defined. They then get filtered into the course level where our courses are developed through teams of faculty and instructional designers and all the various roles weve talked about. I love the fact that we are at the upper right quadrant in Phils chartthats our model. Its selfpaced.it is a model that we design and deliver we think is the bestwhat we are really doing is finding efficiency because each professor isnt reinventing lectures and their lesson plans and developing assessments. Our professors, our facultywe call them mentorsthey get to do what they really love to do and that is to work oneon-one with students. We can scale that up really well because we already have identified and blueprinted and developed a high quality course and all the materials are there to support the self-paced learner and separately weve developed validated secure assessments that are rigorously developed by assessment specialists and they are carefully (?) so they cover all of the competencies we have identified. And we have evidence that a student knows and can do everything that a student with that course under transcript should have evidence of. So there are faculty now are very scalable. They dont have to do those administrative tasks that most of them dont enjoy. At least the faculty we attractthey want to focus on students and they want

to work one-on-one with students and they appreciate that we are supporting them by providing those things and which also provides for us consistency in standardization. When a students transcript course through WGEU, you dont have to wonder who is in their cohort in that term. Who was their instructor? Did they really learn it? Or were they just as good as others who didnt study in their course. We know that they have demonstrated mastery and we believe thats valuable. Competency based can be done in many waysone of the qualities that matters to us is that its self-paced, students can go at their own pacewe know that they come to us knowing different things and having a different background. If they have a mastery already they can prove it and move on if they dont they can engage in as long as they want until they demonstrate mastery. And it is a mastery model. Every course begins with pre-assessmentswe measure what a student already knows. We present back to themwe call it a coaching report. We identify for them what they have mastered and what they still need to learn and that results in a personalized study planindividualized guidance to the instructional materials that they need to engage which they can engage on their own and in our model that is supported by the most important thing which is our faculty. For us, the notion of attendance is based on a requirement that a student has a one-on-one interaction with a faculty every week. And if a student is doing well, they can relax to every other week. Last month, our 900 full-time faculty conducted more than a quarter million phone calls one-on-one with our 40,000 students. Thats fantastic. Thats more one-on-one interaction than I ever had in classrooms in college. So, we think thats fantastic. Last thing is that we do leverage the best quality instruction material wherever we can find it. We dont think we add value to higher education by developing another calculus course, another college algebra, another biology, history, or anything. If we can find it whether it would be commercial, open, the MOOCs, other institutions, we use Carnegie Mellon courseswe will embrace those and scaffold that with our faculty with our analytics and data with our assessments with our individualized study plantheres a common scaffolding for every course and theres a point where instruction material is the best material we can find wherever we can get it. And we think thats really efficient model and it really works well for us. So perhaps there are ideas in there that California can leverage and look forward to continuing the discussion today.

-Burck Smith
- CEO of the Straighterline Prior to starting Straighterline, I started an online tutoring company called Smartthinking that sells tutoring services to colleges that was 1990. Before that my background is in public policy and a lot of my thinking about higher education really

comesand my beliefs about some of the issues that California is facing and general education is facing drive back to the experiences in the public policy. In the mid-90s, the question that I was thinking about is very interested in technology and education what the impact of education would be with all these new technologies that were coming to the floor (?). So I looked at other industries and what happens to another when technology comes into another industry. Any provider of another industry who would adapt new technology whether thats computer or robot or wheel or whatever it iswe are doing it for productivity increase. They want to decrease the cost of delivery, increase the quality or hopefully both. As I was other providers start to respond that first providerthose productivity, improvements transfer to the consumers, students, tax payers and the industry improves its productivity in general. If you look at education, we make massive investments in technology from the past 15 or 20 years yet arguably the opposite has happened. We see the prices go up, arguably the quality go down yet we continue to make massive investments in the technology. So, what is it about higher education that prohibits or has kept these kinds of productivity improvements at bay? And the answer for me was the problems largely to regulatory which I will get to towards the end. First slide that you would have seenshould they have been upwas three points which I really want to use to set the context for the discussion. First its something that Jeff said earlier. 1/3 of all students transfer colleges before they graduate from college and that does not include students who might have brought credits from different places like high school, AP, DDST.there are all sorts of ways where students are bringing credits into the system. So what this means is that students see credits as fungible. Not only that, colleges allow credits to be fungible to a certain extent. So, these are not things that necessarily held by single institutionsthese are things that move around particularly among less elite colleges. Second, 1/3 of all students took an online course in 2009. Thats probably much higher now. 2/3 of colleges offered online classes in 2009. So the question is online as good as face-to-face or vice versa is really a non-question because colleges are offering online courses in large volumes to large volumes of students for equivalent academic credit. Lastly and this is the most important point for me. 93% of colleges charge same or more for online courses than they do for face-to-face courses despite the fact that online courses have almost none of the overhead as face-tofaces. There was an article I think in the SF Chronicle about the UC system prizing its online courses at 14 or 15 hundred dollars per course and not getting a whole lot of students. I think thats a great example of some of the problems in the market place. So hold this for secondlet me talk about Straighterline and I will come back. So what we do at Straighterline is general education courseswe have about fifty. Things like accounting 101, eon 101, college algebra, English comp, etc. We are not allowed to be accredited because to be accredited you must offer a complete degree program. You cannot simply offer courses. Instead we have all of our courses go towards ACE credit recommendation service. Weve added other good housekeeping seals (?) approval to demonstrate our courses are just like courses that any other college would offer. We also go to colleges themselves and save our courses look like the kinds of courses that you would already wanting credit for and transferwe do the same for ours. So we create an equivalency agreementsits simply an articulation of agreements entirely free to the college, removes all the ambiguities from the students about what will happen to the transfer from us to that college. Now for the student, we charge $99 a month and a one-time fee of roughly $49 per

course starting. There are some variances depending on the course. Students can pick either self-paced, tutor-supported courseswhich is what we started with. Or just recently we have allowed the ability for professors to teach on top of our courses and set their own prices and now students can choose either self-paced tutor-supported courses or professor lab courses with lots of information about reviews prior to enrollment. This price model does a couple of things. One at the price model we do, we are comparable, in fact, cheaper than most community colleges without having any taxpayers subsidiaries. Second, by pricing on subscription basis, its a really low rate to start. So right now when students start a college, they take 4-5 courses, they take few thousand dollars in debt, taxpayers kick in some money and 50% dont succeed. And maybe they dont succeed because they are not prepared but could also be they dont succeed because life gets in the waychild care, health care, jobwhatever it istheres all sorts of reasons people dont succeed in college. And in our model, if you start and after the first month (inaudible) you get a $140 so you are not stuck with the whole bill if you dont end up finishing the course. Weve seen a number of sour outcomes. Weve been doing this for long enough now and we can draw some conclusions about 62% of courses started or successfully completedsuccess is passing 70 or above that ranges between 60 and 65% sort of terms or semesters. Those vary by the courses though. We have partnerships with colleges. We can now see what students have done after going to the colleges. So they work for Western Governors for instancethey are close partner of ours. Over 90% of students are still enrolled after the first year, after coming from us to Western Governors. And this is similar to the kinds of results what we see from prior learning assessment studies and others. Within California, we have one CA partner college, Concord University in Irvine, and again its just a creation of the articulation of the agreement. Thetwo questions often come up assort of telling people about what we do. One is how can you be so cheap? What is it that youre doing or not doing that colleges are doing and maybe not doing that allows to be so much cheaper than they are? And the answer is NOTHING. We built our courses just as the college would. The difference is we dont price our courses as if they were face-to-face courses. We dont price our general education such that general education courses such that they subsidize not general educationswe dont price our online courses such that they subsidize face to face courses. Because higher education cost structure.theres a massive cross-over subsidizes and most collegesthey are online programsthey are communal programsthey are extension programsanything except the core on-ground undergraduate experience if often a profit center which is subsidizing the face-to-face experience. The second question we often get well why do colleges work with you at all? Why would they enter into your articulation program if you are offering equivalent courses that are 30-80% cheaper than they are? And the answer is many DONT. Many do not want to have cheaper alternatives to their own courses. But some do. And those that do are the ones who I see them as early adopters in the spacethey are the ones accustomed to working with students who bring lots of different credits and experiences to the process. They are also the ones who are working within a very competitive market place which is the adult learner market a place in the online market place. So, adult serving institutions are the ones that are early leaders, early adopters that are willing to embrace these models and work credit for different sources. So those tend to be what we have direct partnerships.

Again, Western Governors being a great example. We have about 30 colleges with whom we have direct partnership. About 300 colleges where students have told us their credits have been accepted. And there are probably more that are coming. What keeps our model from growing? In 2008, when we startedfrankly we were very controversial. The idea of non-college offering college courses much more cheaply than colleges do for path was a little more than many people can get their minds around. We were really controversial really until 2012 when Coursera and Udacity came along, blessed by elite colleges. Now we are not so radicalwe looked a little more prescient, which is greatso thank you all for that. That saved a lot of difficult conversation for me. But usand I venture to saythe MOOCs as wellI think there are two principles that underlie us and online learning in general. These are difficult principles for traditional higher education to deal with to say the regulatory structure of higher education to deal with. One is online course should be much cheaper than face-to-face courses. They simply do not have the overhead. Its just not true that they are more expensive than face-to-face courses. Second is that anyone can offer a college course. You dont have to be a college to offer a college course anymore. Theas long as you meet whatever the standards for college and thats the problem the standards are very ambiguousso this creates issues. Because colleges businesses, business models are built upon course delivery thats how they get paid. However, they can also keep other providers out by not awarding credit for their courses. So now theres a conflict of interest between provision of courses and credentialing courses. And it really has not been resolved. So this kind of conflict of interest is maintained accreditation structure which we all existwell I dont but many of you do. So higher education gets by my count about 20 billion dollars in taxpayers subsidies per year and thats a conservative estimateand that comes from direct state supportthat comes from federal grants for students, its pell grants, its tax credits, its tax saving plans, non-profit tax statusall these sorts of ways taxpayers support higher education. You cant access any of it unless you go to an accredited college. To be accredited you must offer full degree program as opposed to individual courses so a bundle of courses. The accreditation program is staffed and financed by colleges themselves. So its more of trade associations, not regulatory authority. The value in our input as opposed to outcomesthe business model should be more less the sameand colleges have (inaudible) to any degree they choose. So what this means is that its very difficult for new providers to come in and competition with different business models with the existing system which is maintained by massive taxpayers subsidies. Now the upside is that the difference in cost of delivery the price being charged got so dramatic that others are doing it anyway. So, us, Udacity, Coursera, others are moving they are more coming into the market everydaybut they are still outside of the accrediting structure. So, what can be done? These arent easy questions. Its going to be very difficult to both reduce prices to students and maintain revenue for colleges. If I was in the California policymakers shoes, I would look to provide some kind of equivalent treatment of all course providers, colleges or non-colleges alikeif you set a single either a standard tuition or set standards where everybody adheres to and you can start to have price competition which will benefit students and taxpayers. But these are NOT easy questions. So, I think thats why we are all here.

Chari Leader Kelley

-Vice President for LearningCounts.org (a program of CAEL Counselor Adult Experiential Learning; non-profit)

Weve been working on prior learning assessments way before it was cool for about 40 years. And what weve been able to do was leverage technology to make it more available to students. For me, just being in this room quite humbling because there are so many prestigious people here and Im a person who started colleges at age 27 so Im appealing to all those students out there that took different pathways to colleges and learningcounts.org helps with that. In fact, about 85% of our students today are post-traditional learnersthat means that they come to schools with burdens. Some of them have childrenmany of them have jobsthey all have worriesthey dont have the luxury of being able to be on a campus like this and focus solely on their learning. So I thought I would answer the questions quite easily for you in terms of prior learning assessments. How many of you are familiar with PLA? PLA is a process of recognizing learning that occurs outside of the university. Non-collegiate sponsored learningthis would include MOOCs by the wayit includes any form of OERit includes the workplace.the way we learn by doing is what several of us talked about. What PLA is there are several methodologies. One is to take a for-credit test, like club, DSST, associate college examsany of national exams that are worth credit. And that will help students to prove that they have the same knowledge compared to students who took the classroom. Another mechanism is to be able to produce ACE credit transcript like from Burkes area.straighterline. Or for military training or employer training that has been evaluated by ACE. And probably the lesser known and the most difficult form is portfolio assessment But this is the form that most individualized and personalized for the learner. So at learnercounts.org, what weve done from funding from Lumina(?) foundation and Kresge(?) and a number of other granters is weve established a portal for prior learning assessments. And at this online portals, students can click and talk to a person about PLA to determine whether they might be a good candidate and become informed about how to prove they can obtain college level learning during their lifetime. If you go to the website, you will see a brand new tool which just launched this week. Its called a PLA credit predictor. I call it a PLA wizard. And what it does is that it enables students to actually begin to look at the work experiences and where they may have college level learning thats been acquired over the years whether thats through volunteer work or technical skills and it begins to wine for the students how that might match up with the for-credit exams and how they would request a transcript or if they might be a good candidate for portfolio assessment. Now the portfolio is solely based on learning outcomes. And what the student has to do is they have to prove that they have obtained the same learning outcomes as students who take that course in a traditional manner. So if you think about Californians who have been in and out of the education the 30% or so who didnt get in to begin withthey are out there they are cumulating learning and if they are being toward leveraging that learning through degrees and prior learning assessments you have an immediate cost savings. And more importantly, what we have found through PLA is that students who do it see themselves in a totally different light. They

see themselves as a degree material. I can go to college, I can do this. And they are affirmed in that way because they cannot earn credit through portfolio unless they have explained how they learned it, when they learned it, how they applied itthe theoretical constructs behind what they have done and actually reflected upon the learning. So what we use are national pool of trained faculty assessors and they are organized by their subject expertise and we root portfolios to those assessors for evaluation. They all use a common rubric so that the decision is totally removed from the learning process so we canthat a student that I know who is really great and she gets credit for thatmy assessor is not going to know that. They make a decision that its either thumbs up or thumbs down and if the student earns credit, they can significantly save money on tuition and save time towards the degree. So how can I talk so authoritatively like this? We did a major study. In March 2010, we looked at over 62,000 student records from 48 colleges and universities across the country. Students who age at age 25 years old or older. And what we found is thiswhat we found is that students who had some form of PLA credit on their transcript that could be an AP course, etc. they were more likely to persist, more likely to take more courses, more likely to graduate than the students in the group that didnt have PLA credit on their transcripts. So I would ask why wouldnt CA consider PLA as a strategy? The important piece of this is that its tied to (not recorded) its the faculty who make determination if the credit is equivalent. So, you immediately have quality. You have control. CAEL can train faculty on how to do this. If you want to do your own thingof course I would love it. If learningcounts.org could be utilize by CA students and we DO have CA students. In terms of being able to leverage learning and being able to access all these different forms. Imagine a student who may be first generationmaybe isnt taking the traditional pathwaymight be able to begin to get on an attraction towards a degree and when you think about graduation rate possibly doubling for baccalaureate students theres a greater chance of them actually succeeding and completing the degree that they already dreamt of earning. Why not consider PLA? Its not newits been around for 40 years and its something that faculty can make decision on.

(Findings from the major study mentioned above) - 48 institution study of PLA and academic outcomes (funded by Lumina Foundation) 62,475 total adult students in sample (age 25 or older); PLA increases graduation rates and persistence; decreases time to degree completion

PLA students in this study had better graduate rates than non-PLA students; regardless of institutional size, level (2 year or 4 year) or control (private for profit, non profit, or public), regardless of the individual students academic ability to grade point average, regardless of the individual students age,

gender, or race/ethnicity, regardless of whether or not the individual student receives financial aid.

PLA students earned BA faster, compared with non-PLA students; between 2.5 and 10.1 months faster, depending upon the number of PLA credits earned PLA learners with AS degrees saved on average between 1.5 and 4.5 months in earning their degrees, compared to non-PLA students earning AS degrees.

Phillip Regier
-ASU Online There are an incredible number of good ideas that have been flooded and I find myself in agreement with all of them. I also find myself actually uncomfortable I thin with the overall subtext of the discussion and I will get to that in just a minute. First off, let me introduce myself. I do have two pieces of good news. First off, Im gonna be very brief. Secondly, I have no powerpoints. In relation to that technology I always think Im running an online campus with 75,000 studentsIm always amazed that we can do that oftentimes when I cant get a conference call in my officebecause technology doesnt work right. Let me briefly introduce myself and what we do at ASU online and explain discomfort with some of the subtext of the conference. First off, Im Phillip RegierIm from ASU online. Ive been a faculty member in the accounting department for 27 yearsin the state university. I was an undergraduate dean in the school of business and executive dean for many years now. About 3 and a half years ago, the provost came to me and she said I would like you to become a dean of ASU online in the extended campus. Those of you in academics know that extended educationthe dean of the extended education in the university is the elephant grave yard of university administrators. But I took the job because I was

interested in the aspiration of Michael Crow. As you know PhoenixArizona state is in Phoenix and we are therefore the epicenter of the for-profit education. Mike Crow got to Phoenix ten or twelve years ago and looked around and saw that the University of Phoenix with 350,000 students at that time and said its ridiculous that a real university doesnt have that type of reach and impact in access. We are working diligently at it. When I took over 3 years ago we had about 150 students and now we have about 75,000 fully online students where we haveweve gone from 4 degree programs to 60 degree programs. But the think I want to emphasize that we did not establish a different unit that runs on online education. So, if you get a history degree from ASU online, you get it from the history faculty in the college of liberal science. Engineering...go to the school of engineeringpsychology goes to the new college at the south campus, etc. etc. etc. We did not want to establish a different entitywe wanted to take advantage of the strengths of tremendous research national based institution to deliver the best quality education we possibly could to people because of who circumstances of their life are incapable of going to a face-to-face university. And thats what we begun to succeed in doing. Now, Im a business faculty member, so early onI thought back to Michael Porter who is a great business strategist and he said hey look in a competitive environment and higher education is a competitive environmentin order to differentiate yourself you need to develop strategies that differentiate yourself. So we established 4 overriding strategies in ASU online. Number 1, we were going to deliver excellent, high quality classes. We tell our instructional designers and faculty if the course they are building will not be as good as the ones we are delivering face-to-face students we will not build it and we will re-engineer the course. Number 2, and this is something we learned from for-profitswe learned a tremendous amount from the for-profits. We have to deliver excellent student services. This is something for-profits do tremendously well. Transfer of evaluation for for-profits occurs in 24 hours and in most public universities the transfer of evaluation occurs in 4 weeks. And online students have been conditioned by Amazon. They expect to be enrolled in tomorrow. So, excellent student services Third, we want to stay in the cutting-edge of technology innovation. That is not easy to do. But weve had terrific workshops that has introduced us to many many great start-up companiesmany large companies and we have partnership strategies and we partner like crazy with companies in the for-profit sector. Number 4 strategy is that we want to have high graduate rates. This isnt about brining students in and giving them some courses and letting them go in debt and not having them succeed in graduating from a university. So, we are about graduation. Most of our students are transfer students. The average age is 31 for ASU online students. The graduation rate is very high. Students who come in tend to complete. Our term-to-term retention rate is over 90% which is terrific for an online education. And we want to continue to work on that. And so at the end of the day what we did was we took all the powers of research institution and we tried to develop the very best online degree program that we can for students. And that includes not developing classes with great faculty members and instructional designers and people in the technology but also working with the researchers to understand how we can improve the courses going forward and understanding what learning is taking place in classes.

Now, what makes me a bit uncomfortable.I think thatits maybe because Ive been sitting here for 2 and a half hours that lookIm a true believer, right? I think that you can deliver learning outcomes in online education that are equivalent or better than what you can do in face-to-face in almost every domain. I think you can do it in the engineering by the way. Maybe not mechanical engineering but there might be some domains in the engineering that you can do. However, I also believe that we have to be very careful about thinking about what education is. And what an educated individual is in this country. And at the end of the day I dare sayoh I will just ask ithow many of you guys have a university degree? How many of you received your degree from an online university? Just a couple of people. Now that number is going to go up in the future. But the point I want to make is. The university degree is incredible experience and American universities are incredible institutions and we have to be very careful about thinking through what makes educated men and women going forward in this country.

Dr. Andreea M Serban

-Vice Chancellor of Education Services and Technology at Coast Community College District So, Im the first one speaking on behalf of the large community college segment in California. For that reason, Im going to make a few points about assistant (inaudible) perspective from my own institutional perspective and knowing that we are followed by the two exceptional people who will speak on behalf of the community college, Michelle and Berry. And also try to respond to the trick questions that we were given. And also I do like to thank the foundation for organizing this event for such a short notice. First of all, I think that its important to note that California community colleges have responded to embracing online education. And because my major is mathematics, I have to speak about numbers which I think are extremely exciting. For those who dont know, we serve about 2 million students and in terms of full-time equivalent students 1.1 million. And if we look at the progression of online, distance education for California community colleges90 to 93which is the earliest year I could find the data for our system. We only had 5600 full time equivalence students in distance educationwhich represented mere 0.7% all our full-time equivalence students. By 2005/2006(??), our system was generating 56,000 full-time equivalence students in distance education, representing 5% of all our full-time equivalence students. And 11/12 last academic year, we now have almost 114,000 full-time equivalence students in distance education, representing almost 11% all of our full time equivalence students. So, the point being thatliterally over the past 5 yearsour system has doubled in terms of embracing online education in our very large system. Some of our colleges actually have been particularly focused on delivering and developing models and instructional models and entrepreneurial models to deal with expansion of online education. Thats where I think my district is specialthe Cost Community College District. We have three colleges: Orange Coast college, Golden West College, and Coastline Community College enrolling about 42,000 students every semester. In some ways, we are similar to the others and some ways we are extremely unique. I will start with the unique part. One of our colleges, the Coastline Community College, is the only community college in California (112 community colleges in CA) which was created in 1973 is a college without borderswas created to be focused and primarily offered distance education and to this day remains focused primarily on offering online distance education. Of its 10,000 students, the majority are actually online students which is quite remarkable and different of the other 111 community

colleges in our system. Its also true that entrepreneurial activities at coastline. Several models have emerged and it was interesting that in a sense that ahead of their time. And now certain significant reforms and policies embraced through statewide effortsour now is to become required for all community colleges and I will refer just two examples as to why Coastline has been really a leader in distance online educational over its 40+/30 years existence. One relates to being able to deliver on a type of education that community colleges are allowed to offer to online education. Coastline has more than 3000 students active military worldwide who are involved through the contract education division of the college in online only courses. And this group of students which is really many ways has led to the need to look at how to deliver online education in an environment that really needs to deal with students all over the world with technical capabilities that are somewhat different than regular students. And as a result, because commercial learning management system that the three colleges were using at the time. Security reasons related to interfacing with military sectors around the world actually were not working with that particular commercial learning management system and that was one of the reasons that led Coastline community college to develop its own learning management system and really be at the forefront of integrating the best features of various learning management systems.open source and commercial into the system then was the platform for providing fully online education to military students all over the world. Also, because of these efforts and of the entrepreneurial aspect allowed to contract education component that California now allows for community colleges, Coastline has become a major content developer with publishers and colleges and the universities worldwide developing various digital content that is delivering and owning loyalties and copyrights on. So in that regard this particular activity is have been on the more unique side of what California community colleges provide. Where we are similar with many other community college districts around the state and where the challenges.one of the questions posed to us what are the challenges of expanding online education particularly as it relates to the community college sector. When we are looking at our 42,000 students, like many other community colleges, about 70% of incoming students are not college readythat is they dont perform in terms of English-readiness, math-readiness, and college-level. 65% of our students are part-time which is also similar across the California community colleges. And 62% of our 42,000 students are minority students so white students are the minority students throughout our three colleges. So this then aspect pose some particular challenges to the expansion of online educationand by that truly fully online education because I agree with the points made previously that there are components of online in all coursesblended education now is the norm and thats certainly a very good point. But for fully online then the issues of college-readiness and the issues of individuals who primarily are part-time and juggled many responsibilitiesmany of them are returning, older students. What we have found that its still a challenge is overall the student success ratecore student success rate in fully online courses remains 9-8% below what is called the traditional face-toface. And this hasnt changed much over the last 15+ years. That all three colleges have offered online education and if we look at the data system wide we are reflective of system-wide data so even though national studies are pointing to the effectiveness of the fully online for community college students. With the level of readiness, fully online courses are not as successful at least looking at the course success rates as the traditional face-to-face students. College-readinesswhen we look at 77% of our students not being college ready but question being (not recorded)education fully online is yet to become to be developed in a way to be effective for this level of students. The issues relate to articulation and transferability. Our articulation officers from our colleges continue to tell us that in terms of the

acceptance of particularly the lab science coursesif they were to be fully online their acceptability for transfer to some four year institution including public 4-year institutions in our own statethat would be a challenge. So that provides frankly not much incentive for lab based science courses at our community colleges to be developed fully online even though we know that there are extraordinarily effective examples around the country where fully online biology courses, lab courses are online taking advantage of the advances of technology. Last but not least relates to the issue of regulations as previously mentionedas one who has worked in community colleges for a while and working in higher education systems in the country I would venture to submit the California community college as the most regulated system of higher education in the nation. And its much of the regulation in this sort of remaining vintage regulation that comes from the K-12 education. Until the regulations that many of them are really not reflective of what modern higher education is about (inaudible) I think will continued to be hampered in expanding the kind of innovations that other states have done successfully to date.

Ray Cross
-University of Wisconsin Colleges I agree with Phillip Regierobviously 37 year old mother with 3 kids you cant give your kids to in-laws and say Im gonna go back to school. Those work. So how do we serve that 37 year old mother of 3 that traditional or non-traditional students? Thats what the University of Wisconsin is trying to dofind a way to serve a target adult market and to do it in a fashion that makes sense.now we believe that the currency of the future of higher education is and will shift from credits to competencies. So if you think about that for a minutein some point in time businesses are going to create taxonomy of competencies they want in an employee. And can colleges align with that in a way that makes sense to both maintain that liberal arts fundamental education as well as serve the needs of the business in industry. The self-paced format is important because we want students to start and stop as they please but with some level of discipline. How do we do that? What makes our approach unique or different? Well, lets talk about the title for a moment. We purposely called it a flexible option for 2 principle reasons. The first is that not only is it self-paced but it creates flexibility but we want learners to digest at their pace in a model that acquires them to learning like they do other things in small pieces or in large amounts. Western Governors Model is an all-you-can-learn model but a lot of learners also need to be able to purchase small amounts. Instead of buying a whole course, can they secure a series of competencies or a modulea piece of that in a fashion that allows them to complete it more quickly. Now this creates a new problem, it would be much easier as Steve has suggested to simply treat all our students as full-time students. But adult learners also need to bewe need to accommodate part-time students. Now, we initially focused our new degreesour competency based flexible degrees on the needs of Wisconsin. So our first flex option degrees we will be starting in next fall, principally deal with the areas of healthcare in the areas of information technology and in business although we dont have a fuller array of business programs available yet. But these are existing University Wisconsin degreesthey are not part of the separate entitythey are our existing degrees with faculty from those degrees working with our instructional designers in order to produce a competency based self-paced model that is attractive and serves adult learner. A couple of things I want to bring up that are principle components that a way answers some of the questions we were asked. Lets start with the first one. There are some principle components that I think are important in how we approach this.

First of all, theres cost issuecan we save students money? Yes. If I dont care how they learned it, where they learned it, how they are assessing whether they know what they can do and they can prove it. Then online position to leverage free MOOCs or other learning in this particular erathis is an era of ubiquitous information. What we need to be able to do is find ways to assess whether they know it and can do it can prove it in a mastery level. We also need to help curate information in a way that allows them to align and go after the kind of learning experiences, the content, and the instructional materials so thats aligned with the competencies we are seeking. So its curated in that if you want to prepare for this competency or this series of competencies please roll over here you can take this and that, etc. We need to curate that information to assess whats good and whats appropriate. Another concern we face within the university of Wisconsin is the fear that we are going to cannibalize existing programs. Oh, if this is much cheaper or if this is in a fashion that will allow students not to come to our campusyou will cannibalize our programs. We disagree. We actually think that it will enhance their program in a way that it will allow it to grow. But I think its very simple to look at this from a residential campussome would argue that what will the future of higher education look like? Im gonna ask it in a different way. What will the future of residential higher education look like? I think its gonna be a flipped classroom that will allow particularly freshmen and sophomore students to experience something entirely different. I can envision faculty in a flipped model, meeting with students much like they do in Oxford and in Cambridge and historically that model of how you serve students in a true learning environment positions the faculty member not as a lecturer but as a tutor/counselor/advisor and lifelong colleague. And thats the kind of faculty I want to be and I think most our other faculty do as well. Our couple final points and I will close. What are some of the final principles that we are finally working on and hoping to develop that are important to us. There are four key components in our program one is we want to establish competency with clarity and I think the Western Governors has done an excellent job on that. We want to design assessment tools that effectively measure actively what we said we want to establish competency levels and adding mastery level. We want to curate information that will make it available so our students know where to best look at. And look for that kind of content and finally the fourth point the most of retention rates and success rates in these programs is rather embarrassing and if I were a legislator I would say gee how could that be more cost effective when you have so many students failing and not moving through this program? We need a wraparound advisor model that is so intrusive to students.and we help them whether they want it or not.help them succeed and thats the piece that I think is missing in a lot of this. And finally I think we are public institution, University of Wisconsin is proud to be a public university in the united states so are some of my colleagues here. But if we dont understand that in our effort to accommodate reduction in state aid that we concurrently raise tuition if we dont change that we should no longer be able to call ourselves a public university.

CA University Policymakers Perspective Keith Williams -Interim Director UC Online

There has been a perception that UCs havent been doing much with online educationboth true and falsea lot of workthere are courses that are fully online as well as hybrids; offer 2500 courses right now most of which are through

extensionscontinued learning phase of UCswe are starting to make effort at lower and upper division No one problem and no one solutionand that fits well this toolots of great ideas herenot all of them will work for us Quality has to be one of the first things that we are concerned about; evaluations of coursestrying to use that to inform what we are doingfaculty are really busythey are not only transmitters of knowledge but also creators of knowledge; research is important aspect and they try to bring that into the classrooms in order to enrich the students learning. Flexibility..what works on one discipline may not work in other discipline Hope that this will result in a cost savingcost saving and quality have to be tied together when we look for ways to move forward

Barry Russell -California Community Colleges Vice Chancellor of Academic Affairs

Out of the 2.6 million students that are community college students across the state; 1 in 4 community college students are in CA; about 85% of the 2.6 million students taken an online course; out of average 15 units they take about 1/3 of those in online; 2/3 in regular classes; there is a HUGE book of regulations how to do that and how not to do that Online education in CA community colleges for a long timewe can get better at itthings that providers are discovering in your researchthose are things that we can learn fromits not either orits something evolving and we feel like we need to be included in that evolutionso for instance we have St. Decento college (part of the Gates grant process this last year) they are developing a case study

Lt. Governor Gavin Newsom -UC Regent, CSU Trustee

Do we believe that we can transform the large CA public higher education based on these technologies? What are the capabilities we have and based on that what are the biggest challenges we have? Young children are digital era; you cant educate my daughter the way I got educated; she cannot exist in a world where its one-way broadcasting model education Redesigning something thats different to fit the world we live in; not from the cost perspectivesI dont like that approachI want to focus on the issue of quality and access

John Welty -Chair CSU Online

Teaching, learning fundamental mission Over the last several monthshow to take advantage of online education in order to improve access for students in California; cost of higher education4 years agotoday vs. 4 years ago the total cost has gone down over 1200 dollars per student (thats after taking into account the fee increases driven by the disinvestment in higher education by the state)

We were forced to make the changes quickly in short period of time in order to serve the students but the work weve been engaged in led us to have discussion; How can we improve access to online education for students who can benefit from online education and also increase access to students who have difficulty getting into certain courses Cal State Online which is NOT highly innovatedlargely as a result that was modeled after some degree of ASU online U Mass Online and some others that have been successful in other systems; launching website; degree programs that are already developed by their faculties on campuses that are already demonstrated to be highly effective; our goal is to expand access to those degree programsfocusing on Cal state reconnect which will focus on those who were enrolled in the program and didnt complete the degreewill have this opportunity to do thatwe partnered with Pierson E College; helping with infrastructureready to go and launchsuccessful launch of better serving students in CA

Questions/Answers
John Welty: I guess one question would be..I think we are at the front end of the tremendous changeand it would be interesting as particularly with regard to the MOOC area where based upon what we know today where they think its gonna go in the next few months or a year. Is it the analytics thats gonna really make the difference? What is it thats going to be making a huge difference? Sebastian Thrun: Futures are of course uncertain. The opportunities whether it take a year or more to redefine learning experience and you see this in a medium in a way its completely different from what weve seen before. A lot of what we have todaysmall classesits all based on one basic assumption (inaudible) and a lot of online learning up to this point has tried to replicate this trying to use this digital media using the same metaphors that are tied to this assumption and the only thing thats changing is that we are taking that assumption away.it drives me crazy that most high school children dont want to learnyou are born to learnif you have school systemssystems where you turn off students to learn math for examplejust by forcing on them specific mechanisms.specific time schedulespecific records and a lot of embarrassment and a set of failures(?)I think all of these things people will overcome... Daphne Koller: I would like to basically repeat what Sebastian said which is that I think the key aspect of what we are gonna see is the brand new pedagogy. I think thats going to be a collaborative effort. Thats going to involve instructors from different kinds of institutions thats going to involve all the instructional designers and I think we are going to learn a lot about how to teach in this brand new environment. I also think thatyou mentioned the role of analytics and I think the role of analytics is actually twofold. The first is the kind of analytics that Candices been doing which is very rapid psychoanalytics where students own actions immediately drive them towards one path or another thereby meeting significantly personalization of the curriculum and I think thats the one cycle that we are gonna see as a consequence of the ability of onlinethe other one which comes to the point of pedagogyis the fact that we can now do the kind of rapid revolution in education that the internet industry has seen in the last few years. So whereas the kind of rate of evolution in education typically has been on a five to ten year cycle by the time you write a grant proposal, get it approved by NSF, construct instruction involving 20 people which is not sufficient doesnt have sufficient statistical power to reach any kind of meaningful conclusion. And then five years later maybe youve learned something and nobody reads it yes that too. But now some of you may be familiar with AB testing. Who knows AB testing? Okay, about half . Its a quiz, its a pop quiz. AB testing is what company

like Google does in every single day of the year. If you log on Google or any other internet site, 10% of users are directed to subjected to different experience than the other 90% and that experience is measured by detailed analytics of the measure that company cares about and within a matter of days if the user experience is more productive than the 10% the entire experience shifts for everyone else which means that this website evolves in a matter of days or weeks rather than years and its something that now we can do in education in a way that weve never done before. We can now evaluate whether for example does it help for students to see the instructors faces as they are being taught. Thats a question which Ive seen in papers written saying that the answer is yes. Ive seen papers that says the answer is no. Each of them are based on the populations about 10. And now we can actually measure that and have an experience about 20,000 that asks exactly that question and sees which one has better retention, better attention, better performance on whatever outcomes we care about. So, I think the data is important both in short individual personalized for individual students but also for metalevel in developing new pedagogy. Keith Williams As I mentioned that UCs courses start with the faculty and what they regularly do in in-person courses but now for online courses they dont spend the time and effort that everybody else does looking at in this room whats available there. So, getting them familiar with whats there can be a challenge. We can have various fields of people we can help them but sometimes they start out with resistance thats one of other place where having information and data that will help and convince them in their research faculty they will listen things that have been done independently and done well. But I would put to the group over there what are the ways we can provide inside to the faculty that will get them more enthused about the opportunities about online education that will help us facilitate what we are trying to do? Daphne Koller: I can get started with that so we have set up a close site for instructors teaching online that allows them to share experiences in a protected way so that if they did something really well they can share that with their colleagues and in a gallery that says look this is what the final project from my own classand then theres also going to be ways for them to share what didnt work and thats how we all hope to be able to educate everyone and get them enthusiastic so thats one short answer.

Burck Smith So we have a little different perspectiveso first of all its early. But we are putting a different set of variables in a sense that faculty can come and teach on top of our courses can set their own prices and can add to what we have developed already as a templates. They can add more content, add more assignment, add more support functions, add a different angle to teaching whatever their special sauce is they want to bring to the course they can do that and set their own prices and difference between their price and ours is what they get. Also, if they use social media tools or other marketing channels to track students that can also be paid for tracking students to courses for themselves. So the difference is that we are unlike UC I suppose where we dont have to convince our recalcitrant faculty to get online. We are creating incentives for them to come online if they choose to and thats a very different starting point. Phillip Regier Id say because weve kind of been pounding faculty on education and innovation for the past decade I no longer have to worry about 70-75% of the faculty they are on board where we do have difficulty on occasion is getting faculty to try more radical innovations like adaptive learning like moving on when ready become host of other things we are trying to work on. But its really one of perspective than in construct not one willingness or intent. So they are willing

they are ready to go and I think they do need kind of clear understanding and direction of why we want to try to do what you want to do.

Dr. Andreea M Serban I would like to comment from the community college perspective as well that willingness and enthusiasm of faculty not the issue we have a tremendous number of faculty in many disciplines who have already developed, taught online courses for quite a while. I think where our challenge lies is related to resources for ongoing training and providing the support for our faculty to be able to constantly have that ability to have the infrastructure that was mentionedinstructional designers the kind of models that for example we have project managers and instructional designers in the whole thing that actually helps faculty member. We dont have the resources in California community college system that private providers and even our colleagues in the CSU and UC system haveso and that 46 dollar per unit frankly is we are as cheap as it comes I dont think the issue are going back to the points of even you are uncomfortable that. So I think our faculty is looking at whats best for our students thats what is really driving. What kind of online courses makes sense for people and recognizing that frankly that online is not a panacea and its not an answer for everything but one option that we will enhance educational experiences and success of some of our students.
John Welty: Can I clarify one thing? Im not really implying that our faculty are not enthusiastic because a lot of them are. One of the issues that I see is that theres such a dizzying array of different things out there that are all new and innovating and its hard to see how to focus in one thing. Sebastian Thrun: So I think one of the important (inaudible) is does it work? The answer is we dont know. And theres a danger of treating anything new as a shiny object that we jump for and call it silver bullets and just jump onto it. So I want to caution us while at the same time having enthusiastic about the bigger vision that it requires basic research and it requires sustained innovation. Sustained innovation should be nothing new in research to the UCs or all the CSUs in the room but its sometimes shocking how little we apply to ourselves when it comes to online learning. We need to dive ininternational science foundation recently had a talk that was about (inaudible) science foundation who felt that they were coming in too lateone year into the develop. Hey look you guys gotta crank up the effects of different education because we havent found the answer quite yet. Before we all jump into it I think you have to do more research.

Phillip Regier I think the idea that faculty are unwilling to involved in anything that involves education technology is a red hair. I think they are willing to engage. I think the question is how best to channel the engagement and they are not used to working with others and they are not used to thinking that teaching is a team sport and by the way your average biology professor knows nothing about learningI mean nothing about learning science at all and so you have to go through a process of educating and working and developing teams that work. As I said its not a question of enthusiastic engagement with educational technology.

Moderated Faculty Panel with Questions to Providers

Michelle Pilati
-President, Community College Academic Senate

I teach onlineif anyone is paying instructional designer that who dont need and you want to give them something to do, send them to me. I would be happy to use them. We would love to do courses that would rival what you are doing in coursera. We dont have the funds to do that. We would love to do that to take our courses to the next level but we have individual faculty members building things on their own without opportunities to work with others which would be really good to sit down and really talk things. I know that one of the best things that I liked about my courses are things that I learned from other peoplehaving those kinds of opportunities. I think there are things that are happening that we can learn. I think we can agree to have lots of ways for students to learn and consider that learning way in an academic setting. But at the same time, we also have to think about what does it mean to getting a degree? Does getting a degree just cumulating 120 units in under $10,000? Or is it more meaningful experience that has different components to it that leads it to better than sum of the whole at the end. We do have efforts in our system to provide support but there are often theres just time to do it. I would have to take a sabbatical and revamp all my courses and find everything out there to make them as best as they could possibly be.

Diana Wright Guerin

-President, CSU Academic Senate I would just agree with everything Michelle had said. We have faculty that are doing very innovative things. Very excited to you them. For us it is recruiting and maintaining faculty ranks full, so that we have faculty who have time to do it. We have focused on the teaching part of the university mission, not so much the research and service part and I think thats important for us to remember that Universities do much more than just teaching courseswe are all aware of that. And other thing, as a parent, someone who will look to these students to lead us into the future, I do have personal concerns. These are mine alone. Students do learn important skills when they navigate a system and we have helicopter parents who took care of their students business on campus. And part of what our higher education does it gives students to navigate the challenges that confront them. So its important for us to remember that theres a lot that goes on outside of the classroom in the university experience that can tribute to students development. We need to make sure that we dont lose that by attract students and make learning fun that students graduate with an understanding that they have a responsibility to be at a certain place at a certain time and carry out their duties and some duties are not fun. Im at all favor at making students meaning engaging environment for studentswe understand the importance of active learning and we incorporate that into our courses even if they are face-to-face.

Bob Samuels
-President, University Council AFT I have very different perspectives on many different things. So, I have never been at a faculty meeting where faculty got up where they said we need to change, we need to move online soin no way I think this is faculty-driven. And that may be a good thing or a bad thing but do not think its faculty-driven. Maybe theres a way to make it faculty-centered, I dont know. But when we look at the university of California the biggest cost-drivers are not undergraduate instruction. We had about a billion dollars of state cuts, we made classes bigger, we have non-tenured faculty, we are using

course software, course management programswe are driving down the cost of instruction. Thats not where the costs are. If you want to cut the costs, you would have to look at research, sponsored research, administration, sorry, staff, you would have to look at athletics, you have to look at these things that are not core missions. Those are the cost-drivers. So, Im still strugglingwhat is this really about? We all agreeif you want to do high quality education, its not gonna be a low costits gonna cost even if its online. If you are gonna have interaction, high quality materialsit takes more time to develop. I teach digital media, it takes more time to develop digital media course, just getting the course together and then the amount of time you have to spend just with technology and dealing with the studentsits incredibly labor intensive. So, I dont think its gonna save faculty timeI dont think its gonna save costand I think its really obscuring the major problemthat we hadthis reduction of funding for universities. The costs are going up and we have to find ways to drive down the cost but also defending the quality of education. Because what I hear is a lot of comparing of online courses to our worst courseswell those courses are bad because we are forced to do this by state cuts and other expenses. So, we need to define what quality education is because theres no consensus. What is good quality? I dont think we agree on what quality is or how to determine that. And I think no one really knows how much it costs to teach an undergraduate class. Its getting cheaper and cheaper and I dont think these online courses are going to be less expensive. So Im wondering why we are going to pour money into this. Finally I would say, theres a tremendous need for these types of coursesI dont think they are just not for research universitiesI think they are great for credentialing, I dont think everyone should go to college, I think there should be alternative paths that are recognized and supported and I think online education can play a gigantic role in that I just dont know why I feel like we are being invaded by these outside forces.

Lillian Taiz
-President, California Faculty Association
I agree very much with Bob. I will go in a totally different direction. One of the issues that I havent heard discussed here and I talked a little bit with Sebastian about thismany students in CSU for the practical issuesarent wired. You may be rightsomeday we might do this on our cell phonesbut right now Cal state students dont all have computers, dont all have iPads, dont all have DSL lines, dont all have the resources that they need to engageso whats the alternative of instead of going to collegedoing online courses just adds another dynamic to their lives on top of jobs, kids, so on and so forth. For me part of the access challenge is how much access does someone have who doesnt have access to all of the equipment and connectivity that they need. Some online courses are happening at the general fund side of the university where students pay their tuition and part of their tuition they can take online course. Many of them are being offered through extended education where the tuition is higher. Thats not an expansion of access. Thats restriction of access. Not all financial works in them. I completely agree with anyone whos suggesting that subjecting this new direction to rigorous researchwe owe it to the students because a whole generation will be harmed if we dont. So I really do hope that we are as enthusiastic for holding ourselves accountable for what we are doing with them as we are moving this impulse forward. But I think right now as we are thinkingwe need to be carefulnot only to the students whether or not they have equipment whether they have the very practical stuff for them to engage in online educationbut you can go onI mean universities are going to have to refresh their equipment, hire new folks in IT, there are a lot of cost

there that we are not talking about that really we have to address. If part of this impulse here is to save money and maybe it is and maybe it isntI think the very worst model of teaching is as Ive heard it a sage on the stagewe get 500 kids in class we prop someone in front and we call it teachingnone of us think thats a good model. Those of us and Ive also heard today that faculty doesnt want to share information and dont want to learn how to teach better and my experience in CSU for 20 years couldn t be more different than that. We are constantly trying to figure out what works, what works better, what works better than that. What works with this group but not with this group. So, let me just say that dont need to be doing online teaching to want to be a better teacher.

Discussion with Providers

Daphne Koller: I wanted to speak to the cost and quality issue that were raised. So first of all, we are all in the agreement that the quality should be the highest priority that is we should have high quality online education and we shouldnt be cutting cost at the sacrifice of quality. And I also agree that producing a good, high quality online class is an expensive endeavor. So our cost estimates are 25 to 50 thousand dollars for one of our courses and that doesnt include faculty timethat includes mostly TA support. We found that videographer time is not an huge expense because its a commodity now and you can get it for very economically but TA support for content support is a fairly significant expense which is why it costs 25-50 thousand dollars depending on how much support is required. But I think the important to keep in mind is that cost gets amortized over a much much larger number of students both across iterations and within an iteration and the marginal cost for students now comes down to under a $1 or very close to $0. And I think what we discovered is that even with those numbers its possible to give students very high quality education and thats because some of the things that we are the most bottleneck for a traditional education system are something that we find solutions for using technology whether its the use of the auto-graded assignments that avoid the need for inflicting on instructors the huge amount of grading but at the same time the onerous chore of answering questions even from 300 students that now gest relegated to peer-to-peer interaction where students are very effective at providing each other useful, meaningful information in response to their questions. Students often make better teachers for other students because they still remember what it s like confused about a particular point where those necessarily not have that. WE find that community is central part of the effort in a way that doesnt impose on faculty time and weve seen that peer-to-peer question answering as well as peers grade each other s work. Its not only a significant strategy but also tremendously valuable for students. We keep hearing from students who were involved in peer-grading aspects of their courses they learn even more from grading each others work and seeing different students approach the same problems as they did from the work by doing themselves. So, yes the cost is high but once you amortize them the they are not that high after all. Bob Samuels You are a Stanford faculty member, right? So if you are teaching a course, who s paying for the class? Whos paying for the bills in the classroom? Who pays for the technology in the classrooms? Who pays the utilities? Who pays for everything that s associated with that course? Even if you sell it to someone else or you make it online for freewhos actually paying? Whos paying for your total salary? I dont see how marginal cost goes down so much because there are all these long-term costs. Daphne Koller They arebut you divide the cost that you would pay for the traditional face-to-face class not over a 100 students you are teaching but over a 10,000 students you are teaching. Yes, so the fixed costs are there. The question is whats the denominator?

Sebastian Thrun At this point, Im not as quite bullish as my colleague Daphne in a sense that 50,000 dollars you can educate 30,000 students. The dropout rates are staggering and only highly motivated students join in. There could be many reasonsone would be that they dont pay moneyeasy to sign up and easy to drop out. Theres no credit at the end. Our courses are significantly more expensive to produce. We put a lot of work in these classes.People matter. Instructors matter. Its not just the computer system that matters. We fell in this trap thinking wow we have 160,000 students it was quite amazingI think we slice this specific slice of students who could do it and forgot the other 99% that cant do this. So our current ambition is actually going backwards. I think its still gonna be cheaper than the existing mode, but again it s about the quality of education. So I cant see how we can educate people under single buck. Daphne Koller Let me clarify, I dont think we can educate students at the current system under single buck. The access that we provide amortize over a very large number of people open the doors to increasing for example the enrollments or increasing revenue by having students in the system rather than sending them to other countries to study. No, but I dont think we can educate students in our system for a dollar. Phil Hill Is it possible to partnership with a lot of the online providershave we seen area where it could provide supporting faculty who want to get in online who want to support what they need.

Phillip Regier
I will address that if you dont mind because we have a lot of for-profit partners and certainly the biggest one we use is e-college at Pierson. You have to think about whats the core of the university. Thats what we would view the most vital in terms of building online classes. Almost everything else is context: marketing, student recruitment, call center activity, website activity, etc. That s not the core what university does. One thing we talked about is the importance of courses but there s also rethink between the courses thats incredibly important and in any of those areasfor-profits can be extremely helpful and extremely beneficial. I want to say one other thing. I think theres stillthat online class is not as good as a face-toface class and Im gonna argue thats not in reality from 2 aspects. First off , my personal experience, it s obviously an anecdote. But I have worked with over 200 faculty over the past three years to develop their classes and I got a deal with every one of them. I say if what you are doing at the end of the day is not as good as the face-to-face class then come to me and we will reengineer and we will change it. I have not had one faculty taking me up in three years. Secondly, theres a lot of research about online education vs. face to face educationthe department of education did a great meta-analysis a couple of years ago. It was revised in September 2010 and based on the research at that time online education dominated what happened in the face-to-face classrooms. Well, you might say, that s not true at UCLA, Berkeley or Stanford and thats probably true. But, thats not the environment in which 98% of the students are attending. And by the way, that study was done in the time when the technology wasnt very good. The technologies are getting better and better. Its not only because we lack creativity that we cant figure out how online learning and hybrid learning is what everything will be learned in 10 or 15 years from now. Michelle Pilati Its important to recognize the population of students who choose to go into a MOOC and complete a MOOC is NOT our population. When I m teaching a class of 40-50 students, Im not in there watching the discussion. They come up with the wrong answer and convince themselves over and over that the wrong answer is the right answer. Thats my population of students and that s not always the case but sometimes thats the case and I have to keep a close eye and I have to be there. And I m there. And its the presence that communication in the community college we value. Because we dont have big lecture classes with TAs to do our grading and all that. We are teaching. We are there. We are with our students. So, the idea of bigger classes doesnt make sense. What makes sense is how to get students through the first time. Then weve got a long way toward addressing issues. The more you try to cut the cost of education and pull back out support for studentsthe less likely they are gonna get through the first time. We need to figure out how to get through students in all through classes the first time.

Phill Hill Does that get at the whole readiness and the Michelle Pilati Absolutely and yea not having students going into the courses where they are not ready to take yet because they cant get into the classes that they are ready to take. Diana Wright Guerin Im curious in terms of five years outyou knowone choice for econ 101, one choice for psych 101and that 50,000 students are taking those courses and how do you see your the services you provide fit into the entire higher education landscape? Not only in California?

Burck Smith
For me it comes from the assumption that Im not sure whether its correctwhich is that the funding stream stays as is. The price points stay as they are. Our changing of points of higher ed. I have an article coming up in change magazines next month and many colleges should get on the online learning not because online learning is bad or wrong but because the prices are plummeting zero. The colleges are getting onto the online with the assumption that they can price the online courses just as the same as their face-to-face classestheres now ample evidence that comparable courses can be offered at much lower prices only matter of time that policy makers and students will sayhey why is my publicly subsidized college 10 times the price for a comparable online course? Whats going on? So these things are coming and I dont think that we can keep the same model with the same kind of pricing college is only intermediary (?) in that pathway. Whether you like it or notits changing because of the consumers and policy makers. One thing that always gets me when I hear the word quality people talk about it all the time. Its never defined. Quality means different to different people. For some people quality is something that they can get quicly, affordably in their home. We use quality and use it to attack different models without defining it particularly well. Ray Cross Just a couple of thingsif state budget is doing what they are doing as Jeff aligned in his comments this morning. By 2022, Colorado will get zero state funding and by 2059 as well. The problem from the public university perspectiveI believe deeply in quality and the quality is defined by something fit for use. I think its really important for us to say we need to lock arms and go over this together as faculty and administrator. If we dontwhat will be the future of public higher education would look like? That scares me. Now, as costs go down, one would argue that is this a way to use analytics effectively and part of what Daphne was talking about earlier is that the response time for research in this arena has to be a whole lot faster than a lot of other things we do. We need to make adjustments and part of convincing faculty and working with them and engaging them in this process is the honesty we are seeing from Sebastian. Hey this isnt workingthis is a problemthat candidness go a long way and engaging them to help all us to look at this. So the comment that this is not faculty driven, it may simply be reality-driven. How do we get to the point where we are together and not looking at it from the adversary perspective and I think thats a critical point. Andreea M Serban I just wanted to make 2 comments. This week on Monday the president of academic center of one of our colleges, our Orange Coast College, who is a brilliant professor, she enrolled in one of the MOOCs classes and her comment was it was an experiment. Her comment was that the peer grading system is actually leading to lowering the standards and since theres no credit given in that it doesnt really matter. Im looking at this from the perspective of while one of the missions of the California community colleges is to actually prepare workforce and economic development. I think we should also think what the employers in the US need and there has been a lot of discussion actually there are many fields where higher education is falling behind in producing preparing individual in other areas particularly career technical education types of areas. I dont think we have fully addressed that in this discussion. The issue is one of the employer driven and from an economic development perspective in our case CA but also looking from the national perspective.

Daphne Koller I wanted to respond to the comment about whether there will be single econ course and my answer is a clear no. Theres multiple textbooks in econ and in biology and even in more advanced disciplines. There s usually more than 5 to 6 textbooks and I would expect that would be the case here as well. One of the things weve also seen in the internet specifically is that the proliferation of long talethat is that now people can find the long time audience worldwide people like bloggers can now have the audience of 10,000 around and that allows the proliferation of different points of view that are now accessible to much larger population. So I would expect to see a small number of blockbusters and in the given area and it s gonna be a very long tail and obviously thats gonna be nice to have a bigger variety of ideas. The other thing I w anted to point out was that I dont think they are going to move to a world where it s the vast majority of people are going to get their education purely through online experience. So I think what we are going to see is that the faculty will make their own variations and the same thing with what they do with textbooks today. So we are going to see permutations that cater to the needs and interests of the populations. Sebastian Thrun Same here. Theres not just one car. We should experiment and the instructors should be in charge. You are gonna crank up money for these two weeks on the classes the reason is just like in movie industry when 100 million watch the movie you can actually spend 100 million dollars on it and just charge a person a dollar. Especially Udacityweve been somewhat shy on the mass production of MOOCs (???) or specialized computer science but we have the biggest MOOC between 40,000 students, partially because we are investing more and more into the visual courses. Some panel member said that it s time for faculty members to get out of online. Its time to get online. This thing is coming along the blogging and quality and stay out of it. Dont worry about it, keep doing what you are doingthe effect was somewhat devastating because of the bloggers were better than some of the mainstream media.

Burck Smith
I just wanted to clarify. I didnt mean to say we should get out of online. I didnt mean that colleges should not use online resources. Those are two very different things. The current model the most colleges pursue is offering full programs online. Those credit bearing online courses and those profit models for many colleges and thats where price points start plummeting towards zero and becoming very obviously so. But, I agree whole-heartedly that online materials, tools, those are clearly useful in any hybrid face-to-face environments. Chari Leader Kelley I probably is the one who s in the farthest in the peripheral here. But I want to just point out that in the US right now we have major achievement gap. K-12. We have haves and we have have-nots. And if we relegate online learning, MOOCs, and noncredit sources of learning to the group of people who cannot afford to have an education like the students here have access to. We will further exacerbate that gap between the haves and have-nots and thats something that would be very compelling to me if I were a faculty member thinking about the world that s emerging and changing the phase of our students and how can we help to level the plane somewhat through online learningbut by not relegating it to noncredit status or an experiment. Phil Hill This was driven in large part by the governors recent push about disruption and online education to address state-wide issues. Unfortunately, it was driven by the cost I know that was not the ideal situation but its a reality. Given the fact that 20 million minds set up this symposium and the discussions we are having are in large part reacting to that or response dealing with that question. Part of the question is what s the likelihood of that leading to any systematic change that benefits CA public higher education? Is the current push by the governor going to lead to any real change? What s going to happen about it? Robert Powell To some degree weve seen a lot of change of the online program at UC initiated a lot of activity also it ended up being diffused not only on campus but over 100 online courses were 18 of the MOOCs at this

point across the system. Partly fueled by UCOE but largely fueled by whats going on in individual campuses and fueled by individual faculty. I think there are changes coming I cant denyI mean no matter what we saywe cant deny the fact that the student-faculty ratio at UC is up from 18.7 to 23.5. Its cost of educating an undergraduate in real dollars went to 21,000 from 17,000 per student. We cant ignore all of these facts that are out there. Its stressing. The faculty is stressing. Our lecturers, our staff but at the same time there are other thingsI think we cant just focus on one thingthe teachingthere are also huge changes coming in research and stresses that are put on faculty are as a result of probably decrease in federal granting opportunities. Theres a lot of different things going on. Whenever a country wants to figure out how to deal with designing higher education system for public we come to California and its not because of the UCs, its not because of the CSUsits the fabric that we put togetherbecause a lot longer than that and it has offered opportunity and access and affordability to qualified students to get the best education from public education and that s the thing that we cant lose sight of. Phil Hill Thank you. Bob if you dont mind. You were talking earlier about the instructional cost that it might not be right from the cost-perspective. How is this current push with online education and the disruption and the general thrush therewhats it likely to lead to? Is it gonna lead to change? Bob Samuels I was kind of saying that its a distraction. I think we will incorporate a lot of online stuff and analyze that and try to do the best we canbut thats minor part of the problem and we re gonna have to figure out the state funding issue. We are gonna have to figure out the other types of cost at the university and I think this is becoming more and more of the distraction. It s easy for the governor to say ok we dont have to increase your funding or agree on a multi-year funding deal because you can just try to save money through online. And thats my fearwe are not going to be able to save money online and we are going to continue to get state-cuts and the state is not going to restore any of the funding. We are just gonna keep on having more and more problems. Fewer faculty, more students, more debt for students, higher tuitionI just think that we need to respond to those problems and so far the online thing does not make me confident that we are addressing our real issues. Michelle Pilati My concern is whats best for the students. If there are dollars behind distance education and online education in CA that are dedicated towards improving courses such that students improving things we are doing so we could increase the success rate so that the success rate in online courses look more like the success rate even better than the campus-based courses. If we can put energies in that direction, not just singlehere are some good ideas so use themhere are some good ideas go use them sit down use them and get all the way to the end so we are actually creating changes in our courses then I think you could see an actual systemic changes such that CA starts leading the quality doing with online instruction.

Closing Thoughts by Jeff Selingo I just wanted to share some of my key take-aways from the notes I took today. Phil asked earlier online coursesthat how many of us have taken online courses the question that I wished he would have asked in addition to that was how many of us went to residential colleges. And how many of us have went to elite residential collegesbecause I would assume that many people in this room probably did and you know the chronicle a couple of weeks ago wrote this piece that generated many many comments because we asked this uncomfortable question about whether we are designing a two-tiered system here. A system where face-to-face educationtheres elite face-to-face thats there for those who can afford for the high price and there s this disaggregated system for everyone else. Its definitely something I worry about and that we should all be worried about. And I just think it s question thats rarely addressed at sessions like this. Every college is looking for the magic bullet solution. President s boardsthey all want one single answer and I think that we know that theres no one single answer. You know one thing

that I worry about all the staff that Daphne and Sebastian are getting which is great for that I feel like its overshadowing all the other work that s been done in innovation now. Think about the competency based degrees and the stuff from the Keil we heard from today. Stuff Candice is doing. I met up with her a couple of weeks ago and I realized how hard it is to build a quality online courses like these and learning scientists and the data that the scientists usethis is not easy work. I just wish that we would get more press coverage of all the efforts being done in higher ed because there s not one solution and sometimes presidents boards think that theres one. That theres a easy one that some day they will go find it. So, thats number 1 Number 2 is we need more research. This is an academic issue. I think that s that academic should investigate it. And who will support it. Academic research in especially the sciences is decreasing in the DC and with cutbacks in the NSF and NIH. If it werent taken over by the congressman who were trying to deliver local project this would be great thing. Another thing is disseminating researchits a huge problem. I talked about UVA the board members were making decisions based on the newspaper article they read. Not based on anything on research literature and I think disseminating whatever we find in the research is also important. Number 3 is really defining what we mean by higher education. We talked a lot about teaching, but not so much about research which is an important function of the higher education . We also didnt talk about the maturing experience that especially residential colleges like this provide to 18 year olds. For all of us who went to residential colleges think about living with others and making mistakes and the serendipity of college campuses like this. Very difficult to replicate that online. And not seeing that sometimes we romanticize what happens in campuses like this. So that doesnt mean that we cant do things differentlybut I was interested in Daphnes comment about the MOOCs helping with the drop-out crisis we have in the US with students with somehow investigate what they are interested and realizing wow I dont really want to major in stats, or health sciences because I dont like blood. Finally, to me, the degree really means something. We talked a lot about Sebastian and Daphne had a lot of success with employers this is a certain subset of employers probably particularly in the Silicon Valley where we are used to the idea of having certifications and not degrees for the workers but employers I talked to still liked the idea fo the degree. To me, degree is not just a collection of credits its actually means something. At the end of the dayI like the ideabut someone has to figure out and certify it as a learning experience thats worth something. Finally, about the students who cant get into classes here in Californiamajor problemits the number one thing we hear. Of students who are frustrated many of them are sticking around but many of them are not. Not only CA facing this problemwhen you read NY times80% of Americans go to public colleges and we cant figure out what to do with their future that really worries me about the future of higher education.