Improving the quality of learning and teaching processes by designing and utilizing learning

outcomes.

In the Bologna documents it is quite clear that quality assurance is a highly valued
concept. I want to talk with you today about that idea of quality and to look just briefly at its
history. Perhaps the first recorded evidence of the scientific method which is a process that is
clearly connected to quality, occurred in the Novum Organum (1620) by Francis Bacon. He
suggested that a reiterative process would allow the investigator to find evidence that would
support a hypothesis and create a loop that would provide data to check for the achievement (or
not) of the hypothesis. Much later, in the 1940s in the US an engineer named Walter Shewhart
working for the Ford Motor Company began to do research on this same notion of a reiterative
process. He believed that data have no meaning apart from the context that they come from.
Working with Shewhart was the young W. Edwards Deming, himself an engineer and statistician
who was also quite interested in quality and the process that at least in manufacturing could
guarantee consistent products. During World War II Deming was working in Japan. As the war
ended, General MacArthur enlisted his help to try to impact the then destroyed manufacturing
industry. Prior to WWII products from Japan were considered quite shoddy. In that environment
Deming began to use this challenge as the place to develop and try out his theories about what it
would take to enhance quality production. Fast forward to the 1950s and his ideas and practices
had begun to take hold. Within another decade Japan was out-manufacturing most of the world
in such areas as steel production, car manufacturing and electronics. What did Deming do that
caused such a dramatic change? (Deming, 2000).

He had come to understand, as Bacon did, that quality happens only with continuous
improvement. His short hand for this process is Plan, Do, Check and Act and then repeat.
Perhaps the most surprising effort that he created was to convince the Japanese that the factory
workers should be involved in the decisions about how to improve the quality of the products.
Deming believed that no human should ever evaluate another human. This concept provided
opportunity for the oppression of the work place to change and to take the external or
behavioristic psychological model out of the manufacturing process and replace it with the
cognitive psychological model that supports the value and importance of intrinsic motivation.
 Along with the four step reiterative process Deming also created what has come to be called
the System of Profound Knowledge. The key to this was an expectation that in transforming the
views of the individual the system will transform.

This transformation is discontinuous. It comes from understanding of the system of profound

knowledge. The individual, transformed, will perceive new meaning to his life, to events, to
numbers, to interactions between people. Once the individual understands the system of
profound knowledge, he will apply its principles in every kind of relationship with other
people. He will have a basis for judgment of his own decisions and for transformation of the
organizations that he belongs to. The individual, once transformed, will:

 Set an example;
 Be a good listener;
 Continually teach other people; and
 Help people to pull away from their current practices and beliefs and move into the new
philosophy without a feeling of guilt about the past.

Deming advocated that all managers (or educators) need to have what he called a System of
Profound Knowledge that includes four points: an appreciation of a system; knowledge of
variation; a theory of knowledge; and knowledge of psychology (Delavigne, 1994). Although as
educators we might use slightly different language to describe these elements, nonetheless, our
classrooms and schools are indeed systems that demonstrate the same elements as any
manufacturing process. From these ideas came the development of his Fourteen Points which I
have adapted here for the university setting.

1. Create constancy of purpose toward improvement of teaching, learning and customer

service
2. Adopt the new philosophy and as educators we must take on the leadership for change.
3. Cease dependence on rote assessment as the way to achieve quality.
4. Improve constantly the cycle of teaching and learning
5. Institute training and faculty development as part of the job
6. The aim of supervision and teaching should be to help people to do a better job.
7. Drive out fear so that everyone may work effectively.
8. Break down barriers between departments and across job types.
9. Remove barriers that rob employees and students of their right to pride of workmanship.
10. Institute a vigorous program of education and self-improvement.
 11. Put everybody in the university or school to work to accomplish the transformation. The
transformation is everybody's job.

Unfortunately creating a quality response in the classroom, work place or manufacturing

world is quite difficult and as we know few other nations or companies have been able to adopt
what some have come to call Total Quality Management. Deming understood that many things
can get in the way of achieving this goal. These can include a lack of constancy of purpose, an
emphasis on short-term efforts while neglecting long range planning, seeking examples to follow
rather than developing solutions, making excuses, such as "our problems are different", using
techniques that are not research based, reliance on quality control processes rather than people,
placing blame on the workforce (or students) and finally relying on quality inspection rather than
improving product quality (Deming, 2000). How does this view of quality matter in the
university and what can we do about these ideas?

Perhaps the first place to look is not at our teaching but rather at what our students are
learning. So how does this notion of quality effect learning? Quality is a dynamic process for
continuous improvement. This is also true of the teaching-learning continuum. Just as the
manufacturing process is a system so is the classroom. If we follow the Deming model,
instruction must be designed in that 4 step process as well and learning new processes is required
to instill the courage to break with tradition. Every activity and every job is a part of the process

So who am I to talk about this process? I am a professor in Los Angeles at Antioch

University which is a small historic liberal arts university. As the department chair of education I
also teach like many of you. I began teaching secondary students 40 years ago and have been
teaching in higher education for 25 years. My students are post-baccalaureate students who are
learning to become teachers and also graduate students pursuing master’s degrees in education.
Our program is robust and provides a critical, constructivist approach to the teaching/learning
continuum. Teaching is my favorite thing to do and in many ways the most difficult.

But as a faculty member what do we do first in preparing for our classes? We think about
what we want our students to know, to experience and to be able to do at the end of the semester.
We set goals and plan how we will proceed. In recent years with the development of the
“outcome” movement or as some might call it the authentic assessment movement, we are now
expected to be able to describe not only what we will teach, and not only what we expect
students to learn but to provide evidence that in fact learning has taken place. This effort as you
well know encourages us to add to our plans more than what we will teach.

This places a different emphasis on what we do and in fact is difficult. Why is that? I
imagine that it is because we have all concentrated on our teaching and getting excellent at
delivering our subject (after all, we have studied our subject for many years particularly if you
are as old as I am) . But as more is discovered about learning theories, it is clear that our young
people need to be successful with 21st Century Skills. These have been recently established as the
following: Knowledge of Core Subjects; Learning and Innovation which include creativity,
innovation, critical thinking, collaboration, problem solving and communication; Information
Media and Technology which include Information and media literacy; and Life and Career
Skills. The research has become compelling. Simply teaching well is necessary but not
sufficient. We need to plan for what we want students to know, understand and be able to
demonstrate after the teaching is completed (Kehm, 2010). This effort becomes easier once we
can articulate the learning outcomes we expect.

In a recent major research process conducted by Ken Bain (2004) and reported in his text,
What the Best College Teachers Do, he interviewed and observed hundreds of university
professors. He and his research team were keen to find out the answer to that age old question,
what makes a good teacher? In a concluding remark he states that those who were deemed
“excellent” said that they believe that students must learn the facts while learning to use them.
Learning makes little sense unless it has some sustained influence on the way the learner
subsequently thinks, acts or feels so they teach facts in a rich context of problems, issues and
questions (29).

I would like to give you an example from my own work. At our university we recently spent
a great deal of time in preparation for our external accreditation visit and had to become very
serious about student learning outcomes (SLO). I recently have become quite interested in the
research of Mark Kishiyama and his colleagues from the Helen Wills Neurocognitive Institute at
the University of California Berkeley. Here is a brief synopsis of what piqued my curiosity.

Kids from lower socioeconomic levels show brain physiology patterns similar to
someone who actually had damage in the frontal lobe as an adult. We found that kids are
 more likely to have a low response if they have low socioeconomic status, though not
everyone who is poor has low frontal lobe response (Kishiyama, 2008).
This intrigued me since we prepare teachers to teach in very low Socio-Economic
schools. I created a course that looked at the cognitive neuroscience research and the impact of
classroom methodology on how students learn. One objective for the course was to introduce
my graduate students to the research from neuroscience. But I wanted them to DO something
with the information and at the same time demonstrate to me that they understood the concepts
and could put them into practice, using the highest levels of Bloom’s taxonomy. It was not
enough for me to have them just have knowledge but to move along the Bloom Taxonomy levels
through comprehension, and apply, analyze, synthesize and evaluate what they were learning.
My students and I created three SLOs for the course. (I want to add an aside here that in my
courses my students and I always co-create the syllabus together and work in a consensus
process to decide on assignments, the text books we will use and determine how each session of
the class will be conducted and how we will work together in the most democratic effort
possible.) Here is one of the three SLOs that we created.

Each of you have agreed to create a lesson plan that includes student learning outcomes
and that incorporates one or two methods that are consistent with cognitive neuroscience
research. After you teach the lesson to a group of children and videotape the lesson,
prepare a reflection of the lesson. In class you will share the video and your reflection
with your peers and the instructor. Everyone will prepare feedback and determine how
successfully the lesson met the learning outcomes and will make constructive suggestions
for improvement. Using this peer feedback, the lesson will be revised and perhaps re-
taught incorporating peer suggestions, The original lesson plan, the reflection, the video
and the revised lesson plan will be submitted with a final reflection and presented to the
class during the capstone process. Include a clear analysis of the significant changes in
your thinking that occurred during this process.
Since the goal of the course is three fold—teach content, demonstrate methodology and
provide practice to students, SLOs are designed to measure all three areas. By creating a
reiterative process for students to Plan, Do, Check and Act and then repeat as necessary, each
step allows opportunity for improvement. This builds confidence and demonstrates to the
students the 21st Century skills that are a key element to our Department Learning Outcomes.
Students have opportunity to revise and to make continuous improvement in the learning
process. This supports a psychology taught be William Glasser through Choice Theory (Glasser,
1994). External rewards are not necessary and students have a clear understanding of what is
expected of them, knowing that they can trust that there is opportunity for success. Glasser
makes clear in his work that if our aim is for quality work, then we must provide students with
multiple means for demonstrating their efforts. In my classes I try to provide students with
various forms of ongoing feedback about their work in a process that encourages continued
reflection and improvement.

Deming’s admonition that no human should ever evaluate another human is also
consistent with Choice theory and cognitive psychology. Both Deming and Glasser would say
that self-evaluation has great merit and can encourage the individual to self-assess and make
continuous improvement, whether it is in the classroom, in a relationship or in the work place.
Each of us knows when we have accomplished something of quality. We also know when we
have chosen to not do our best. Deming learned throughout his career that setting external limits
prevents the attainment of quality. It really is almost counter-intuitive for us as educators since
all of our lives we have been measured and ranked. Zacharias (J. Zacharias, personal
communication, New York University, 1983) has said that just because something is measured
does not guarantee that it will grow. The authentic assessment process is designed to help us
connect measurement with evidence that truly demonstrates learning.

I would like to return briefly to Shewhart’s statement that data have no meaning outside
of the context is also a key point about quality. How do we assess student learning in a
contextual manner? Many of us could say that at some point in our education we received an
unfair grade. We might even say that we received an inflated grade. Grade inflation and
subjectivity as a topic could fill another plenary. At my university, for example, we have
eschewed grades for more than 40 years, replacing them with narrative evaluations. In my
department where pedagogy is central to our work, students collaborate with their instructors to
co-write the narratives. The student learning outcomes serve as one set of guidelines for what is
evaluated but we also ask students and faculty to look at the key dispositions that we expect all
students to demonstrate. For our department these include being dedicated, optimistic (positive,
enthusiastic), adaptive (flexible), patient, collaborative (cooperative), compassionate
(empathetic), principled (concerned with social justice), proactive, open minded, creative, and
inquisitive. But evaluating the students is only one part of the teaching learning continuum. Are
the professors doing quality work as well?
 Our students are encouraged to evaluate the faculty at the midpoint of the semester and
these evaluations are shared in class so that any concerns or confusions can be immediately
addressed. At the conclusion of the semester students complete an anonymous evaluation and
they are asked to assess the faculty as a teacher, mentor and scholar utilizing the disposition as
one measure. The department chair reviews both sets of evaluations and meets with the faculty
member to plan for any changes that are suggested. This is done to help the faculty member to
continually improve their practice.

So how did my department move to add student learning outcomes to our work? How
did we create this student centered process? Just as Deming started with small steps, that is what
we did. Once we understood the teaching/learning connection, the next step was to determine
exactly what we wanted students to demonstrate as a result of our teaching. Working as a team,
the faculty talked about the kinds of evidence we wanted to gather and how we would collect the
evidence. In the past we only had objectives in our syllabi. An example might be, The student
will read Chapters 2-15 in the text. But we knew that we did not have good evidence to prove
that the student has accomplished our objective. Now as we design assignments we know that
we must have evidence that what we taught was “caught.” I must say that what before was
intuition about what students knew is no longer guess work. The process allows us to determine
if our hypothesis about what students know has in fact been proved (or not).

I want to include another quote from Bain. He spends a great deal of time in the study
describing many teaching practices but says…

The magic does not, however, lie in any one of these practices. I cannot stress enough
the simple yet powerful notion that the key to understanding the best teaching can be
found not in particular practices or rules but in the attitudes of the teachers, in their faith
in their students’ abilities to achieve, in their willingness to take their students seriously
and to let them assume control of their own education, and in their commitment to let all
policies and practices flow from central learning objectives and from a mutual respect
and agreement between students and teachers.
As Paul Baker, one of the participants of the study said, “we are not trying to force you
into some kind of a mold. On the contrary we are trying to help you escape.”(79)
I do want to offer another point of view on this notion of quality. I have been reading
with great interest the documents that define the Bologna process. It is clear in the US that there
is distrust and concern about external evaluation systems and the role they play in our daily lives,
What I have tried to say today is that for me and my department, being clear about what we want
students to demonstrate has increased our satisfaction with our work. But I do not want to leave
you today with the idea that this is the only view. Several international scholars have made
another view quite clear and I would like to quote Blackmur to have that other voice
represented. He says,

The idea that quality improvement in higher education is a good thing has almost
axiomatic status in the literature. It does not, however, deserve such prestige. It is
difficult to find a clear definition of quality improvement. Does the term mean, for
example, increasing the rate of compliance with a given minimum standard; exceeding a
given minimum standard by more and more; raising minimum standards; and so on?
Where multiple standards are involved, does, say, exceeding one of them more and more
over time while progress is static with respect to the others constitute quality
improvement? And how is ‘‘continuous’’ quality improvement defined and
operationalized? The conventional wisdom, moreover, ignores the distinct possibility that
any quality improvement may be costly (in financial and/or other terms) and, given this,
as a matter of logic, the value of any quality improvement beyond some point may be less
than the costs of securing it. It is thus arguably inappropriate for higher education quality
assurance agencies to include quality improvement in their objectives with respect to
higher education institutions since they are not responsible for university funding,
budgets and costs and thus lack the information which must necessarily underpin
prescriptions regarding many matters of quality improvement.

These are important points to discuss among and between your faculty. I do want to say
however that our teaching has improved as has our understanding of how we can increase
student learning. Could we have done that on our own without outside pressure? Yes but the
truth is that we did not. Peer review was very valuable for us and I know that these same kinds
of conversations that we had as a department and as a university were enlivening and scholarly.

So I want to thank you for your patience today and leave you with what Paulo Freire
would remind us. Teaching is an act of love and of optimism and delivers back to us all that we
give it. Thank you for being a teacher for that is how we make the world a better place. Deming
always ended his talks with this phrase. “Thank you. I have done my best.” This is a worthwhile
goal for our system of education, for each other and for ourselves that we can say that we have
done our best. Afterall, that is really all that we can do.
References

Bacon, F. (1620). Novum Organum. Retrieved from

http://www.constitution.org/bacon/nov_org.htm

Badri, M. A., & Abdulla, M. H. (2004). Awards of excellence in institutions of higher education:
an AHP approach. International Journal of Educational Management. 18(4), 224-242.

Bain, K. (2004). What the best college teachers do. Cambridge, MA: Harvard University Press.
Blackmore, J. (2009). Academic pedagogies, quality logics and performative universities:
Evaluating teaching and what students want. Studies in Higher Education, 34(8), 857-
872.

Blackmur, D. (2004). Issues in higher education quality assurance. Australian Journal of Public
Administration. 63(2), 105-116.

Blackmur, D. (2008). A critical analysis of the INQAAHE Guidelines of Good Practice for
higher education quality assurance agencies. Higher Education. 56(6), 723-734.

Blackmur, D. (2010). Does the Emperor have the right (or any) clothes? The public regulation of
higher education qualities over the last two decades. Quality in Higher Education. 16(1),
67-69.

Cret, B. (2011). Accreditations as local management tools. Higher Education, 61(4), 415-429.

Damme, D. V. (2001). Quality issues in the internationalisation of higher education. Higher

Education, 41(4), 415-441.

Delavigne, K. T., Robertson, J. D., & Robertson, D. (1994). Deming's Profound Changes: When
Will the Sleeping Giant Awaken? Englewood Cliffs, NJ: Prentice Hall.

Deming, W. E. (2000). Out of Crisis. Cambridge, MA: MIT Press.

Freire, P. (1998). Pedagogy of the Heart. New York: Continuum Press.

Glasser, W. (1998). Quality School. New York: Harper Collins.

Kehm, B. M. (2010). Quality in European higher education: The influence of the Bologna
Process. Change. 42(3), 40-46.

Kishiyama, M. M., Boyce, W. T., Jimenez, A. M., Perry, L. M., & Knight, R. T. (2009).
Socioeconomic disparities affect prefrontal function in children. Journal of Cognitive
Neuroscience. 21(6), 1106-15.

Michael, S. O. (1998). Restructuring US higher education: analyzing models for academic

program review and discontinuation. Association for the Study of Higher Education.
21(4), 377-404.

Partnership for 21st Century Skills. (2011). Framework for 21st century learning. Retrieved
from
http://www.21stcenturyskills.org/index.php?option=com_content&task=view&id=254&I

Raizada, R. D. S. & Kishiyama, M. (2010). Effects of socioeconomic status on brain

development, and how Cognitive Neuroscience may contribute to leveling the playing
field. Frontiers in Human Neuroscience. 4(3), DOI:
 http://dx.doi.org/10.3389/neuro.09.003.2010. PDF.

Reisz, M. (2010). Europe is told to relearn the ABCs of quality assurance. Times Higher
Education, 1975,18-19.

Shattock, M. (2008). The change from private to public governance of British higher education:
Its consequences for higher education policy making 1980-2006. Higher Education
Quarterly. 62(3), 181-203.

Singh, V., Grover, S., & Kumar, A. (2008). Evaluation of quality in an educational institute: a
quality function deployment approach. Educational Research and Review. 3(4), 162-168.

Smith, K. (2010). Assuring quality in transnational higher education: a matter of collaboration or

control? Studies in Higher Education. 35(7), 793-806.

Snee, R. D. (2008). W. Edwards Deming's "Making another world": A holistic approach to

performance improvement and the role of statistics. American Statistician. 62(3), 251-
255.

Tsinidou, M., Gerogiannis, V., & Fitsilis, P. (2010). Evaluation of the factors that determine
quality in higher education: an empirical study. Quality Assurance in Education. 18(3),
227-244.

Tsutsui, W. (1996). W. Edwards Deming and the origins of quality control in Japan. Journal of
Japanese Studies.  22(2), 295-325.