The National Academies Press

THE NATIONAL ACADEMIES PRESS
This PDF is available at http://nap.edu/10024 SHARE

   
Evaluating and Improving Undergraduate Teaching in Science,

Technology, Engineering, and Mathematics (2003)
DETAILS
232 pages | 8.5 x 11 | HARDBACK

ISBN 978-0-309-38465-0 | DOI 10.17226/10024
CONTRIBUTORS
GET THIS BOOK Marye Anne Fox and Norman Hackerman, Editors; Committee on Recognizing,
Evaluating, Rewarding, and Developing Excellence in Teaching of Undergraduate
Science, Mathematics, Engineering, and Technology; Center for Education;
Division of Behavioral and Social Sciences and Education; National Research
FIND RELATED TITLES Council
SUGGESTED CITATION
National Research Council 2003. Evaluating and Improving Undergraduate Teaching

in Science, Technology, Engineering, and Mathematics. Washington, DC: The
National Academies Press. https://doi.org/10.17226/10024.

Visit the National Academies Press at NAP.edu and login or register to get:
– Access to free PDF downloads of thousands of scientiﬁc reports

– 10% off the price of print titles
– Email or social media notiﬁcations of new titles related to your interests
– Special offers and discounts

Distribution, posting, or copying of this PDF is strictly prohibited without written permission of the National Academies Press.
(Request Permission) Unless otherwise indicated, all materials in this PDF are copyrighted by the National Academy of Sciences.
Copyright © National Academy of Sciences. All rights reserved.

Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics
Committee on Recognizing, Evaluating, Rewarding, and Developing

Excellence in Teaching of Undergraduate Science, Mathematics,
Engineering, and Technology
Marye Anne Fox and Norman Hackerman, Editors
Center for Education

Division of Behavioral and Social Sciences and Education
The National Academies Press

Washington, D.C.
www.nap.edu
Copyright National Academy of Sciences. All rights reserved.

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the
National Research Council, whose members are drawn from the councils of the National Academy of
Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the
committee responsible for the report were chosen for their special competences and with regard for
appropriate balance.
This study was conducted under an award from the Presidents of the National Academies. Any
opinions, findings, conclusions, or recommendations expressed in this publication are those of the
author(s) and do not necessarily reflect the views of the organizations or agencies that provided
support for the project.
Library of Congress Cataloging-in-Publication Data
Evaluating and improving undergraduate teaching in science, technology,

engineering, and mathematics / Marye Anne Fox and Norman Hackerman,
editors ; Committee on Recognizing, Evaluating, and Rewarding
Undergraduate Teaching, Center for Education, Division of Behavioral and
Social Sciences and Education National Research Council.
p. cm.
Includes index.
ISBN 0-309-07277-8 (pbk.)
1. Science—Study and teaching—Evaluation. 2. College
teaching—Evaluation. I. Fox, Marye Anne, 1947- II. Hackerman, Norman.
III. National Research Council (U.S.). Committee on Recognizing,
Evaluating, and Rewarding Undergraduate Teaching.
Q181 .E93 2003
507.1—dc21
2002013890
Additional copies of this report are available from the National Academies Press, 500 Fifth Street,
N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington
metropolitan area); Internet, http://www.nap.edu
Printed in the United States of America
Copyright 2003 by the National Academy of Sciences. All rights reserved.
Suggested citation: National Research Council. (2003). Evaluating and improving undergraduate
teaching in science, technology, engineering, and mathematics. Committee on Recognizing, Evaluating,
Rewarding, and Developing Excellence in Teaching of Undergraduate Science, Mathematics,
Engineering, and Technology, M.A. Fox and N. Hackerman, Editors. Center for Education, Division of
Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distin-

guished scholars engaged in scientific and engineering research, dedicated to the furtherance of
science and technology and to their use for the general welfare. Upon the authority of the charter
granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal
government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National
Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National
Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its
administration and in the selection of its members, sharing with the National Academy of Sciences the
responsibility for advising the federal government. The National Academy of Engineering also
sponsors engineering programs aimed at meeting national needs, encourages education and research,
and recognizes the superior achievements of engineers. Dr. Wm. A. Wulf is president of the National
Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure
the services of eminent members of appropriate professions in the examination of policy matters
pertaining to the health of the public. The Institute acts under the responsibility given to the National
Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon
its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is
president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to
associate the broad community of science and technology with the Academy’s purposes of furthering
knowledge and advising the federal government. Functioning in accordance with general policies
determined by the Academy, the Council has become the principal operating agency of both the
National Academy of Sciences and the National Academy of Engineering in providing services to the
government, the public, and the scientific and engineering communities. The Council is administered
jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. Wm. A. Wulf are
chair and vice chair, respectively, of the National Research Council.
www.national-academies.org


COMMITTEE ON RECOGNIZING, EVALUATING, REWARDING, AND

DEVELOPING EXCELLENCE IN TEACHING OF UNDERGRADUATE
SCIENCE, MATHEMATICS, ENGINEERING, AND TECHNOLOGY
MARYE ANNE FOX (Co-chair) North Carolina State University

NORMAN HACKERMAN (Co-chair) The Robert A. Welch Foundation, TX
TRUDY BANTA, Indiana University-Purdue University at Indianapolis
JOHN CENTRA, Syracuse University, NY
BARBARA GROSS DAVIS, University of California-Berkeley
DENICE DENTON, University of Washington
DIANE EBERT-MAY, Michigan State University
TIMOTHY GOLDSMITH, Yale University, CT
MANUEL GOMEZ, University of Puerto Rico
EILEEN LEWIS, University of California-Berkeley
JEANNE L. NARUM, Project Kaleidoscope, Washington, DC
CORNELIUS J. PINGS, University of Southern California
MICHAEL SCRIVEN, Claremont Graduate University, CA
CHRISTINE STEVENS, St. Louis University, MO
DENNIS WEISS, The Richard Stockton College of New Jersey
JAY B. LABOV (Study Director)

TERRY K. HOLMER (Senior Project Assistant)

COMMITTEE ON UNDERGRADUATE SCIENCE EDUCATION

JULY 1999
MARYE ANNE FOX (Chair), North Carolina State University

BONNIE J. BRUNKHORST, California State University, San Bernardino
MARY P. COLVARD, Cobleskill-Richmondville High School, NY
ARTHUR B. ELLIS, University of Wisconsin-Madison
JAMES M. GENTILE, Hope College, MI
RONALD J.W. HENRY, Georgia State University
HARVEY B. KEYNES, University of Minnesota
RONALD M. LATANISION, Massachusetts Institute of Technology
R. HEATHER MACDONALD, College of William and Mary, VA
RICHARD A. McCRAY, University of Colorado at Boulder
GRACE McWHORTER, Lawson State Community College, AL
EDWARD E. PENHOET, University of California-Berkeley
JAMES W. SERUM, SciTek Ventures, PA
ELAINE SEYMOUR, University of Colorado at Boulder
CHRISTY L. VOGEL, Cabrillo College, CA
vi

Preface
Americans have long appreciated the new opportunities for graduate and
need for high-quality education and undergraduate students at many univer-
have invested accordingly, at levels from sities to participate in original research
preschool through graduate education. projects. Recognition of the importance
Because of the impact of science and of original peer-reviewed research in
technology on the nation’s economic institutions of higher learning is clearly
growth, these fields have received laudable. As Robert Gavin noted in the
substantial government and private 2000 publication Academic Excellence:
research funding at colleges and univer- The Role of Research in the Physical
sities. Indeed, since World War II, Sciences at Undergraduate Institutions,
federal funding through peer-reviewed “research activity plays a central role in
grants and contracts has placed in the keeping the faculty up to date in the
hands of university faculty the primary field and improves their teaching.”
responsibility for more than half of the Because of the key role of science,
nation’s basic research in these fields. technology, engineering, and mathemat-
This investment has contributed signifi- ics (STEM), mechanisms for careful
cantly to making the United States a scrutiny and evaluation of the quality of
world leader in the discovery and research in these fields are highly
application of new knowledge and has developed, and academic scientists and
produced a well-respected system for engineers often derive reward and
graduate training in science and engi- recognition from their research achieve-
neering. In recent years, additional ments. As is the case with most scholar-
financial support from industry and ship, the criteria used in these evalua-
nonprofit organizations has provided tions differ from one discipline to
vii

another, and faculty evaluations at and achievements in the balance be-

research-intensive universities generally tween teaching and research. However,
solicit the candid judgments of national if the broad teaching missions of col-
or international peers from outside the leges and universities are to be attained,
home institution when a faculty member rigorous evaluation to improve teaching
or program is to be evaluated. Reliance and learning must become integral to
on one’s disciplinary colleagues for a STEM departmental culture. If so,
critique of the merits of one’s research faculty and administration must be
accomplishments and proposals is convinced that objective and compre-
widely accepted as a necessary invest- hensive methods exist for performing
ment of faculty time and effort. such evaluations and that these tech-
In contrast, the evaluation of teaching niques can be used without imposing
accomplishments has been more hap- undue burden or impossible time
hazard and less rigorous, particularly at commitments on already busy faculty.
research universities. Some faculty are Our study points out ways in which the
not convinced of the objectivity of fair evaluation of teaching and learning
techniques used for describing the in STEM disciplines can be institutional-
effectiveness of teaching and learning, ized as the basis for allocating rewards
especially at institutions at which com- and promotions, at a level of effort
peting demands on faculty time make it consistent with a department’s or
challenging to balance all of the normal college’s educational mission.
faculty responsibilities and to focus on Over the past several years, the
classroom and laboratory instruction. National Research Council (NRC) has
Even though the dominant values, assumed an aggressive role in strength-
beliefs, culture, and missions of many ening STEM education. The NRC’s
U.S. higher education institutions often Committee on Undergraduate Science
emphasize high-quality instruction, Education has coordinated this effort in
particularly in lower division under- colleges and universities. This study,
graduate teaching, a common percep- undertaken by the Committee on
tion is that teaching is less closely Evaluating Undergraduate Teaching,
scrutinized and less clearly rewarded examines the crucial issue of how best
than is research. Given the variety of to evaluate the effectiveness of under-
goals among the many different sizes graduate instruction to improve student
and types of American colleges and learning and to enhance faculty teach-
universities, it is not surprising that ing skills. The committee members
substantial differences exist in capability included faculty and administrators in
viii P R E FA C E

science, mathematics, and engineering; It is the committee’s view that a well-

experts in assessment and evaluation; structured evaluation of teaching can be
and representatives of several higher meaningful to those being evaluated and
education organizations dedicated to the to those who must render personnel
improvement of education. (See Appen- decisions based on these evaluations.
dix D for biographical sketches of the Conducted appropriately, such evalua-
committee members.) tions would be crucial components of
This is a timely undertaking. Pres- the institution’s efforts to improve
sures are mounting from within and education. Indeed, progress in educa-
beyond academe (e.g., state boards of tional research has clarified the effec-
regents and legislatures, business and tiveness of new methods, linking them
industry) to improve learning, particu- with demonstrable outcomes: improved
larly in introductory and lower-division student learning and academic achieve-
courses. These calls also request ment. It is the committee’s hope that
accountability of academic departments, the recent research findings presented
including a new emphasis on improved in this report will be incorporated into
teaching and enhanced student learning existing evaluative practice.
through curriculum revision and colle-
gial peer mentoring. Marye Anne Fox, Co-chair
Norman Hackerman, Co-chair
Committee on Recognizing,
Evaluating, Rewarding, and
Developing Excellence in
Teaching of Undergraduate
Science, Mathematics, Engi-
neering, and Technology
P R E FA C E ix


Acknowledgments
The committee members and staff of the NRC’s Center for Education
acknowledge the contributions of a (CFE) and currently executive director
number of people for providing presen- of the NRC’s Division of Behavioral and
tations, additional data, and valuable Social Sciences and Education, for
insight to the committee both during providing critical support and leader-
and between committee meetings: John ship during the writing and report
V. Byrne, President Emeritus, Oregon review phases of this study; Kirsten
State University and Director, Kellogg Sampson Snyder, CFE Reports Officer,
Commission on the Future of State and for her support and guidance in
Land-Grant Universities; Barbara shepherding this report through report
Cambridge, Director, Teaching Initia- review and in working on the final
tives, American Association for Higher stages of production; Rona Briere and
Education, and Director, Carnegie Kathleen (Kit) Johnston for their editing
Academy Campus Program; R. Eugene skills and insight; Eugenia Grohman
Rice, Director, Assessment Forum, and Yvonne Wise, for their assistance
American Association for Higher Educa- and support in revising the report at
tion; Alan H. Schoenfeld, Professor of several stages of its development; and
Education, University of California- also Rodger W. Bybee, former executive
Berkeley, and Chair, Joint Policy Board director of the NRC’s Center for Sci-
on Mathematics Task Force on Educa- ence, Mathematics, and Engineering
tional Activities. Education, and current Executive
At the National Research Council Director of the Biological Sciences
(NRC), we also would like to acknowl- Curriculum Study. Dr. Bybee helped
edge Michael J. Feuer, former director conceive this study and offered support
xi

and guidance for it during the time that and Learning and Department of
he was affiliated with the NRC. Mathematics, Harvard University
This report has been reviewed in Peter D. Lax, Courant Institute of
draft form by individuals chosen for Mathematical Sciences, New York
their diverse perspectives and technical University
expertise, in accordance with proce- Susan B. Millar, College of Engineer-
dures approved by the NRC’s Report ing, University of Wisconsin,
Review Committee. The purpose of this Madison
independent review is to provide candid Robert E. Newnham, Department of
and critical comments that will assist the Materials Science and Engineer-
institution in making the published ing, Pennsylvania State University
report as sound as possible and to Sheri D. Sheppard, Associate Profes-
ensure that the report meets institu- sor of Mechanical Engineering,
tional standards for objectivity, evi- Stanford University, and
dence, and responsiveness to the study Michael J. Smith, Education Director,
charge. The review comments and draft American Geological Institute.
manuscript remain confidential to
protect the integrity of the deliberative Although the reviewers listed above
process. We wish to thank the following have provided many constructive
individuals for their participation in the comments and suggestions, they were
review of this report: not asked to endorse the conclusions or
recommendations nor did they see the
David F. Brakke, Dean, College of final draft of the report before its re-
Science and Mathematics, James lease. The review of this report was
Madison University overseen by Frank G. Rothman, Brown
Brian P. Coppola, Department of University, and Pierre C. Hohenberg,
Chemistry, University of Michigan Yale University. Appointed by the NRC,
James Gentile, Dean, Natural Science they were responsible for making
Division, Hope College, Holland, certain that an independent examination
Michigan of this report was carried out in accor-
Melvin D. George, Department of dance with institutional procedures and
Mathematics, University of that all review comments were carefully
Missouri considered. Responsibility for the final
Daniel Goroff, Associate Director, content of this report rests entirely with
Derek Bok Center for Teaching the authoring committee and the institu-
tion.
xii ACKNOWLEDGMENTS

Contents
EXECUTIVE SUMMARY 1
PART I WHAT IS KNOWN: PRINCIPLES, RESEARCH FINDINGS,

AND IMPLEMENTATION ISSUES 9
1. Recent Perspectives on Undergraduate Teaching and Learning 11
2. Characterizing and Mobilizing Effective Undergraduate Teaching 25
3. Aligning the Cultures of Research and Teaching in Higher Education 40
4. Evaluating Teaching in Science, Technology, Engineering, and

Mathematics: Principles and Research Findings 51
PART II APPLYING WHAT IS KNOWN: STRATEGIES FOR

EVALUATING TEACHING EFFECTIVENESS 69
5. Evaluation Methodologies 71
6. Evaluation of Individual Faculty: Criteria and Benchmarks 100
7. Evaluation of Departmental Undergraduate Programs 108
8. Recommendations 115
xiii

REFERENCES 128
APPENDIXES
A Selected Student Evaluation Instruments 139
B Samples of Questionnaires Used to Evaluate Undergraduate

Student Learning 145
C Examples of Questions for Conducting Peer Evaluations of Teaching 185
D Biographical Sketches of Committee Members 196
INDEX 203
xiv CONTENTS



Executive Summary
This report recommends a set of determine whether fair and objective

strategies to evaluate undergraduate methods exist for the evaluation of
teaching and learning in science, tech- teaching and learning, and if so, how
nology, engineering, and mathematics such methods could be used as a basis
(STEM1 ). It is based on a study con- for the professional advancement of
ducted by a National Research Council faculty. The committee found that many
(NRC) committee charged with synthe- such methods exist, and that their utility
sizing relevant research in pedagogy deserves wider appreciation and applica-
and practice as a basis for developing tion in the evaluation of both individuals
resources to help postsecondary STEM and departments.
faculty and administrators evaluate and The committee found that summative
reward effective teaching. The study evaluations of teaching, such as those
committee was a subcommittee of the used in some faculty promotion and
NRC’s Committee on Undergraduate tenure decisions, often do not rely on
Science Education. evidence of student learning, and this
The committee’s principal goal was to relationship needs to be strengthened
and formalized. The committee also
found that formative evaluations (e.g.,
ongoing informal feedback from stu-
dents and colleagues) can serve several
1
This abbreviation for science, technology,
important educational goals: (1) cou-
engineering, and mathematics education, taken
from the official designation of the National pling candid teaching evaluation with
Science Foundation for education in the disci- opportunities for ongoing professional
plines, is used as shorthand throughout the
report. development; (2) supporting faculty

who wish to explore the scholarship of throughout their careers; hiring prac-
teaching and learning; and (3) applying tices should provide a first opportunity
such formative evaluation techniques to to signal institutions’ teaching values
departmental programs, not only to and expectations of faculty.
individual faculty.2
Four fundamental premises guided Underlying these premises is the
the committee’s deliberations: committee’s recognition that science,
mathematics, and engineering instruc-
(1) Effective postsecondary teaching tors face a number of daunting chal-
in science, mathematics, and technology lenges: the need to apply principles of
should be available to all students, human learning from research in
regardless of their major. cognitive science to the assessment of
(2) The design of curricula and the learning outcomes, to teach and advise
evaluation of teaching and learning large numbers of students with diverse
should be collective responsibilities of interests and varying reasons for
faculty in individual departments or, enrolling, to prepare future teachers, to
where appropriate, through interdepart- provide faculty and students with
mental arrangements. engaging laboratory and field experi-
(3) Scholarly activities that focus on ences, and to supervise students who
improving teaching and learning should undertake original research. Simulta-
be recognized as bona fide endeavors neously addressing these challenges
that are equivalent to other scholarly requires knowledge of and enthusiasm
pursuits. Scholarship devoted to im- for the subject matter, familiarity with a
proving teaching effectiveness and range of appropriate pedagogies, skill in
learning should be accorded the same using appropriate tests, ease in profes-
administrative and collegial support that sional interactions with students within
is available for efforts to improve other and beyond the classroom; and active
research and service endeavors. scholarly assessment to enhance teach-
(4) Faculty who are expected to work ing and learning.
with undergraduates should be given Yet the committee found that most
support and mentoring in teaching faculty who teach undergraduates in the
STEM disciplines have received little
formal training in teaching techniques,
in assessing student learning, or in
2
evaluating teaching effectiveness.
Detailed definitions of formative and
summative evaluation can be found in Chapter 5. Formal programs aimed at improving
2 E VA L U AT I N G A N D I M P R O V I N G U N D E R G R A D U A T E T E A C H I N G

teaching are still rare. A firm commit- could be compared with the other
ment to open intradepartmental commu- independent evidence. The instructor’s
nication about teaching effectiveness is willingness to seek external support to
therefore critical to any convincing improve teaching and learning also is
evaluation of teaching based on these evidence of her or his commitment to
premises. And because considerable effective undergraduate teaching.
variation exists across institutions and Effective evaluation also emerges
disciplines, there is no single formula or from a combination of sources of evi-
pathway to effective evaluation of dence. Current students, those who had
teaching. taken a course in previous years, and
The research literature suggests that graduating seniors and alumni could
some combination of the following kinds provide evidence about the instructor’s
of formative and summative evidence role in their learning. Graduate teaching
about student learning can be helpful in assistants could discuss the instructor’s
evaluating and improving a faculty approaches to teaching, levels of interac-
member’s teaching: tions with students, and the mentoring
Departmental and other colleagues can that they receive in improving their own
provide informed input about teaching teaching skills. Departmental and other
effectiveness through direct observa- faculty colleagues, both from within and
tion, analysis of course content and outside the institution, could evaluate
materials, or information about the the currency of the materials the in-
instructor’s effectiveness in service and structor presents and his or her level of
interdisciplinary courses. Undergradu- participation and leadership in improv-
ates and graduate teaching assistants ing undergraduate education. The
could offer useful information based on faculty member being evaluated can
their experiences in the instructor’s provide critical information about his or
courses and laboratories, the her teaching challenges and successes
instructor’s supervision of research, and through self-reflection and other evi-
the quality of academic advising. Addi- dence of effective teaching, student
tionally, graduate students could com- learning, and professional growth.
ment on the supervision and mentoring Institutional data and records offer
they have received as they prepare for insights about changes in enrollments
teaching. The faculty member being in a faculty member’s courses over time,
evaluated could provide self-assessment the percentage of students who drop the
of his or her teaching strengths and instructor’s courses, and the number of
areas for improvement; this assessment students who go on to take additional
EXECUTIVE SUMMARY 3

courses in the discipline and related graduate education is critical to the

subject areas. success of such approaches. Improving
Each of these criteria is subject to summative evaluation also requires that
multiple interpretations and should be the faculty at-large, academic adminis-
viewed with care. For example, re- trators, and committees on promotion
search suggests that grade distributions and tenure have confidence in the
are not as useful an indicator of teaching credibility of the process.
effectiveness as other types of indicators
and should be used cautiously, if at all.
A central idea behind formative RECOMMENDATIONS
evaluation of teaching and learning is a
two-way feedback system known as An undisputed strength of American
“outcomes assessment.” Faculty need higher education is that each institution
to set clear goals for their students and has a unique mission. It is very unlikely
ascertain whether students are meeting that any general model for evaluating
those goals throughout the course. teaching and learning could pertain to
Students need to have a clear idea of all schools. Several broad recommenda-
what is expected of them and whether tions, however, may be generally useful
they are meeting those expectations. when adapted to local goals and visions.
Chapter 5 describes in detail a variety of
procedures that close these feedback 1. Overall Recommendations
loops, providing faculty with credible (1.1) Teaching effectiveness should
information about what students know be judged by the quality and extent of
and can do as a result of instruction student learning. Many different
while giving students information about teaching styles and methods are likely
how well they have mastered the course to be effective.
material. Whatever the means of (1.2) Scholarly activities that focus
outcomes assessment that are em- on improving teaching and learning
ployed, measures of students’ concep- should be recognized and rewarded as a
tual understanding are critically impor- bona fide scholarly endeavor and
tant in judging the success of a course. accorded the types of institutional
Implementing such processes can be supports aimed at improving scholar-
time-consuming and involve faculty ship generally.
other than the instructor in charge of (1.3) Valid summative assessments
the course. Departmental commitment of teaching should not only rely on
to the shared goal of improving under- student evaluations, but should include

peer reviews and teaching portfolios same time, institutions should recognize
used for promotion, tenure, and post- that disciplines approach teaching
tenure review. Such assessments differently and that such differences
should be designed to provide fair and should be reflected in evaluation proce-
objective information to aid faculty in dures.
the improvement of their teaching. Much of this report offers recommen-
Building consensus among faculty, dations to faculty about how they can
providing necessary resources, and use evaluation to improve their teach-
relying on the best available research on ing. Accordingly, the following set of
teaching, learning, and measurement recommendations is directed toward
are critical for this approach to evalua- policy makers, administrators, and
tion. leaders of organizations associated with
(1.4) Individual faculty—beginners higher education.
as well as more experienced teachers—
and their departments should be re- 2. Recommendations for
warded for consistent improvement of Presidents, Overseeing Boards,
learning by both major and nonmajor and Academic Officers
students. All teaching-related activi- (2.1) Quality teaching and effective
ties—such as grading, reporting of learning should be highly ranked
grades, curriculum development, institutional priorities. All faculty and
training of teaching assistants, and departmental evaluations and accredita-
related committee work—should be tion reviews should include rigorous
included in evaluation systems adopted assessment of teaching effectiveness.
for faculty rewards. University leaders should clearly assert
(1.5) Faculty should accept the high expectations for quality teaching to
obligation to improve their teaching newly hired and current faculty.
skills as part of their personal commit- (2.2) Campus-wide or disciplinary-
ment to professional excellence. De- focused centers for teaching and learn-
partments and institutions of higher ing should be tasked with providing
education should reinforce the impor- faculty with opportunities for ongoing
tance of such professional development professional development that include
for faculty through the establishment understanding how people learn, how to
and support of campus resources (e.g., improve current instruction though
centers for teaching and learning) and student feedback (formative evaluation),
through personnel policies that recog- and how educational research can be
nize and reward such efforts. At the translated into improved teaching
EXECUTIVE SUMMARY 5

practice. Such centers should provide mentoring graduate teaching assistants

equipment and facilities required for and advising them about their duties to
innovative teaching. undergraduate students.
(2.3) At least one senior university-
level administrator should be assigned 3. Recommendations for Deans,
responsibility for encouraging depart- Department Chairs, and Peer
mental faculty to adopt effective means Evaluators
(as proven by research) to improve (3.1) Departments should periodi-
instruction. cally review a departmental mission
(2.4) Faculty who have excelled in statement that includes appropriate
teaching should be publicly recognized emphasis on teaching and student
and rewarded. Endowments should be learning. These reviews should address
established to recognize the serious not only the major curriculum, but also
contributions of faculty who have made service offerings—such as courses
a sustained contribution to quality designed for nonmajors and prospective
teaching. teachers.
(2.5) Faculty should be encouraged (3.2) Individual faculty members
to develop curricula that transcend should be expected to contribute to a
disciplinary boundaries, through a balanced program of undergraduate
combination of incentives (including teaching. Participation of established
funding), expectations of accountability, faculty in lower-division, introductory,
and development of standards for and general-education courses should
disciplinary and interdisciplinary be encouraged. Faculty who are most
teaching. familiar with new developments in the
(2.6) Willingness to emphasize discipline can provide leadership in
student learning and to make allocations departmental curricular review and
of departmental resources in support of revision. Not all faculty must contribute
teaching should be an essential require- equally to instruction at every level, but
ment in appointing deans, department it is a departmental responsibility to
chairs, and similar administrative ensure that the instructional needs of all
positions. students are met by caring, responsible
(2.7) Graduate school deans should faculty.
require that departments that employ (3.3) Departments should contrib-
graduate students in fulfilling their ute to campus-wide awareness of the
teaching mission should show evidence premium placed on improved teaching.
that their faculties are effectively They should build consensus among

their own faculty about the suitability of dent learning.

the institution’s procedures for (3.7) As appropriate for achieving
summative evaluation of teaching, departmental goals, departments should
recognizing that the way that practitio- provide funds to faculty to enhance
ners of a specific discipline approach teaching skills and knowledge and
learning will affect the ways that teach- encourage them to undertake or rely
ing should be evaluated. upon educational research that links
(3.4) In addition to numerical data teaching strategies causally to student
from end-of-course student evaluations learning. Additional funds should be
and on participation in specific courses, made available to departments that
effective peer reviews of teaching adopt this strategy.
should provide a subjective assessment (3.8) Departments should recognize
of a faculty member’s commitment to that in the course of their careers, some
quality teaching. Generally, this should faculty may shift the balance of their
include evaluation of a faculty member’s departmental obligations to place a
knowledge and enthusiasm for the greater emphasis on instruction or
subject matter; familiarity with a range educational leadership. These shifts
of appropriate pedagogical methods; should be supported, consistent with a
skills in using appropriate tests and departmental mission, so long as active
laboratory experiences; quality of engagement with innovative teaching is
advising and other professional interac- being addressed.
tions with students within and beyond
the classroom; and active scholarly 4. Recommendations for
commitment to enhancing top-quality Granting and Accrediting
teaching and learning. Agencies, Research Sponsors,
(3.5) Department heads, in submit- and Professional Societies
ting personnel recommendations, (4.1) Funding agencies should
should provide separate ratings on support programs to enable an inte-
teaching, research, and service, each grated network of national and campus-
with supporting evidence, as key com- based centers for teaching and learning.
ponents of their overall rating and An important goal of such a network is
recommendation. to conduct and disseminate research on
(3.6) Normal departmental profes- approaches that enhance teaching and
sional development activity should learning in STEM. The network can
include informing faculty about re- also provide information on the use of
search findings that can improve stu- formative and summative assessment
EXECUTIVE SUMMARY 7

for improving teaching and learning. To primary criterion for program accredita-
the extent possible, these investments tion.
should not be made at the expense of (4.4) Professional societies should
sponsored research. offer opportunities to discuss under-
(4.2) Funding agencies and re- graduate education issues during annual
search sponsors should undertake a and regional meetings. These events
self-examination by convening expert might include sessions on teaching
panels to examine whether agency techniques and suggestions for over-
policies might inadvertently compro- coming disciplinary and institutional
mise a faculty member’s commitment to barriers to improved teaching.
quality undergraduate teaching. (4.5) Professional societies should
(4.3) Accreditation agencies and encourage publication of peer-reviewed
boards should revise policies to empha- articles in their general or specialized
size quality undergraduate learning as a journals on evolving educational issues
in STEM.

PART I
What Is Known:
Principles, Research Findings,
and Implementation Issues


1
Recent Perspectives
on Undergraduate
Teaching and Learning
This report addresses a crucial particularly in the STEM disciplines.

challenge to changing and improving Top-ranking policy makers (e.g.,
undergraduate education in the United Greenspan, 2000; Seymour, in press)
States: how to evaluate the effective- have stated that globalization of the
ness of undergraduate teaching in economy, significant advances in scien-
science, technology, engineering, and tific discovery, and the ubiquity of
mathematics (STEM1 ) in ways that will information technologies make it
enable faculty to enhance student imperative for all U.S. students (grades
learning, continually improve teaching K–16) to understand the methods and
in these fields, and allow faculty to basic principles of STEM if they are to
develop professionally in the practice succeed. Recent reports from the
and scholarship of teaching and learn- National Science Foundation ([NSF],
ing. Although many view higher educa- 1996, 1998), the National Science Board
tion in the United States as among the (2000), the National Research Council
best such systems in the world, there (NRC), (1996b, 1999a), and others (e.g.,
have been numerous calls for reform, Boyer Commission, 1998) have chal-
lenged the nation’s colleges and univer-
sities to ensure that all undergraduates
increase their knowledge and under-
1
standing of STEM and the relevance of
This abbreviation for science, technology,
engineering, and mathematics education, taken these disciplines to other areas of
from the official designation of the National learning and human endeavors.
Science Foundation for education in the disci-
plines, is used as shorthand throughout the
report.
11

IMPETUS FOR AND CHALLENGES rate leaders and the public alike are
TO CHANGE focusing on the need for a scientifically
and technologically literate citizenry
Calls for Accountability from and a skilled workforce (Capelli, 1997;
Outside of Academe Greenspan, 2000; International Technol-
ogy Education Association, 2000;
The reforms within K–12 education
Murnane and Levy, 1996; National
that have been enacted in almost every
Council of Teachers of Mathematics,
state and many districts include systems
2000; NRC, 1996a, 1999a, 2000d).
for measuring achievement and account-
Corporate leaders also have made it
ability. State legislatures, departments
increasingly clear that their workforce
of education, school boards, and the
needs more than basic knowledge in
general public expect those responsible
science, mathematics, and technology.
for educating students to be held specifi-
They expect those they hire to apply
cally accountable for the quality of the
that knowledge in new and unusual
outcomes of their work (Rice et al.,
contexts, as well as to communicate
2000).
effectively, work collaboratively, under-
The call for accountability is also
stand the perspectives of colleagues
being clearly heard at the
from different cultures, and continually
postsecondary level. State legislatures
update and expand their knowledge and
are demanding that public universities
skills (Capelli, 1997; Greenspan, 2000;
provide quantifiable evidence of the
Rust, 1998).
effectiveness of the academic programs
being supported with tax dollars. Other Calls for Change from Within
bodies, including national commissions, Academe
institutional governing boards, and
While public pressure for reforming
professional accrediting agencies, also
undergraduate teaching and learning
have begun to recommend that universi-
and holding educators accountable for
ties and colleges be held more account-
such improvements is real and growing,
able for student learning (see, e.g.,
recent surveys also suggest that in-
National Center for Public Policy and
creasing numbers of faculty are advo-
Higher Education, 2001; see also Chap-
cating strongly for quality teaching and
ter 3, this report).
are paying close attention to how
One aspect of the call for accountabil-
effective teaching is recognized, evalu-
ity in higher education is particularly
ated, and rewarded within departments
important for faculty in STEM. Corpo-

and at the institutional level (Ory, 2000). expectations for their performance are
In a recent survey of doctoral students, vague and sometimes conflicting. They
for example, 83 percent indicated that also indicated that feedback on their
teaching “is one of the most appealing performance often is insufficient,
aspects of faculty life, as well as its core unfocused, and unclear. Many ex-
undertaking” (Golde and Dore, 2001, p. pressed concern about the lack of a
21). “culture of collegiality” or a “teaching
In recent interviews with new faculty community” at their institutions (Rice et
members, Rice et al. (2000)2 reported al., 2000).
that interviewees overwhelmingly During the past decade, there also
expressed enjoyment of and commit- has been increasing concern among
ment to teaching and working with senior faculty and administrators about
students. However, early-career faculty improving undergraduate STEM educa-
expressed concerns about how their tion. These efforts have been spurred
work is evaluated. They perceive that by reports from a variety of national
organizations (e.g., Boyer, 1990; Boyer
Commission, 1998; NRC, 1996b, 1997a;
NSF, 1996; Project Kaleidoscope, 1991,
1994) calling for reform in these disci-
plines. Professional societies also are
2
This report by Rice et al. (2000) is a product devoting serious attention to enhancing
of the American Association for Higher
undergraduate teaching and learning in
Education’s (AAHE’s) ongoing Forum on Faculty
Roles and Rewards. The report provides the these disciplines (e.g., Council on
results of structured interviews that were Undergraduate Research <http://
undertaken with 350+ new faculty members and
graduate students aspiring to be faculty members www.cur.org>; Doyle, 2000; McNeal and
from colleges and universities around the D’Avanzo, 1997; NRC, 1999b, 2000b;
country. The aim of that study was to obtain
perspectives from those who are just beginning Howard Hughes Medical Institute
their academic careers and to offer guidance for <http://www.hhmi.org>; National
senior faculty, chairs, deans, and others in higher
education who will be responsible for shaping the Institute for Science Education <http://
professoriate of the future. Rice et al. offer ten www.wcer.wisc.edu/nise>; Project
“Principles of Good Practice: Supporting Early-
Career Faculty,” accompanied by an action
Kaleidoscope <http://www.pkal.org>;
inventory to prompt department chairs, senior Rothman and Narum, 1999; and
colleagues, and other academic leaders to
websites and publications of increasing
examine their individual and institutional
practices. These principles and specific action numbers of professional societies in the
items are also available in a separate publication natural sciences, mathematics, and
by Sorcinelli (2000), which is available at <http://
www.aahe.org/ffrr/principles_brochure.htm>. engineering).
RECENT PERSPECTIVES 13

Challenges to Change teaching or a department’s curriculum

Although there are many pressures objectively (Glassick et al., 1997). This
on postsecondary institutions to exam- challenge becomes especially difficult
ine and change their practices and when one of the measures is the amount
assumptions about teaching and learn- students have learned.
ing, it also is clear that the circum- Finally, perhaps the most significant
stances in which such changes must challenge is that many undergraduate
occur are exceedingly complex. One faculty in the STEM disciplines have
challenge is the diversity of the U.S. received little or no formal training in
higher education community. Institu- techniques or strategies for teaching
tions range from those that serve effectively, assessing student learning,
several hundred students to those that or evaluating the effectiveness of their
enroll many thousands. Institutional own teaching or that of their colleagues.
histories and academic missions vary Such training is not a firm requirement
widely, as do their sources of support, for being hired as a college-level faculty
means of governance, and student member. Formal, ongoing programs for
populations. These differences inevita- professional development aimed at
bly result in varying expectations on the improving teaching are still rare at
part of students, faculty, parents, and many postsecondary institutions.
funders with respect to the relative Faculty may discover what is known
balance among research, teaching, and about assessing learning only by perus-
service. ing the research literature, by participat-
A second challenge is that some ing in workshops on teaching and
deeply entrenched aspects of university learning (e.g., Bloom, 1956; see also
culture need to change if undergraduate Anderson et al., 2001; Chickering and
teaching and learning are to improve Gamson, 1987; and Osterlind, 1989), or
(Mullin, 2001). One perception of the by discussing problems with colleagues.
current culture is that more professional The ultimate goal of undergraduate
rewards and recognition accrue to those education should be for individual
faculty who succeed at research than to faculty and departments to improve the
those who devote their energies prima- academic growth of students. A consid-
rily to teaching (Brand, 2000). This erable body of research now exists on
perception persists because many how students learn (summarized in How
postsecondary faculty and administra- People Learn: Brain, Mind, Experience,
tors believe that it is difficult to measure and School, NRC, 2000c); on the assess-
the effectiveness of an instructor’s ment of teaching and learning (e.g.,

Knowing What Students Know: The The goal of this project is to develop
Science and Design of Educational resources to help postsecondary science,
technology, engineering, and mathemat-
Assessment, NRC, 2001); and on other
ics (STEM) faculty and administrators
research findings that relate closely to
gain deeper understanding about ways
the responsibilities of undergraduate convincingly to evaluate and reward
faculty and could lead to direct improve- effective teaching by drawing on the
ments in undergraduate education. results of educational research. The
Overviews and summaries of research committee will prepare a National
on learning and the application of that Research Council report on the evalua-
tion of undergraduate STEM teaching,
scholarship to the assessment of learn-
with a focus on pedagogical and imple-
ing are provided at the end of this
mentation issues of particular interest to
chapter in Annex Boxes 1-1 and 1-2, the STEM community. The report will
respectively. emphasize ways in which research in
Many college faculty are not familiar human learning can guide the evaluation
with that literature, however, nor do and improvement of instruction, and will
they have the time, opportunity, or discuss how educational research find-
ings can contribute to this process.
incentives to learn from it. Moreover,
assessing whether students actually In responding to this charge, the
have learned what was expected re- committee embraced four fundamental
quires that faculty rethink course premises, all of which have implications
objectives and their approaches to for how teaching is honored and evalu-
teaching. Extending the assessment of ated by educational institutions:
learning outcomes beyond individual
courses to an entire departmental • Effective postsecondary teaching
curriculum requires that faculty collec- in STEM should be available to all
tively reach consensus about what students, regardless of their major.
students should learn and in which • The design of curricula and the
courses that knowledge and those skills evaluation of teaching and learning
should be developed. should be collective responsibilities of
faculty in individual departments or,
where appropriate, performed through
STATEMENT OF TASK AND other interdepartmental arrangements.
GUIDING PRINCIPLES • Scholarly activities that focus on
improving teaching and learning should
The committee conducted its work be recognized as bona fide endeavors
according to the following statement of that are equivalent to other scholarly
task from the NRC: pursuits. Scholarship devoted to im-

proving teaching effectiveness and growing segment of the higher educa-

learning should be accorded the same tion community. Accordingly, this
administrative and collegial support that report is organized according to six
is available for efforts to improve other guiding principles:
research and service endeavors.
• Faculty who are expected to work (1) A powerful tool for increasing
with undergraduates should be given student learning is ongoing, informal
support and mentoring in teaching assessment (formative assessment).
throughout their careers; hiring prac- Emerging research on learning shows
tices should provide a first opportunity that thoughtful and timely feedback
to signal institutions’ teaching values informed by pedagogical content knowl-
and expectations of faculty. edge3 is critical for developing among
students at all levels a more advanced
Thus, the central theme of this report understanding of key concepts and
is that teaching evaluation must be skills in a discipline.
coupled with emphasis on improved (2) Formative assessment has benefits
student learning and on departmental for both students and faculty. Faculty
and institutional support of improved
teaching through ongoing professional
development. Although the challenge is
daunting, it is far from impossible. To
the contrary, there is mounting evi- 3
Shulman (1986, p. 9) was the first to propose
dence that colleges and universities of the concept of pedagogical content knowledge,
all types are embracing the challenge of stating that it “. . . embodies the aspects of
content most germane to its teachability. . . .
improving undergraduate teaching and [P]edagogical content knowledge includes . . .
resolving these issues in innovative the most powerful analogies, illustrations,
examples, explanations, and demonstrations—in
ways (Suskie, 2000). The committee a word, the ways of representing and formulating
was convinced by its examination of a the subject that makes it comprehensible to
others. . . .[It] also includes an understanding of
wide range of literature that well- what makes the learning of specific concepts
designed and implemented systems for easy or difficult: the conceptions and preconcep-
tions that students of different ages and back-
evaluating teaching and learning can grounds bring with them to the learning.” Thus,
and do improve undergraduate educa- teachers use pedagogical content knowledge to
relate what they know about what they teach
tion. Research on effective evaluation of
(subject matter knowledge) to what they know
teaching points to a number of prin- about effective teaching (pedagogical knowl-
ciples that are increasingly well sup- edge). The synthesis and integration of these
two types of knowledge characterize pedagogical
ported by evidence and embraced by a content knowledge (Cochran, 1997).

who use formative assessment effec- and so on to produce portfolios and

tively also benefit because the feedback other materials that make their work
loop that is established as they obtain visible to wider communities in reliable
information about student learning and valid ways. Producing such materi-
enables them to determine rapidly and als demonstrates the accomplishments
accurately how to adjust their teaching of faculty in fostering student learning,
strategies and curricular materials so in developing themselves as scholars,
their students will learn more effectively and in contributing to their fields.
(see Chapter 5, this report, for addi- (5) Embracing and institutionalizing
tional details). effective evaluation practices can ad-
(3) Appropriate use of formative vance the recognition and rewarding of
evaluation facilitates the collection and teaching scholarship and communities of
analysis of information about teaching teaching and learning. By adopting
effectiveness for more formal personnel policies and practices that inform and
decisions (summative evaluation). If support the effective use of formative
formative evaluation is employed regu- evaluation, departments, institutions,
larly, faculty also generate information and professional societies can develop
they can use for purposes of document- effective criteria for evaluating
ing the effectiveness of their teaching summatively the teaching effectiveness
when they are involved with personnel and educational scholarship of faculty.
decisions such as continuing contracts, (6) Effective and accepted criteria
salary increases, tenure, promotion, or and practices for evaluating teaching
professional awards. Departments and enable institutions to address the concerns
institutions also can use data compiled of those who are critical of undergraduate
by individual faculty to examine student teaching and learning. As links between
learning outcomes and to demonstrate formative and summative student
what and how students are learning. assessment and between summative
(4) The outcomes of effective forma- student assessment and faculty evalua-
tive and summative assessments of tion become part of everyday practice,
student learning by individual faculty can higher education leaders will be able to
be used by other faculty to improve their respond more effectively to criticisms
own teaching, as well as by departments about the low visibility and value of
to strengthen existing academic programs teaching in higher education.
or design new ones. Faculty can inte-
grate such information with their own In applying these principles, indi-
course materials, teaching philosophy, vidual faculty, academic departments,

and institutions of higher education can issues. Chapter 2 reviews characteris-

benefit from an overview of existing tics of effective undergraduate teaching
research on effective practices for and summarizes challenges that faculty
evaluating faculty and academic pro- may encounter in trying to become
grams. They also need practical guid- more effective teachers. By comparing
ance about how to initiate the process or the “cultures” of teaching and disciplin-
advance it on their campuses. Meeting ary research, Chapter 3 examines
these needs is the primary purpose of barriers associated with making under-
this report. graduate teaching and learning a more
central focus through effective systems
for teaching evaluation. This chapter
ORGANIZATION OF AND also provides suggestions for better
INTENDED AUDIENCES FOR aligning these cultures within the
THIS REPORT university. Chapter 4 presents key
research findings on how to evaluate
undergraduate teaching in STEM more
Report Organization effectively.
In this report, the six organizing Part II (Chapters 5 through 8)
principles stated above are used to applies the principles, research findings,
provide an overview of the current and recommendations set forth in Part
status of research on evaluating teach- I, providing an overview of specific
ing and learning. The report also methodologies and strategies for evalu-
provides a set of guidelines, based on ating the effectiveness of undergraduate
emerging research, for evaluating the teaching in STEM. Chapter 5 reviews
teaching of individuals and the academic a variety of methodologies that can be
programs of departments. Faculty and used to evaluate teaching effectiveness
administrators can adapt these ideas for and the quality of student learning.
evaluating teaching and programs to the Some of these methods also can be
needs of their departments and cam- applied to evaluate teaching, course
puses as appropriate for their institu- offerings, and curriculum at the depart-
tional mission and identity. mental level. Indeed, it is the
Part I (Chapters 1 through 4) pre- committee’s conviction that similar
sents principles and research findings expectations and criteria can and should
that can support improvements in the apply to academic departments and
evaluation of undergraduate teaching in institutions as a whole. Chapters 6
STEM and reviews implementation and 7 provide practical strategies for

using the methodologies presented in pendix C includes examples of peer

Chapter 5 to evaluate individual teach- evaluation forms that are consistent
ers and departmental undergraduate with the findings and recommendations
programs, respectively. Finally, all of of this report. Finally, Appendix D
these findings serve as the basis for a provides biographical sketches of the
set of recommendations aimed at committee members.
improving evaluation practices, pre- This report also provides readers with
sented in Chapter 8. links to a wealth of additional informa-
Four appendixes also are provided. tion and guides available at numerous
Because student evaluations of teaching websites. These links are found prima-
occupy a place of prominence in current rily in footnotes or in the list of Refer-
evaluation processes, Appendix A ences. All of these links were tested
provides an in-depth examination of prior to the release of the report and
research findings on the efficacy and were found to be operable as of July 20,
limitations of input from undergraduate 2002.
students. Based on concerns of many
faculty about the design and analysis of Intended Audiences
student evaluations of teaching, colleges A primary audience for this report is
and universities across the United the individual STEM faculty members
States have begun to revise such forms. who teach disciplinary and interdiscipli-
Appendix B offers specific examples, nary courses at colleges and universi-
used by a variety of types of institutions ties, especially at the introductory level.
that comport with the six guiding This report also is directed to depart-
principles of this report; these examples mental and institutional leaders in
can serve as models for other institu- higher education, including college and
tions that are looking to revamp their university presidents and chancellors,
student evaluation forms. Similarly, as provosts, academic deans, and depart-
peer review of teaching gains greater ment chairs—those who can best
prominence in the instruments for both promote a culture and community of
formative and summative evaluations of teaching and learning and can encour-
teaching, faculty and administrators will age faculty collaboration in improving
require assistance on ways to undertake student learning and academic success.
this process fairly and equitably. Ap-

Annex Box 1-1. Seven Principles of Learning
Research in the cognitive, learning, and brain sciences has provided many new
insights about how humans organize knowledge, how experience shapes understand-
ing, how individuals differ in learning strategies, and how people acquire expertise.
From this emerging body of research, scientists and others have been able to synthe-
size a number of underlying principles of human learning. That knowledge can be
synthesized into the following seven principles of learning:
1. Learning with understanding is facilitated when new and existing knowledge is

structured around the major concepts and principles of the discipline.
Proficient performance in any discipline requires knowledge that is both accessible
and usable. Experts’ content knowledge is structured around the major organizing
principles, core concepts, and “big ideas” of the discipline. Their strategies for
thinking and solving problems are closely linked to their understanding of such core
concepts. Therefore, knowing many disconnected facts is not sufficient for developing
expertise. Understanding the big ideas also allows disciplinary experts to discern the
deeper structure and nature of problems and to recognize similarities between new
problems and those previously encountered. Curricula that emphasize breadth of
coverage and simple recall of facts may hinder students’ abilities to organize knowl-
edge effectively because they do not learn anything in depth, and thus are not able to
structure what they are learning around the major organizing principles and core
concepts of the discipline.
2. Learners use what they already know to construct new understandings.

College students already possess knowledge, skills, beliefs, concepts, conceptions,
and misconceptions that can significantly influence how they think about the world,
approach new learning, and go about solving unfamiliar problems. They often
attempt to learn a new idea or process by relating it to ideas or processes they already
understand. This prior knowledge can produce mistakes as well as new insights. How
these links are made may vary in different subject areas and among students with
varying talents, interests, and abilities. Learners are likely to construct interpretations
of newly encountered problems and phenomena in ways that agree with their own
prior knowledge even when those interpretations conflict with what a teacher has
attempted to teach. Therefore, effective teaching involves gauging what learners
already know about a subject and finding ways to build on that knowledge. When
prior knowledge contains misconceptions, effective instruction entails detecting those
misconceptions and addressing them, sometimes by challenging them directly.

3. Learning is facilitated through the use of metacognitive strategies that identify,

monitor, and regulate cognitive processes.
Metacognition is the ability of people to predict and monitor their current level of
understanding and mastery of a subject or performance on a particular task and
decide when it is not adequate (NRC, 2000e). Metacognitive strategies include (1)
connecting new information to former knowledge; (2) selecting thinking strategies
deliberately; and (3) planning, monitoring, and evaluating thinking processes. To be
effective problem solvers and learners, students need to reflect on what they already
know and what else they need to know for any given situation. They must consider
both factual knowledge—about the task, their goals, and their abilities—and strategic
knowledge about how and when to use a specific procedure to solve the problem at
hand. Research indicates that instructors can facilitate the development of
metacognitive abilities by providing explicit instruction focused on such skills, by
providing opportunities for students to observe teachers or other content experts as
they solve problems, and by making their thinking visible to those observing.
4. Learners have different strategies, approaches, patterns of abilities, and learning

styles that are a function of the interaction between their heredity and their prior
experiences.
Individuals are born with a potential to learn that develops through their interaction
with their environment to produce their current capabilities and talents. Among
learners of the same age, there are important differences in cognitive abilities (such as
linguistic and spatial aptitudes or the ability to work with symbolic representations of
the natural world), as well as in emotional, cultural, and motivational characteristics.
Thus, some students will respond favorably to one kind of instruction, whereas others
will benefit more from a different approach. Educators need to be sensitive to such
differences so that instruction and curricular materials will be suitably matched to
students’ developing abilities, knowledge base, preferences, and styles. Students with
different learning styles also need a range of opportunities and ways to demonstrate
their knowledge and skills. Using one form of assessment will work to the advantage
of some students and to the disadvantage of others; multiple measures of learning and
understanding will provide a better picture of how well individual students are learning
what is expected of them.
5. Learners’ motivation to learn and sense of self affect what is learned, how much
is learned, and how much effort will be put into the learning process.
Both internal and external factors motivate people to learn and develop competence.
Regardless of the source, learners’ level of motivation strongly affects their willingness
to persist in the face of difficulty or challenge. Intrinsic motivation is enhanced when
students perceive learning tasks as interesting and personally meaningful, and pre-
sented at an appropriate level of difficulty. Tasks that are too difficult can frustrate;
those that are too easy can lead to boredom. Research also has revealed strong
continued on next page

Annex Box 1-1. Continued

connections between learners’ beliefs about their own abilities in a subject area and
their success in learning that subject. For example, some students believe their ability
to learn a particular subject or skill is predetermined, whereas others believe their
ability to learn is substantially a function of effort. The use of instructional strategies
that encourage conceptual understanding is an effective way to increase students’
interest and enhance their confidence about their abilities to learn a particular subject.
6. The practices and activities in which people engage while learning shape what is
learned.
Research indicates that the way people learn a particular area of knowledge and
skills and the context in which they learn it become a fundamental part of what is
learned. When students learn some subject matter or concept in only a limited context,
they often miss seeing the applicability of that information to solving novel problems
encountered in other classes, in other disciplines, or in everyday life situations. By
encountering a given concept in multiple contexts, students develop a deeper under-
standing of the concept and how it can be used and applied to other contexts. Faculty
can help students apply subject matter to other contexts by engaging them in learning
experiences that draw directly upon real-world applications, or exercises that foster
problem-solving skills and strategies that are used in real-world situations. Problem-
based and case-based learning are two instructional approaches that create opportu-
nities for students to engage in practices similar to those of experts. Technology also
can be used to bring real-world contexts into the classroom.4
7. Learning is enhanced through socially supported interactions.

Learning can be enhanced when students have opportunities to interact and collabo-
rate with others on instructional tasks. In learning environments that encourage
collaboration, such as those in which most practicing scientists and mathematicians
work, individuals have opportunities to test their ideas and learn by observing others.
Research demonstrates that providing students with opportunities to articulate their
ideas to peers and to hear and discuss others’ ideas in the context of the classroom is
particularly effective in enhancing conceptual learning. Social interaction also is
important for the development of expertise, metacognitive skills (see learning principle
#3), and formation of the learner’s sense of self (see learning principle #5).
4
Specific techniques for structuring problem-based learning and employing technology in
college classrooms are discussed on the website of the National Institute for Science Education.
Suggestions for creative uses of technology are available <http://www.wcer.wisc.edu/nise/cl1/
ilt/default.asp>. Each site also provides further references. Additional resources on problem-
based learning are found in Allen and Duch (1998).
SOURCE: Excerpted and modified from NRC (2002b, Ch. 6). Original references
are cited in that chapter.

Annex Box 1-2. Overview of Research on Effective Assessment

of Student Learning
• Although assessments used in various contexts and for differing purposes often
look quite different, they share common principles. Assessment is always a process of
reasoning from evidence. Moreover, assessment is imprecise to some degree. Assess-
ment results are only estimates of what a person knows and can do. It is essential to
recognize that one type of assessment is not appropriate for measuring learning in all
students. Multiple measures provide a more robust picture of what an individual has
learned.
• Every assessment, regardless of its purpose, rests on three pillars: a model of
how students represent knowledge and develop competence in the subject domain,
tasks or situations that allow one to observe students’ performance, and an interpreta-
tion method for drawing inferences from the performance evidence thus obtained.
• Educational assessment does not exist in isolation. It must be aligned with
curriculum and instruction if it is to support learning.
• Research on learning and cognition indicates that assessment practices should
extend beyond an emphasis on skills and discrete bits of knowledge to encompass
more complex aspects of student achievement.
• Studies of learning by novices and experts in a subject area demonstrate that
experts typically organize factual and procedural knowledge into schemas that support
recognition of patterns and the rapid retrieval and application of knowledge. Experts
use metacognitive strategies to monitor their understanding when they solve problems
and perform corrections of their learning and understanding (see Annex Box 1-1,
principle 3, for additional information about metacognition). Assessments should
attempt to determine whether a student has developed good metacognitive skills. They
should focus on identifying specific strategies that students use for problem solving.
• Learning involves a transformation from naïve understanding into more complete
and accurate comprehension. Appropriate assessments can both facilitate this process
for individual students and assist faculty in revising their approaches to teaching. To
this end, assessments should focus on making students’ thinking visible to both them-
selves and their instructors so that faculty can select appropriate instructional strategies
to enhance future learning.
• One of the most important roles for assessment is the provision of timely and
informative feedback to students during instruction and learning so that their practice
of a skill and its subsequent acquisition will be effective and efficient.
• Much of human learning is acquired through discourse and interactions with
others. Knowledge is often associated with particular social and cultural contexts, and
it encompasses understanding about the meaning of specific practices, such as asking
and answering questions. Effective assessments need to determine how well students
engage in communicative practices that are appropriate to the discipline being

Annex Box 1-2. Continued

assessed. Assessments should examine what students understand about such practices
and how they use tools appropriate to that discipline.
• The design of high-quality assessments is a complex process that involves
numerous iterative and interdependent components. Decisions made at a later stage of
the design process can affect those occurring at an earlier stage. Thus, as faculty
develop assessments of student learning, they must often revisit their choices of ques-
tions and approaches and refine their designs.
• Although reporting of results occurs at the end of an assessment cycle, assess-
ments must be designed from the outset to ensure that reporting of the desired types of
information will be possible. Providing students with information about particular
qualities of their work and about what they can do to improve is crucial for maximiz-
ing learning.
• For assessment to be effective, students must understand and share the goals for
learning that are assessed. Students learn more when they understand and, in some
cases, participate in developing the criteria by which their work will be evaluated, and
when they engage in peer and self-assessment during which they apply those criteria.
Such practices also help students develop metacognitive abilities, which, in turn,
improve their development of expertise in a discipline or subject area.
SOURCE: Excerpted and modified from NRC (2001, pp. 2–9). References to
support these statements are provided in that report.

Characterizing and Mobilizing

Effective Undergraduate Teaching
In a recent address, Zimpher (1998) 6. Curriculum and program design

offered the following predictions: will be inseparable from teaching and
learning. Coordination, integration, and
teamwork will be hallmarks in the future.
1. Teaching will be more public than
7. Diversity will be seen as asset-
it ever has been before. It will be open
based. Higher education will realize that
to inspection, discussion, and increasing
all benefit when different perspectives
accountability.
and cultures are included.
2. The nature and quality of assess-
8. Different pedagogies that stu-
ment will change. Faculty will teach
dents have experienced prior to
within a culture of evidence that will place
college will change their expectations
great importance on demonstrating
about good teaching. They will come
learning outcomes.
with values for collaborative and active
3. Evaluation and documentation of
learning, and for contextual, experiential
teaching will change. It will be done
approaches, such as service learning.
more systematically and rigorously and
9. Higher education facilities will
will involve multiple methods and sources.
have to look different. Rooms will have
4. Teaching will become technologi-
to be flexible to accommodate the new
cally enabled. Instructional technology
pedagogies and they will have to be
will be used within the classroom as well
technologically sophisticated.
as for anytime, anyplace learning.
10. A new scholarship of teaching will
5. Content transmission will not be
occur. Value will be placed on systemati-
the focus of teaching. As information
cally exploring teaching issues and
continues to grow and be readily available
researching experiments with new
in many forms, the focus will be on
approaches and conditions affecting
helping learners to know how to access
information, evaluate it critically, and use student learning.
it to solve problems.
25

In light of these predictions, what steps dents who now enroll in STEM courses
are institutions of higher education and as undergraduates.
supporting organizations taking to Many individual faculty and depart-
mobilize faculty and resources to ments are actively engaged in moving
enhance learning for undergraduate undergraduate education from a faculty-
students? centered teaching model to a student-
Graduate students, faculty, and centered learning model (Barr and
administrators from all types of Tagg, 1999). Moreover, numerous
postsecondary institutions in the United campuses in the United States and
States are increasingly interested in the abroad are establishing teaching and
revamping of teaching practices to learning centers.1 As these centers
enhance student learning in science, evolve, they are supporting new
technology, engineering, and mathemat- pedagogies and more efficient methods
ics (STEM) (see Rothman and Narum, of assessing teaching and learning, and
1999). In part, this increased interest are serving as focal points for efforts to
has stemmed from observations by advance the scholarship of teaching and
faculty that their approaches to teaching learning (Boyer, 1990; Glassick et al.,
may not result in the expected levels of 1997; Ferrini-Mundy, personal commu-
student learning (e.g., Hestenes, 1987; nication). Many of these centers are
Hestenes and Halloun, 1995; Mazur, increasingly tailoring their assistance to
1997; Wright et al., 1998). Some faculty faculty to reflect differences in ap-
and departments are confronting the proaches and emphases among disci-
pedagogical and infrastructural chal- plines. Experts in these discipline-
lenges of offering smaller classes (e.g., based centers are often disciplinary
the need for additional instructors to faculty with expertise in pedagogical
teach more sections), especially for content knowledge, assessment of
introductory courses. Others are using learning, and other issues specific to
innovative approaches to teaching based their disciplines (see also Huber and
on emerging research in the cognitive Morreale, 2002).
and brain sciences about how people
learn (e.g., National Research Council
[NRC], 2000c). Still others are experi-
menting with the effectiveness of 1
A list of websites of teaching and learning
different learning strategies to accom- centers of colleges and universities in Asia,
modate the broader spectrum of stu- Australia and New Zealand, Europe, and North
America is available at <http://www.ku.edu/
~cte/resources/websites.html>.

Many of the professional organiza- teaching, as well as challenges faced by

tions and disciplinary societies with those seeking to become more effective
which university and college faculty instructors.
affiliate are making the improvement of
teaching and learning in undergraduate
STEM a component of their missions CHARACTERISTICS OF
and programs. Higher education EFFECTIVE TEACHING
organizations, government agencies,
and private foundations are sponsoring If teaching and student learning are to
workshops on student learning and improve, faculty and those who evaluate
supporting summer workshops on new them must recognize the characteris-
teaching methods. They are engaging tics of effective teaching. The research
graduate students in programs that can literature contains many examples of
better prepare them to become stimulat- successful standards and practices for
ing future faculty and encouraging effective teaching that are based on
faculty to present papers or posters on evidence of enhanced student learning
their teaching or research in education (e.g., Braskamp and Ory, 1994; Centra,
at professional meetings.2 These 1993; Davis, 1993: Lowman, 1995;
organizations also are publishing books, McKeachie, 1999; Neff and Weimer,
reports, and journal articles that ad- 1990; Perry and Smart, 1997; references
dress teaching and learning (e.g., Boyer in NRC 2000c, 2001, and 2002b). On the
Commission, 1998; Herron, 1996; Ireton basis of that literature, the committee
et al., 1996; Landis et al., 2001; National articulates five characteristics of effec-
Institute for Science Education, 2001c; tive teaching that can be used as a
NRC, 1991, 1995b, 1996b, 1997a, 1999a; starting point for improving teaching.
Uno, 1997). In Chapter 6, these characteristics are
The remainder of this chapter reviews elaborated as criteria that could serve as
the key characteristics of effective the basis for evaluating teaching effec-
tiveness.
1. Knowledge of Subject Matter
2
Although it appears obvious, any list
Examples are Microbiology Education,
published by the American Society of Microbiol- of characteristics of high-quality teach-
ogy; Journal of Chemical Education, published by ing of STEM that is centered on desired
the Division of Chemical Education of the
American Chemical Society; and Physics Today, student outcomes must begin with the
published by the American Institute of Physics. premise that faculty members must be
CHARACTERIZING AND MOBILIZING 27

well steeped in their disciplines. They • They can help students learn and
must remain active in their areas of understand the general principles of
scholarship to ensure that the content of their discipline (e.g., the processes and
their courses is current, accurate, and limits of the scientific method).
balanced, especially when presenting • They are able to provide students
information that may be open to alterna- with an overview of the whole domain of
tive interpretation or disagreement by the discipline (e.g., Coppola et al., 1997).
experts in the field. They also should • They possess sufficient knowledge
allow all students to appreciate “. . . and understanding of their own and
interrelationships among the sciences related sub-disciplines to answer most
and the sciences’ relationship to the students’ questions and know how to
humanities, social sciences, and the help students find appropriate informa-
political, economic, and social concerns tion.
of society” (NRC, 1999a, p. 26). • They stay current through an
Knowledge of subject matter can be active research program or through
interpreted in other ways. For example, scholarly reading and other types of
several recent reports (e.g., Boyer professional engagement with peers.
Commission, 1998; NRC, 1999a; Na- • They are genuinely interested in
tional Science Foundation [NSF], 1996) what they are teaching.
have emphasized that the undergradu- • They understand that conveying
ate experience should add value in the infectious enthusiasm that accompa-
tangible ways to each student’s educa- nies original discovery, application of
tion. Faculty must teach subject matter theory, and design of new products and
in ways that encourage probing, ques- processes is as important to learning as
tioning, skepticism, and integration of helping students understand the subject
information and ideas. They should matter.
provide students with opportunities to
think more deeply about subject matter 2. Skill, Experience, and
than they did in grades K–12. They Creativity with a Range of
should enable students to move intellec- Appropriate Pedagogies
tually beyond the subject matter at and Technologies
hand.
Faculty who possess deep knowledge Deep understanding of subject matter
and understanding of subject matter is critical to excellent teaching, but not
demonstrate the following characteris- sufficient. Effective teachers also
tics: understand that, over the course of their

educational experiences, undergradu- goals through discussions in class, out-

ates develop different strategies for of-class assignments, and other forms of
maximizing their individual abilities to assessment.
learn, reason, and think critically about • They have the ability to recognize
complex issues (King and Kitchener, students who are not achieving to their
1994; National Institute for Science fullest potential and then employ the
Education, 2001c; NRC, 1997a, 1999a). professional knowledge and skill neces-
To be most effective, teachers need to sary to assist them in overcoming
employ a variety of learning strategies academic difficulties.
and contextually appropriate
pedagogies3 that serve the range of Along with these characteristics, an
students’ learning styles (see, e.g., increasingly important component of
Annex Box 1-1, Chapter 1). Faculty who pedagogy is the appropriate use and
are effective in this regard demonstrate application of information technologies
the following characteristics: to enhance learning. Electronic net-
working, the Internet, remote sensing,
• They are organized and communi- distance learning, and databases and
cate clearly to students their expecta- digital libraries (e.g., NRC, 1998b,
tions for learning and academic achieve- 2000c; NSF, 1998)4 are changing funda-
ment. mentally the ways in which teaching and
• They focus on whether students learning take place in higher education.
are learning what is being taught and Although no one would suggest that top-
view the learning process as a joint quality instruction cannot be attained
venture between themselves and their without the use of networking re-
students. sources, instructional changes made
• They encourage discussion and possible through information technol-
promote active learning strategies (see ogy are profound and have already
Annex Box 1-1, Chapter 1). imbued research communities in the
• They persistently monitor students’ natural sciences, mathematics, and
progress toward achieving learning
4
For further discussion of digital libraries and
their importance in undergraduate STEM
education, see Borgman et al. (1996) and NRC
(1998b). NSF is now engaged in developing a
digital national library for undergraduate STEM
3
“Contextually appropriate pedagogies” is also education (additional information is available at
known in the research literature as “pedagogical <http://www.ehr.nsf.gov/ehr/due/programs/
content knowledge” (defined earlier in note). nsdl>.

engineering. Professional development • Assess learning in ways that are

can assist faculty in deciding whether consistent with the objectives of a
and how they might use these tools course and integrate stated course
most effectively for enhancing learning. objectives with long-range curricular
The role of information technology in goals.
undergraduate classrooms, laboratories, • Know whether students are learn-
and field environments is an important ing what is being taught. This requires
area for continued investigation (e.g., that faculty be persistent in collecting
American Association for Higher Educa- and analyzing assessments of student
tion [AAHE], 1996; Collis and Moonen, learning and committed to using the
2001; National Institute for Science data collected as a tool for improving
Education, 2001a). their own teaching skills (see, e.g.,
As information and other technolo- principle 5 in Astin et al., 1996).
gies become more pervasive in teaching • Determine accurately and fairly
and learning of the natural sciences, students’ knowledge of the subject
mathematics, and engineering, a faculty matter and the extent to which learning
member’s use of such resources is has occurred throughout the term (not
likely to become an increasingly impor- just at the end of the course).
tant component of teaching evaluations.
As with other areas of pedagogy in 4. Professional Interactions with
which college-level faculty have had Students Within and Beyond
little formal training or professional the Classroom
development, they will have to learn Teaching responsibilities extend
appropriate and effective uses of hard- beyond designing and offering courses.
ware and software that are coupled with Faculty are expected to direct original
new ways of viewing teaching and student research and involve students
learning. as collaborators in their own research,
advise and mentor students, participate
3. Understanding of and in departmental and campus curricular
Skill in Using Appropriate committees, and sometimes supervise
Assessment Practices teaching assistants. Students may also
In part, proficiency in assessment view their teachers as role models for
involves a faculty member’s skill in life as responsible, educated citizens.
evaluating student learning. This skill is For example, beyond helping students
evident when teachers: learn scientific principles or technologi-

cal processes, faculty can help them students clearly are not prepared to
open their eyes to the ethical issues and undertake the challenges of a particular
political decisions that often affect course, faculty should be able to counsel
science and technology (e.g., Coppola them out of the course or suggest
and Smith, 1996). alternative, individualized approaches
Professionalism in a faculty member’s for learning the subject matter.
relationships and interactions with • They uphold and model for stu-
students also should be based on dents the best scholarly and ethical
criteria such as the following: standards (e.g., University of California
Faculty Code of Conduct).5
• Faculty meet with all classes and
assigned teaching laboratories, post and 5. Involvement with and
keep regular office hours, and hold Contributions to One’s Profession
exams as scheduled. in Enhancing Teaching
• They demonstrate respect for and Learning
students as individuals; this includes Effective teaching needs to be seen as
respecting the confidentiality of informa- a scholarly pursuit that takes place in
tion gleaned from advising or student collaboration with departmental col-
conferences. leagues, faculty in other departments in
• They encourage the free pursuit of the sciences and engineering, and more
learning and protect students’ academic broadly across disciplines (Boyer, 1990;
freedom. Glassick et al., 1997; Kennedy, 1997).
• They address sensitive subjects or Faculty can learn much by working with
issues in ways that help students deal colleagues both on and beyond the
with them maturely. campus, thereby learning to better
• They contribute to the ongoing integrate the materials they present in
intellectual development of individual their own courses with what is being
students and foster confidence in the taught in other courses (Hutchings,
students’ ability to learn and discover on 1996; NRC, 1999a).
their own.
• They advise students who are
experiencing problems with course
material and know how to work them in
venues besides the classroom to help 5
The University of California System’s Faculty
them achieve. On those occasions when Code of Conduct Manual is available at <http://
www.ucop.edu/acadadv/acadpers/apm/>.

CHALLENGES TO Preparing Future Teachers

EFFECTIVE TEACHING
Scientists have an obligation to assist in
Faculty in the STEM disciplines face a science teachers’ professional development.

number of challenges in seeking to Many scientists recognize the obligation
become more effective teachers. Some
and are ready to get involved. Scientists
of these challenges are common to all
teaching and learning, while others are can provide opportunities for teachers to
more endemic to these disciplines. learn how the scientific process works, what
Some of the more general challenges scientists do and how and why they do it.
include improving the assessment of
They can provide research opportunities for
learning outcomes and preparing future
teachers. More discipline-specific practicing teachers; act as scientific part-
challenges include teaching a broad ners; provide connections to the rest of the
range and large numbers of students, scientific community; assist in writing grant
providing engaging laboratory and field
proposals for science-education projects;
experiences, and encouraging students
to undertake original research that provide hands-on, inquiry-based workshops
increasingly is highly sophisticated and for area teachers (e.g., NRC, 2000a); and
technical. provide teachers access to equipment,
Improving the Assessment of scientific journals, and catalogs not usually
Learning Outcomes available in schools. They can help teach-
The committee took particular note of ers to review educational material for its
Astin et al.’s (1996) Assessment Forum: accuracy and utility.
Nine Principles of Good Practice for
When scientists teach their undergradu-
Assessing Student Learning. Because
these authors articulate succinctly the ate classes and laboratories, potential
position the committee has taken in this science teachers are present. Scientists
report, their principles are presented should recognize that as an opportunity to
verbatim in Box 2-1. These principles
promote and act as a model of both good
also could be applied in evaluating
departmental programs. process and accurate content teaching and
so strive to improve their own teaching
(NRC, 1996c, p. 3).

Box 2-1. Nine Principles of Good Practice for Assessing

Student Learning
1. The assessment of student learning begins with educational

values. Assessment is not an end in itself but a vehicle for educational improvement.
Its effective practice, then, begins with and enacts a vision of the kinds of learning we
most value for students and strive to help them achieve. Educational values should
drive not only what we choose to assess but also how we do so. Where questions
about educational mission and values are skipped over, assessment threatens to be an
exercise in measuring what’s easy, rather than a process of improving what we really
care about.
2. Assessment is most effective when it reflects an understanding of

learning as multidimensional, integrated, and revealed in perfor-
mance over time. Learning is a complex process. It entails not only what students
know but what they can do with what they know; it involves not only knowledge and
abilities but values, attitudes, and habits of mind that affect both academic success and
performance beyond the classroom. Assessment should reflect these understandings
by employing a diverse array of methods, including those that call for actual perfor-
mance, using them over time so as to reveal change, growth, and increasing degrees
of integration. Such an approach aims for a more complete and accurate picture of
learning and therefore firmer bases for improving our students’ educational experience.
3. Assessment works best when the programs it seeks to improve

have clear, explicitly stated purposes. Assessment is a goal-oriented process.
It entails comparing educational performance with educational purposes and expecta-
tions—those derived from the institution’s mission, from faculty intentions in program
and course design, and from knowledge of students’ own goals. Where program
purposes lack specificity or agreement, assessment as a process pushes a campus
toward clarity about where to aim and what standards to apply; assessment also
prompts attention to where and how program goals will be taught and learned. Clear,
shared, implementable goals are the cornerstone for assessment that is focused and
useful.
4. Assessment requires attention to outcomes but also and equally

to the experiences that lead to those outcomes. Information about out-
comes is of high importance; where students “end up” matters greatly. But to improve
outcomes, we need to know about student experience along the way—about the
curricula, teaching, and kind of student effort that lead to particular outcomes. Assess-
ment can help us understand which students learn best under what conditions; with
such knowledge comes the capacity to improve the whole of their learning.

Box 2-1. Continued
5. Assessment works best when it is ongoing not episodic. Assess-

ment is a process whose power is cumulative. Though isolated, “one-shot” assessment
can be better than none, improvement is best fostered when assessment entails a linked
series of activities undertaken over time. This may mean tracking the progress of
individual students, or of cohorts of students; it may mean collecting the same ex-
amples of student performance or using the same instrument semester after semester.
The point is to monitor progress toward intended goals in a spirit of continuous
improvement. Along the way, the assessment process itself should be evaluated and
refined in light of emerging insights.
6. Assessment fosters wider improvement when representatives

from across the educational community are involved. Student learning is
a campus-wide responsibility, and assessment is a way of enacting that responsibility.
Thus, while assessment efforts may start small, the aim over time is to involve people
from across the educational community. Faculty play an especially important role, but
assessment’s questions can’t be fully addressed without participation by student-affairs
educators, librarians, administrators, and students. Assessment may also involve
individuals from beyond the campus (alumni/ae, trustees, employers) whose experi-
ence can enrich the sense of appropriate aims and standards for learning. Thus
understood, assessment is not a task for small groups of experts but a collaborative
activity; its aim is wider, better informed attention to student learning by all parties with
a stake in its improvement.
7. Assessment makes a difference when it begins with issues of use

and illuminates questions that people really care about. Assessment
recognizes the value of information in the process of improvement. But to be useful,
information must be connected to issues or questions that people really care about.
This implies assessment approaches that produce evidence that relevant parties will
find credible, suggestive, and applicable to decisions that need to be made. It means
thinking in advance about how the information will be used, and by whom. The point
of assessment is not to gather data and return “results”; it is a process that starts with
the questions of decision-makers, that involves them in the gathering and interpreting
of data, and that informs and helps guide continuous improvement.
8. Assessment is most likely to lead to improvement when it is part

of a larger set of conditions that promote change. Assessment alone
changes little. Its greatest contribution comes on campuses where the quality of
teaching and learning is visibly valued and worked at. On such campuses, the push to
improve educational performance is a visible and primary goal of leadership; improv-

ing the quality of undergraduate education is central to the institution’s planning,

budgeting, and personnel decisions. On such campuses, information about learning
outcomes is seen as an integral part of decision making, and avidly sought.
9. Through assessment, educators meet responsibilities to students

and to the public. There is a compelling public stake in education. As educators,
we have a responsibility to the public that supports or depends on us to provide
information about the ways in which our students meet goals and expectations. But
that responsibility goes beyond the reporting of such information; our deeper obliga-
tion—to ourselves, our students, and society—is to improve. Those to whom educators
are accountable have a corresponding obligation to support such attempts at improve-
ment.
SOURCE: Astin et al. (1996); see <http://www.aahe.org/principl.htm>.
This committee agrees with the to the world economy. Standards for
conclusions expressed by other NRC teacher education and professional
committees (NRC 1999a, 2000b) that development for teachers are an integral
science faculty in the nation’s universi- component of the National Science
ties should, as one of their primary Education Standards (NRC, 1996a);
professional responsibilities, model the much useful information can be found in
kinds of pedagogy that are needed to that document to help postsecondary
educate both practicing and prospective faculty understand their role in promot-
teachers. Those NRC reports provide a ing more effective teacher education.
series of recommendations for how Contributing authors in Siebert and
chief academic officers and faculty can Macintosh (2001) offer advice and
work together to promote more effec- numerous examples of how the prin-
tive education for teachers of mathemat- ciples contained in the National Science
ics and science. These recommenda- Education Standards can be applied to
tions include developing courses that higher education settings.
provide all students with a better under- An impending shortage of qualified
standing of the relationships among the K–12 teachers over the next decade
sciences, that integrate fundamental (National Center for Education Statis-
science and mathematics, and that help tics, 1999) will compound the shortage
students understand how these areas of that already exists for elementary and
knowledge relate to their daily lives and secondary school science and math-

ematics teachers. It should be noted special obligation to plan and conduct

that impending teacher shortages do their courses in ways that make these
not apply only to K–12 education. disciplines relevant to the wide range of
Declining graduate student enrollments students who now enroll in them and
in some disciplines suggest that having often constitute the majority of students
enough people who are qualified to in lower division courses (Greenspan,
teach undergraduate students, including 2000; NRC, 1999a). As numerous
those who may go on to become K–12 reports have suggested, this responsi-
teachers, may become problematic in bility applies equally to academic de-
the future (e.g., Lovitts and Nelson, partments (NRC, 1996b, 1999a; NSF,
2000). 1996; Project Kaleidoscope, 1995).
Even if the number of graduate Courses and programs offered to
students were to remain sufficient, it is nonmajors in STEM can be very differ-
important to recognize that most col- ent from similar courses and programs
lege-level faculty who currently teach in in other disciplines. Introductory
the STEM disciplines have never courses and programs (and sometimes
received formal preparation for teaching more advanced courses) in the social
any students, let alone those who aspire sciences and humanities typically are
to be teachers at either the precollege geared toward any student who wishes
or university level. Institutions of to enroll in them. For mathematics and
higher education need to develop science, however, departments and
collaborative strategies for addressing institutions sometimes insist on offering
this problem (Gaff et al., 2000; NRC separate introductory courses for
2000b). prospective majors and nonmajors. In
too many instances, faculty and depart-
Teaching a Broad Range and ments view the offerings for nonmajors,
Large Numbers of Students especially at the introductory level, as
As science and technology play ever “service courses” that may impose
more pervasive roles in society, it is additional staffing and resource de-
imperative that all students, not just mands not found in other sectors of the
those planning careers in these fields, university. As a result, many of these
develop an appreciation for and under- courses for nonmajors (and in some
standing of these subjects. This under- cases, those for majors as well) tend to
standing must involve more than knowl- have large numbers of students enrolled
edge of some specific set of content. and are offered in large lecture halls.
Faculty in the STEM disciplines have a These kinds of facilities do not conform

with the design of classroom and labora- Providing Engaging Laboratory

tory space that has been recommended and Field Experiences
for optimal teaching and learning by A number of national commissions
undergraduates (e.g., Project Kaleido- and organizations have emphasized the
scope, 1995). Accumulating evidence importance of laboratory-rich teaching
suggests that nonmajors often fare environments for undergraduates in the
better in smaller courses and inquiry- natural sciences (NSF, 1996; Project
based laboratory experiences where Kaleidoscope, 1991, 1994, 1998). Large
they become actively engaged with the amounts of time are needed to organize
subject matter.6 Constraints on staff and oversee teaching laboratories and
and limited financial resources may field experiences for undergraduates.
preclude science departments from Providing such experiences also re-
offering these kinds of experiences, quires effective programs to train
however. Such limitations may lead graduate or undergraduate teaching
students to become disenfranchised, assistants. If the laboratory experience
and the students may evaluate the is tied to a specific course, instructors
courses and the instructors that teach also must commit time and effort to
them accordingly. integrating the laboratory exercises
with classroom work or to organizing
the laboratory in ways that provide
students with learning experiences not
6
Recent reports suggest that at least some covered in class. Teaching laboratories
barriers and limitations can be overcome by that are independent from other courses
emphasizing inquiry-based approaches to
learning during classroom instruction (e.g.,
(e.g., a technical skills laboratory) must
Ebert-May et al., 1997). As defined by the respond to needs of both students and
National Science Education Standards, “Inquiry
instructors in other courses. The
is a multifaceted activity that involves making
observations; posing questions; examining books exercise or experiment selected should
and other sources of information to see what is be appropriate for the topic at hand.
already known; planning investigations; review-
ing what is already known in light of experimen- The design and execution of laboratory
tal evidence; using tools to gather, analyze and work, especially in courses with large
interpret data; proposing answers, explanations,
and predictions; and communicating the results. numbers of students, also must empha-
Inquiry requires identification of assumptions, size safety and reflect consideration of
use of critical and logical thinking, and consider-
ation of alternative explanations” (NRC, 1996a, p. potential impacts on the local environ-
23). Additional detail on inquiry-based ap- ment.
proaches to teaching and learning (focused on
grades K–12 but applicable in many ways to
higher education) can be found in NRC (2000a).

If laboratory and field experiences projects. Supervising faculty members

(particularly in introductory courses may need to spend large amounts of
and courses for nonscience majors) are time working with students to introduce
to become integral components of them to the relevant literature, to use
undergraduate science and engineering appropriate instrumentation and re-
education, the effectiveness with which search protocols, and to understand
these additional demands for teaching laboratory safety protocols.
are met should be specifically recog- It also is important for faculty supervi-
nized and evaluated in the reviews of sors to help undergraduates grow as
individual instructors. Appropriate researchers. Part of that supervision
professional development should be should include providing the training
made available to those faculty not and experiences that all undergraduate
familiar with inquiry-based laboratory students need to learn effective commu-
experiences, who as a consequence may nication skills that ultimately will allow
not have structured laboratory and field them to publish successfully in the
experiences to meet this important scholarly literature or to deliver an
learning objective. appropriate presentation to colleagues.
Failure is a routine part of research, and
Engaging Students in students should be allowed to experi-
Original Research ence it as appropriate.
“Education through research” is Students also should be given greater
becoming an increasingly popular and responsibility for overseeing projects
effective way for undergraduates to and for working with other students as
learn about science firsthand. With the they demonstrate increasing maturity
increasing emphasis on engaging and research prowess. Involving
undergraduates in original or applied graduate students or senior undergradu-
research, the one-on-one mentoring that ate students as cosupervisors of projects
takes place in supervised undergraduate can provide important and effective
research is one of the best predictors of introductory training for those who
students’ professional success (e.g., ultimately will seek teaching positions.
Doyle, 2000; NRC, 1999a; NSF, 1996). It also may entail the integration of
To properly oversee and mentor undergraduate research projects with
undergraduate students who undertake those of graduate students or
original research, faculty must have postdoctoral fellows working in closely
sufficient time to help students appreci- related areas in the laboratory or the
ate the scope and significance of their field.

Supervision of undergraduate re- Limitations on Faculty

search should be viewed positively Knowledge of Research on
when evaluating a faculty member’s Effective Teaching
teaching and research. This is espe- Given all of the above challenges,
cially the case if a student’s work merits faculty in STEM who teach undergradu-
publication as a coauthor in the original ates could benefit greatly from practical
literature or in a presentation at a guidance regarding techniques for
professional conference. improving learning among diverse
It is critical for faculty and administra- undergraduate student populations.
tors to understand that the criteria for The scholarly literature and an increas-
evaluating teaching in these environ- ing number of websites now provide this
ments may be very different than is the kind of assistance (see, e.g., Project
case for more traditional classroom or Kaleidoscope <http://www.pkal.org> or
laboratory situations. Department- or the National Institute for Science Educa-
institution-wide instruments for evaluat- tion <http://wcer.wisc.edu/nise>).
ing and comparing teaching quality However, many faculty never were
across disciplines may not reflect the introduced to this knowledge base
different kinds of preparation and during their graduate or postdoctoral
presentation that are required for these years and have not acquired this per-
kinds of activities in the natural sciences spective. These instructors may
and engineering. Thus, efforts should struggle through teaching assignments,
be made to adopt or adapt some of the often redeveloping techniques and
newer instruments that are more approaches that others already have
appropriate for these kinds of teaching. tested and disseminated.

Aligning the Cultures of Research and

Teaching in Higher Education
In calculating academic rewards, it has been painfully difficult to evaluate the quality of
research as separated from its mass. Nevertheless, departments and deans find that for
passing judgment on peers, research productivity is a much more manageable criterion than
teaching effectiveness. Faculty gossip, student evaluations, and alumni testimonials have all
been notoriously weak reeds, and reliable self-evaluation is all but impossible…. At this
point promotion and tenure committees still find teaching effectiveness difficult to measure.
Publication is at least a perceptible tool; the relative ease of its use has reinforced the
reliance on it for tenure and promotion decisions. Evaluating good teaching will always be
difficult, but effective integration of research and teaching should be observable, as should
the development of interdisciplinary approaches to learning. Departments and deans must
be pressed to give significant rewards for evidence of integrated teaching and research and
for the imagination and effort required by interdisciplinary courses and programs. When
publication is evaluated, attention should be paid to the pedagogical quality of the work as
well as to its contribution to scholarship.
Boyer Commission on Educating Undergraduates in the Research University (1998, p. 41)
40

Both within and outside higher research scholar’s work. Serving as

education, the perception (and too often anonymous reviewers for granting
the reality) is that at many colleges and agencies and professional journals,
universities, research productivity is these referees are the main source of
valued more than teaching effectiveness formal critical feedback to researchers.
(e.g., Bleak et al., 2000; Boyer Commis- Less formally, researchers are assessed,
sion on Educating Undergraduates in and assess themselves, when they take
the Research University, 1998; Gray et advantage of their many opportunities to
al., 1996; Rice et al., 2000). At other share ideas and learn from colleagues in
kinds of institutions, such as community their own or other institutions. Home
colleges and some liberal arts institu- institutions bask in the reflected glory
tions and comprehensive universities, of their most distinguished research
teaching is considered paramount, and faculty. In turn, institutions often
the evaluation of teaching and learning provide them with perquisites such as
has received greater attention. Even in endowed positions; additional research
some of these schools, however, the support; laboratory space; higher
increased availability of public and salaries; and few or no other responsi-
private funds for research has shifted bilities, including teaching and advising
this priority such that some faculty may of undergraduate students. On the
question whether effective teaching is other hand, researchers who fail to
valued as highly in their institutions as it produce or who become unproductive
has been in the past. may lose institutional support, are given
This gap can be attributed both to the diminished space in which to work, are
ways in which research is sponsored assigned fewer student assistants, or are
and to the importance ascribed to denied tenure or promotion.
scholarship that emphasizes discovery In contrast to the well-established
of new knowledge, application of that norms for scientific research, many
knowledge through technology transfer, colleges and universities rely heavily on
or impact on regional economic growth. faculty initiative to nurture and sustain
There also is a perceived difference in improvement of teaching and learning.
objectivity and credibility between the Although criteria for assessing perfor-
evaluation of research productivity and mance in the research arena are well
that of teaching effectiveness. established relative to those for assess-
In the world of research, peers who ing performance in teaching, the com-
work in closely related areas are the mittee agrees with Boyer’s (1990)
rigorous evaluators of the quality of a contention that teaching in higher
ALIGNING THE CULTURES 41

education has many parallels with the e.g., Annex Box 1-1, Chapter 1), criteria
research enterprise. The products of and methods for assessing undergradu-
sound teaching are effective student ate teaching performance in accordance
learning1 and academic achievement. with that emerging knowledge have not
The major challenge for colleges and yet seen widespread use. Instead, the
universities is to establish as an institu- measure of a teacher’s effort often is
tional priority and policy the need for reduced to the numbers of courses or
both individual and collective (i.e., laboratory sections he or she teaches,
departmental) responsibility and ac- the numbers of students taught, or
countability for improving student grade distributions. These are not
learning. As this report demonstrates, measures of outcomes and results. End-
criteria and methodologies for assessing of-course student evaluations are com-
teaching effectiveness and productivity mon, but even they usually lead to a
in ways that are comparable with the numeric ranking, which often confuses
measurement of productivity in scholar- evaluation of the teacher and the course.
ship are becoming increasingly available Because many factors, such as the size
(e.g., Gray et al., 1996; Licata and of the course, its grade distributions, or
Morreale, 1997, 2002; National Institute whether it is being taken as an elective
of Science Education, 2001b). Many of or distribution requirement can influ-
these criteria and methods are exam- ence responses on such evaluations (see
ined in Part II of this report. Chapter 4), rankings are rarely directly
While we now know a great deal more comparable among courses or instruc-
about practices that can contribute to tors.
effective teaching and learning (see, The committee maintains that the
goals and perception of excellence in
research and teaching at the under-
graduate level can and must become
1
There are numerous definitions of what more closely aligned. Five key areas in
constitutes effective student learning. For
purposes of this report, the committee has which steps can be taken to this end are
adopted the definition from the NRC report How (1) balancing the preparation provided
People Learn: Brain, Mind, Experience, and
School: Expanded Edition (National Research
for careers in research and teaching; (2)
Council [NRC], 2000c, p. 16): “To develop increasing support for effective teaching
competence in an area of inquiry, students must
on the part of professional organiza-
(a) have a deep foundation of factual knowledge,
(b) understand facts and ideas in the context of a tions; (3) developing and implementing
conceptual framework, and (c) organize knowl- improved means for evaluating under-
edge in ways that facilitate retrieval and applica-
tion.” graduate teaching and learning; (4)

according greater stature to the intellec-

[T]here are many kinds of good teaching,
tual challenge of the scholarship of
learning and teaching for those faculty in many kinds of teaching situations, at
in the sciences, technology, engineer- many different levels. Attempts to reduce it

ing, and mathematics (STEM) who wish to a formula are doomed to failure. There
to pursue such objectives; and (5)
will always be teachers who will break all
recognizing and rewarding those faculty
who pursue such scholarship. our rules and yet be profoundly successful.
In other words, it is the good teacher, not
teaching in the abstract, that counts.
BALANCING PREPARATION FOR
Goheen (1969, p. 80)
CAREERS IN RESEARCH
AND TEACHING
Faculty advisors mentor most gradu- or serving on the dissertation commit-

ate students in science and technology tees of colleagues’ advisees.
in U.S. universities in their selection of In the postgraduate years, when
coursework, choice of research topics, young researchers assume faculty
and research progress. During this positions, they are expected to establish
period, students are encouraged to an independent line of inquiry quickly
participate in professional meetings and and to make significant progress,
conferences where they can present generally within 6 years. The pressure
their findings, receive suggestions on to produce creditable results at many
their work, and learn about new devel- universities and a growing number of
opments in their field. The expectation smaller colleges is extreme (e.g., Rice et
that as researchers, they will interact al., 2000), but young researchers in the
with and learn from colleagues around natural sciences and engineering
the country and the world is ingrained generally can count on a considerable
from the start. Also conveyed to stu- support structure provided by their
dents during the graduate school and home institutions, departments, and
postdoctoral years is the expectation more senior colleagues. Such support
that other members of the research can include generous start-up funds,
community will contribute time and reduced expectations for teaching and
intellectual effort to assist them in their committee work during the pretenure
research efforts by, for example, review- years, and nominations for awards and
ing manuscripts and grant applications for invitations to professional meetings.

In contrast to the more formalized Moreover, because the focus of

preparation for research, many new graduate education is productivity in
faculty who are expected to teach independent research, graduate stu-
undergraduates in the sciences and dents may view negatively the time they
engineering have little training in or spend teaching, or at least assume that
exposure to the craft of teaching and their faculty advisors regard this time as
virtually no experience with the emerg- reducing research productivity. The
ing culture of teaching and learning comments from one graduate student
communities. Depending on the needs cited by Nyguist et al. (1991, p. 2) are
of their graduate institution and its telling:
sources of funding, new faculty mem-
I think any research advisor in their right
bers may have taught an undergraduate mind would kill me for [seeking addi-
laboratory, recitation, or course when tional teaching assistant opportunities].
they were graduate students. They also It’s certainly not something I would do.
may have assisted a course instructor It’d be ludicrously unfair to a professor—
by grading examinations, laboratory to the professor that you are working
for—to seek out another teaching assis-
reports, and other papers. While many
tantship. You are literally robbing them
faculty mentors do offer graduate
of thousands of dollars of effective
teaching assistants helpful formative research. It would almost be stealing
feedback on their teaching (especially in from your employer to do that. The
their roles as laboratory instructors), professor depends on the graduate
the broader paradigms of teaching and students because the graduate students
learning, such as appropriate content, do all of the work in the lab. Not a whole
lot of people tend to volunteer [their
effective pedagogy, and the ways stu-
graduate assistants as teaching assis-
dents learn (e.g., NRC, 1997a, 1999b)
tants] because it would mean sacrificing
often are not discussed in depth (Gaff et their own careers.
al., 2000; Golde and Dore, 2001; Reis,
Thus, implicit messages about the
1997). In addition, the pressures to
importance of preparing to become an
pursue research actively make it diffi-
effective teacher are often conveyed to
cult for many graduate teaching assis-
graduate students and postdoctoral
tants to become acquainted with the
fellows even before they vie for posi-
extensive body of educational research
tions in academe. These messages
that could guide them as they assume
continue beyond graduate school. Job
independent faculty positions (e.g., NRC
announcements may precisely specify
2000b, 2001, 2002a).
research qualifications and areas of

expertise while referring only obliquely mentor the next generation of faculty in
to qualifications for teaching. During new pedagogies or in the use of tech-
interviews, candidates for positions niques for effectively assessing student
usually are required to present in learning. For many faculty, their most
colloquia or other venues details on successful instructional methods are
their current interests, achievements, usually self-taught—a reflection at least
and future plans for research, but may in part of the ways they themselves
not be asked to demonstrate either were taught—and consistent with
teaching prowess or knowledge of personal styles and areas of expertise.
critical teaching and learning issues in Such methods are not necessarily
STEM education. Orientation for new transferable to student assistants or less-
faculty, if it exists at all, is often com- senior colleagues. Moreover, teaching
pleted within a few days prior to the as modeled by faculty advisors has been
beginning of the academic year. During based primarily on the lecture, to the
orientation or earlier, new faculty may point that the unstated assumption of
learn of the existence of a teaching and graduate or postdoctoral students could
learning center on campus, which can very well be that this is the only “real”
provide access to resources that would form of teaching. While lectures may
be useful for development and refine- be an effective method when used by
ment of their teaching skills. Even certain faculty in certain settings, a mix
when such centers exist,2 however, of pedagogies is likely to be more
faculty may or may not be encouraged successful, particularly for the broader
to use their services. spectrum of students that now charac-
Indeed, many faculty in the STEM terizes the nation’s undergraduate
disciplines who teach undergraduates population (Cooper and Robinson, 1998;
are unfamiliar with the burgeoning McKeachie, 1999; McNeal and
research on education and human D’Avanzo, 1997; Shipman, 2001;
learning. This lack of knowledge and Springer et al., 1998; Wyckoff, 2001).
awareness leaves them ill equipped to Senior colleagues could serve as
sources of teaching support, advice, and
feedback for new faculty, but those new
faculty may be reluctant to initiate such
2
a relationship for several reasons. One
Teaching and learning centers on many
campuses are providing leadership in addressing is the tradition of academic freedom, in
these issues. A list of these centers around the which classrooms are viewed as private
world can be found at <http://www.ku.edu/~cte/
resources/websites.html>. domains where faculty members have

the freedom to conduct their courses as INCREASING SUPPORT FOR

they deem appropriate. Less-experi- EFFECTIVE TEACHING BY
enced faculty also may be reluctant to PROFESSIONAL ORGANIZATIONS
share their ideas and concerns about
teaching and learning because they fear Dozens of professional societies and
exposing their pedagogical naiveté or umbrella or multidisciplinary organiza-
missteps to those who may later evalu- tions are devoted to the support and
ate their suitability for tenure and improvement of research. Far fewer
promotion. Such reluctance to seek organizations exist whose primary focus
feedback and advice may be especially is the improvement of teaching and
pronounced should a new faculty learning in STEM, especially for under-
member be experimenting with alterna- graduate students. Most of these
tive approaches to teaching and learning organizations have the potential to
that may appear suspect to faculty influence positively their members’
colleagues. In turn, senior faculty may recognition that teaching can be a
be reluctant to sit in on the courses of scholarly endeavor parallel to research
less experienced colleagues because in the discipline.
they lack the time to do so or believe In the past 10 years, however, disci-
their presence could interfere with plinary societies and organizations have
those colleagues’ abilities to conduct the shown increased interest in finding
classes as they see fit. ways to assist their membership in
Research universities are recognizing improving undergraduate teaching and
this problem and increasingly are learning. For more than a decade, for
developing programs to help graduate example, the research-based American
and postdoctoral students in the art and Mathematical Society and the Society
craft of teaching. The availability of for Industrial and Applied Mathematics
such programs in the natural sciences, have worked closely with mathematics
however, currently lags behind that in education organizations, such as the
other disciplines (Golde and Dore, Mathematical Association of America,
2001). the National Council of Teachers of
Mathematics, and the American
Mathematics Association of Two Year
Colleges. Together they have examined
mathematics curricula and standards for
learning for grades K–14. Likewise, the
American Chemical Society offers

extensive resources for undergraduate improving undergraduate education in

chemistry education and has produced the sciences and mathematics that
a textbook and supporting materials for preceded its annual meeting.6 In 2001,
students not planning to major in chem- the American Institute of Physics
istry.3 And the American Physical published a compendium of papers from
Society sponsors regular meetings of a symposium it had sponsored on the
department chairs where issues sur- role of physics departments in prepar-
rounding undergraduate physics educa- ing K–12 teachers (Buck et al., 2000).7
tion are discussed.4 Foundations also have assigned
Other professional societies also are greater importance to learning out-
beginning to examine their role in comes. The Carnegie Foundation for
supporting the improvement of under- the Advancement of Teaching recently
graduate education. In 1996, for ex- released a new “Millennial Edition”
ample, the American Geophysical Union classification system for American
produced the report Shaping the Future higher education institutions, which
of Undergraduate Earth Science Educa- places greater emphasis on teaching
tion, which advocates an “earth sys- and service after a decades-long focus
tems” approach to teaching and learning on research productivity and the num-
(Ireton et al., 1996). In 1999, the Ameri- ber of doctoral degrees awarded
can Institute for Biological Sciences (Basinger, 2000; McCormick, 2001).8
sponsored a summit of presidents from The Council for the Advancement and
its 63 member organizations to consider Support of Education, in collaboration
comprehensive approaches to improv- with the Carnegie Foundation for the
ing undergraduate education in the life Advancement of Teaching,9 gives
sciences.5 In November 1999, Sigma Xi faculty from higher education institu-
convened a three-day conference on tions national recognition for excellence
3
Additional information about this program is
6
available at <http://www.acs.org/portal/ Additional information about this convocation
Chemistry?PID=acsdisplay.html&DOC=education/ is available at <http://www.sigmaxi.org/forum/
curriculum/context.html>. 1999Forum/forum99.htm>.
4 7
See, for example, Undergraduate Education Additional information about this symposium
in Physics: Responding to Changing Expecta- is available at <http://www.sigmaxi.org/forum/
tions <http://www.aps.org/educ/conf97/ 1999Forum/forum99.htm>.
8
01.Chairs.homepage.html>. This new classification system is available at
5
Additional information is available at <http:// <http://www.carnegiefoundation.org/Classifica-
alidoro.catchword.com/vl=85083249/cl=13/ tion/index.htm>.
9
nw=1/rpsv/catchword/aibs/00063568/v50n3/ Additional information is available at <http:
s13/p277l>. //www.carnegiefoundation.org/>.

in undergraduate teaching.10 The Public and private funding organiza-

American Association for Higher Educa- tions have begun to stress the role of
tion (AAHE) sponsors an Assessment assessment in improving undergraduate
Forum, designed to promote “…effec- teaching and learning. For example, the
tive approaches to assessment that National Science Foundation (NSF)
involve faculty, benefit students, and recently instituted an initiative for
improve the quality of teaching and Assessment of Student Achievement in
learning. It helps campuses, programs, Undergraduate Education. This pro-
and individuals to plan, implement, and gram supports the development and
share the results of their assessment dissemination of assessment practices,
efforts by publishing, networking, and materials, and metrics designed to
sponsoring an annual national confer- improve the effectiveness of under-
ence” (e.g., Cambridge, 1997; Suskie, graduate courses, curricula, programs
2000).11 AAHE also has published a of study, and academic institutions in
directory of some 300 assessment promoting student learning in STEM.13
books and articles, journals, newslet- The Pew Charitable Trust has sup-
ters, audiocassettes, organizations, ported several efforts to make public
conferences, and electronic resources what undergraduates are learning at the
such as listservs and websites (Gardiner nation’s colleges and universities.14 The
et al., 1997). Another important source Howard Hughes Medical Institute,
of exemplary success stories is Project which has contributed more than $475
Kaleidoscope’s Programs That Work. million toward improving undergradu-
Project Kaleidoscope has collected a ate and K–12 education in the sciences
large body of information from a wide since 1988, has begun to compile and
variety of postsecondary institutions will share on a website information
about innovative practices for the about the various kinds of assessments
improvement of teaching, curriculum, being used by its grantees to demon-
and institutionalization of reform.12
10
Additional information about this prize is
available at <http://www.case.org/awards>.
11
Additional information about this forum and
its related activities is available at <http:// 13
Additional information about this NSF
www.aahe.org/assessment/>.
12 initiative is available at <http://www.ehr.nsf.gov/
Additional information about the Project
ehr/DUE/programs/asa/>.
Kaleidoscope program, including specific case 14
Additional information is available at
studies and publications that are available in print
<http://www.pewtrusts.com/ideas/
and on the organization’s website, are available at
index.cfm?issue =22>.
<http://www.pkal.org>.

strate increases in student learning and for undergraduate education in this

greater teaching effectiveness.15 discipline that can be applied to all
Some professional accrediting organi- students who enroll in psychology
zations and disciplinary societies also courses.18 A task force established by
are becoming involved with efforts to APA’s Board of Scientific Affairs has
improve undergraduate education developed a set of guidelines for “under-
within their disciplines. Beginning in graduate psychology competencies”
2001, engineering programs will be (APA, 2002).
subject to new criteria for accreditation The committee applauds the efforts of
established by the Accreditation Board professional and disciplinary organiza-
for Engineering and Technology tions in helping members recognize
(ABET).16 These outcome-based their roles and responsibilities for
standards include a call for engineering improving undergraduate education and
programs to demonstrate that their in offering sessions about how to do so.
graduates have the necessary knowl- However, these groups could contribute
edge and skills to succeed in the profes- significantly to efforts aimed at improv-
sion. To help member institutions ing teaching and learning if they were
prepare to meet these new expectations, also to convene serious discussions
ABET began holding conferences on addressing the broader issues and
Outcomes Assessment for Program conflicts that serve as barriers to those
Improvement and now sponsors annual efforts, such as allocation of faculty
national conferences on this issue.17 time, expectations for professional
Similarly, in 1991 the American Psycho- advancement, and recognition and
logical Association (APA) drafted a set rewards.
of voluntary, outcome-based standards
15
Additional information about the
organization’s increasing emphasis on examining
and disseminating new ideas about assessment is
available at <http://www.hhmi.org/grants/
undergraduate/assessment/>.
16
Additional information is available at <http://
www.abet.org/accreditation.html>.
17 18
Additional information about the ABET APA’s Principles for Quality Undergraduate
conferences is available at <http:// Psychology Programs is available at <http://
www.abet.org/annual_meeting_cover.html>. www.apa.org/ed/stmary.html>.

DEVELOPING AND ments and institutions. Evidence for

IMPLEMENTING IMPROVED effective teaching will need to be
MEANS FOR EVALUATING coupled with greater recognition and
EFFECTIVE TEACHING rewards for teaching by peers, academic
AND LEARNING departments, and institutions of higher
education (Bleak et al., 2000; Boyer,
Finally, if teaching and learning are to 1990; Glassick et al., 1997; Joint Policy
improve, a broader array of equitable Board on Mathematics, 1994). Part II of
and acceptable ways must be found to this report provides more specific
evaluate faculty teaching on the basis of guidance on criteria and methods for
evidence of student learning. The developing effective evaluations for both
issues involved here go far beyond the individual faculty members and aca-
individual faculty member; they also demic departments.
reach deeply into academic depart-

4
Evaluating Teaching in
Science, Technology,
Engineering, and Mathematics:
Principles and Research Findings
Every department, college, and GENERAL PRINCIPLES AND

university is unique, and thus no one OVERALL FINDINGS
model for evaluating teaching effective-
ness that is based on learning outcomes The research literature suggests that
will be appropriate for all institutions. for purposes of any formative or
Nonetheless, if effective methodologies summative evaluation,1 assessment that
for evaluating teaching and student is based on a single teaching activity
learning are to be implemented, admin- (e.g., classroom presentation) or de-
istrators and senior faculty must be- pends on information from a single
come more aware of emerging research source (e.g., student evaluation forms)
on effective practices. Knowledge of is less reliable, useful, and valid than an
this work is particularly important at the assessment of an instructor’s strengths
departmental level, where the evalua- and weaknesses that is based on mul-
tion of individual faculty members tiple sources (Centra, 1993). Compre-
counts most. This chapter reviews what hensive assessments of teaching are
is known about how research findings
can shape best practices in evaluating
1
Informal assessments of a faculty member’s
undergraduate teaching in science,
work that are used primarily to provide feedback
technology, engineering, and mathemat- and reinforcement to a colleague for purposes of
ics (STEM). Chapter 5 builds on this ongoing professional development and improve-
ment are characterized as formative evaluations.
research to highlight ways in which In contrast, evaluations that are used for pur-
expectations and guidelines for evaluat- poses of rendering formal personnel decisions
and that are based on a variety of data are often
ing teaching can be made clear to both called summative evaluations (Scriven, 1993;
faculty and administrators. review by Licata and Moreale, 1997).
51

more accurate, particularly when based Input from Students and Peers
on the views of current and former • Evidence of learning from student
students, colleagues, and the instructor portfolios containing samples of their
or department being reviewed. The writing on essays, examinations, and
process of evaluating teaching has been presentations at student research
found to work best when all faculty conferences or regional or national
members in a given department (or, in meetings. Additional direct and indirect
smaller colleges, from across the classroom techniques that demonstrate
institution) play a strong role in develop- student learning are discussed in
ing policies and procedures. This is the Chapter 5.
case because evaluation criteria must be • Informed opinions of other mem-
clear, well known and understood, bers of the faculty member’s depart-
scheduled regularly, and acceptable to ment, particularly when those opinions
all who will be involved with rendering are based on direct observation of the
or receiving evaluation (Alverno College candidate’s teaching scholarship or
Faculty, 1994; Gardiner et al., 1997; practice. The ability to offer such input
Loacker, 2001; Wergin, 1994; Wergin comes from the reviewer’s observing a
and Swingen, 2000).2 series of the candidate’s classes, attend-
Evidence that can be most helpful in ing the candidate’s public lectures or
formatively evaluating an individual presentations at professional association
faculty member’s teaching efficacy and meetings, serving on curricular commit-
providing opportunities for further tees with the candidate, or team teach-
professional development includes the ing with the candidate. Opinions of
following points: faculty colleagues also can be based on
their observations of student perfor-
mance in courses that build upon those
2
Alverno College has sponsored a comprehen- taught by the faculty member being
sive research program on assessment of student
learning and means of tying that assessment to
evaluated.
ongoing improvement of both teaching by • Input by faculty from “user” depart-
individuals and departmental approaches to
ments for service courses and from
education. For additional information, see
Alverno College Faculty (1994). A more recent related disciplines for interdisciplinary
monograph edited by Loacker (2001) describes courses. Such information can be very
Alverno’s program, with a focus on how students
experience self-assessment and learn from it to helpful in determining whether students
improve their performance. Then from the are learning subject matter in ways that
perspective of various disciplines, individual
faculty explain how self-assessment works in will enable them to transfer that learn-
their courses.

ing to other disciplines or learning Review of Departmental and

situations.3 Institutional Records
• Input from undergraduate and • The number and levels of courses
graduate teaching assistants, based on taught and the number of students
their participation in a range of courses enrolled in each course or section
and laboratories taught by the faculty taught by the instructor over time. This
member being evaluated, as well as post information can provide evaluators with
hoc input some time after they have had insight and perspective regarding the
the opportunity to work with and learn number of preparations required; the
from the candidate. This input can be amount of time needed for advising
solicited from graduating seniors and students; and, in some cases, the com-
alumni selected randomly from a faculty mitment of time necessary to correct
member’s class lists or in accordance examinations, term papers, and reports.
with the candidate’s recommendations. • The number of undergraduate
• Input from undergraduate and students advised, mentored, or super-
graduate students who have worked vised by the faculty member. This
with the faculty member as teaching or information can be accompanied by
research assistants or as collaborators opinions about the quality of the advice
on original research. Input from these or mentoring received.
students can be useful both at the time • The number of undergraduate
they are working with the faculty students the faculty member has guided
member and sometime after that rela- in original or applied research, the
tionship has ended. quality of their research as measured
• A summary of the professional through presentations and publications,
attainments of undergraduate students and their professional attainments while
who engaged in research under the under the faculty member’s supervision
tutelage of the faculty member being and later in their careers.
evaluated. • The number of graduate students
mentored in their preparation as teach-
ing assistants or future faculty members
and their effectiveness in teaching.
3
Accountability to other departments should
include evaluation of individual faculty members
and discussion of departmental program content.
A department’s accountability for its service to
other disciplines is considered in Chapter 8.
Academic deans can provide leadership in
fostering interdepartmental communication.
PRINCIPLES AND RESEARCH FINDINGS 53

Review of the Faculty Member’s into procedures for evaluating teaching

Teaching Portfolio and quality and effectiveness. The commit-
Other Documentation tee acknowledges and emphasizes that
• Evidence of the faculty member’s each source of data for evaluating the
adaptation of instructional techniques teaching of individual faculty members
for courses, laboratories, or field activi- has both advantages and disadvantages.
ties so as to demonstrably improve Multiple inputs to any evaluation pro-
student learning by achieving course cess can help overcome the shortcom-
objectives.4 ings of any single source.
• Evidence of the faculty member’s
Undergraduate Student
participation in efforts to strengthen
Evaluations
departmental or institutional curricu-
lum, to reform undergraduate educa- The use of student evaluations in
tion, or to improve teaching in the higher education is contentious. Fac-
discipline or across disciplinary bound- ulty often complain that student evalua-
aries. tions are predicated on such variables as
• The faculty member’s self-assess- what emotions students are experienc-
ment of his or her own teaching ing when they complete the question-
strengths and areas for improvement. naire, what they perceive as the faculty
• The faculty member’s participation member’s ability to “entertain,” and
in seeking external support for activities whether they were required to enroll in
that further the teaching mission. the course (Centra, 1993). Faculty also
challenge whether the questions on
student evaluation instruments encour-
SPECIFIC SOURCES OF DATA FOR age students to reflect longer-term
EVALUATING TEACHING QUALITY instructional success in their responses.
AND EFFECTIVENESS Despite these misgivings, extensive
research5 has established the efficacy of
This section reviews evidence on the student evaluations when they are used
effectiveness of various kinds of input
4
Under its Course, Curriculum, and Labora-
5
tory Improvement program, the National Science The U.S. Department of Education’s Educa-
Foundation (NSF) now supports faculty mem- tional Resources Information Center system cites
bers who adopt and adapt successful models for more than 2,000 articles on research that focus
courses and pedagogy in their own teaching. on student evaluations. Additional information is
Additional information about this program is available at <http://ericae.net/scripts/ft/
available at <http://www.ehr.nsf.gov/ehr/due/ ftcongen.asp?wh1=STUDENT+ EVALUATION>.
programs/ccli/>.

as one of an array of techniques for evaluating responses, as is often the

evaluating student learning. Students case with performance or portfolio
can, at a minimum, provide opinions on assessments, are likely to render rea-
such dimensions of teaching as the sonably similar judgments (American
effectiveness of the instructor’s peda- Educational Research Association
gogy, his or her proficiency and fairness [AERA], American Psychological
in assessing learning, and how well he Association [APA], and National Council
or she advises students on issues on Measurement in Education [NCME],
relating to course or career planning. 1999).
Students also can assess their own The reliability of student evaluations
learning relative to goals stated in the has been a subject of study for more
course syllabus, thereby providing some than 60 years. Remmers (1934) reports
evidence of whether they have learned on reliability studies of student evalua-
what the instructor intended. Self- tions that he conducted at Purdue
reports of learning have been shown to University in the 1930s. He investigated
be reasonably reliable as general indica- the extent of agreement among ratings
tors of student achievement (Pike, that students within a classroom gave to
1995). their teacher and concluded that excel-
The following discussion focuses on lent intraclass reliability typically re-
three critical issues associated with fair sulted when 25 or more students were
and effective use of student evaluation: involved. More recently, Centra (1973,
reliability, validity, and possible sources 1998) and Marsh (1987) found similar
of bias. A more complete review of the intraclass reliabilities even with as few
various types of instruments used for as 15 students in a class.
student evaluation and specific issues For tenure, promotion, and other
related to their use is provided in summative decisions, both the numbers
Appendix A. The application of these of students rating a course and the
instruments in practice is discussed in number of courses rated should be
Chapter 5. considered to achieve a reliable mean
from a good sample of students. For
Reliability example, Gilmore et al. (1978) find that
Reliability has several meanings in at least five courses with at least 15
testing. Here, the term refers to inter- students rating each are needed if the
rater reliability. The issue is whether ratings are to be used in administrative
different people or processes involved in decisions involving an individual faculty

member.6 To achieve the reliability for ratings in one type of course did not
summative evaluations advocated by necessarily receive similar ratings in
Gilmore et al., a newly hired tenure- other types of courses they taught.
track faculty member would need to be These differences may or may not be
evaluated each term for each course directly associated with variations in
taught during each of his or her teaching effectiveness in different
pretenure years. courses. For example, the same instruc-
On the other hand, the need for such tor may receive better evaluations for a
constant levels of student evaluation course that students elect to take than
could have the negative effect of stifling for a course that fulfills a general educa-
creativity and risk taking by the instruction requirement.
tor in trying new teaching or assess- Research employing coefficient alpha
ment techniques. Indeed, on some analyses7 to establish the reliability
campuses, academic administrators are (relative agreement) of items within
waiving the requirement for counting factors or scale scores has revealed
student evaluations as part of faculty students’ ratings of faculty over short
members’ dossiers (although such periods of time (test–retest within a
evaluations may be collected from semester) to be stable. These results
students) when those faculty agree to suggest that student evaluations are
introduce alternative approaches to unlikely to be subject to day-to-day
teaching their courses and assessing changes in the moods of either students
student learning (Project Kaleidoscope, or teachers (Marsh, 1987).
personal communication).
How reliable are student evaluations Validity
when faculty members teach different Validity is the degree to which evi-
types of courses, such as large, lower dence and theory support interpreta-
division lecture classes and small tions of test scores. The process of
graduate research courses? According validation involves accumulating evi-
to the results of one study (Murray et
al., 1990), instructors who received high
7
Coefficient alpha analysis is a form of factor
analysis, used to verify the major dimensions
(factors) and the items within that dimension in
6
Gilmore et al. (1978) observe that if fewer an instrument. Coefficient alpha determines the
than 15 students per class provide the ratings, a extent to which the items within a factor or scale
greater number of courses need to be rated— are intercorrelated and thus measure a similar
preferably 10. characteristic.

dence to provide a sound scientific basis are generally critical. Two recent
for proposed score interpretations studies have shed light on this question
(AERA, APA, and NCME, 1999). by using students’ ratings of their own
The key questions related to validity learning as a proxy measure of examina-
of student evaluations are how well tion achievement scores. In both
results from student evaluations corre- studies, analyses of large datasets
late with other measures of teaching revealed a highly statistically significant
effectiveness and student learning, and relationship between a student’s self-
whether students learn more from rated learning and his or her rating of
effective than ineffective teachers. To teacher effectiveness in the course
explore the relationship between learn- (Cashin and Downey, 1999; Centra and
ing and student evaluations, Cohen Gaubatz, 2000b).
(1981) examined multisection courses Other validity studies have compared
that administered common final exami- students’ evaluations of their instructors
nations. Mean values of teaching with those prepared by trained observ-
effectiveness from student evaluations ers for the same instructors. In one
in each section were then correlated study, the trained observers noted that
with the class’s mean performance on teachers who had received high ratings
the final examination. A meta-analysis from students differed in several ways
of 41 such studies reporting on 68 from those who had received lower
separate multisection courses suggested ratings. Highly rated teachers were
that student evaluations are a valid more likely to repeat difficult ideas
indicator of teacher effectiveness several times and on different occa-
(Cohen, 1981). Correlations between sions, provide additional examples when
student grades and student ratings of necessary, speak clearly and expres-
instructors’ skills in course organization sively, and be sensitive to students’
and communication were higher than needs (Murray, 1983). In short, student
those between student grades and evaluations appear to be determined by
student ratings of faculty–student the instructor’s actual classroom behav-
interaction. ior rather than by other indicators, such
One limitation of Cohen’s study is that as a pleasing personality (see Ambady
multisection courses are typically lower and Rosenthal, 1993).
division courses. Therefore, the ques- Although all of the studies cited above
tion arises of whether similar correla- address short-term validity (end-of-
tions exist for upper level courses, course measures), critics have argued
where higher level learning outcomes that students may not appreciate de-

manding teachers who have high routinely teaches classes of 50 or more

expectations until years later, when they students, or those of someone who
are able to reflect maturely on their teaches large lecture courses with
classroom experiences. However, hundreds of students.
research into this question has indicated It is important to be aware of possible
that there is good long-term stability—1 biases and to understand accordingly
to 5 years later—in student and alumni how to interpret evaluations fairly.
ratings of the same teachers (Centra, Studies that have examined these
1974; Drucker and Remmers, 1951; effects have been largely correlational
Overall and Marsh, 1980). and thus do not necessarily demonstrate
definite cause-and-effect relationships.
Bias Increasingly, multivariate analyses have
A circumstance that unduly influ- been used that control for extraneous
ences a teacher’s rating but has nothing variables. These analyses have helped
to do with actual teaching or learning clarify the data, as follows.
effectiveness is considered to be a Studies of course characteristics that
biasing variable. Possible biasing might bias the results of student evalua-
effects may derive from the course, the tions have looked at class size, discipline
student, or the teacher’s personal or subject area being taught, type of
characteristics (e.g., dress or appear- course (i.e., required versus elective),
ance). For example, instructors who and level of difficulty of the course.
teach small classes may receive higher With regard to the more favorable
ratings than those who teach large ratings accorded teachers of small
classes (Centra and Creech, 1976; classes noted above (Centra and
Feldman, 1984). However, it is also Creech, 1976; Feldman, 1984), the
likely that small classes produce better difference in ratings based on class size
learning and instruction (because accounted for only about 25 percent of
teachers can more easily address the standard deviation, not enough to be
individual questions, involve students statistically meaningful. The same
more actively, provide one-on-one studies found that the instructor’s
feedback, and so forth). Strictly speak- methods for teaching the course were
ing, small classes may not be a biasing more important, with active-learning
variable in student evaluations, yet it is classes receiving more favorable ratings
probably unfair to compare the ratings than lecture classes.
of someone who teaches only small In comparisons of student ratings in
classes with those of someone who different disciplines, classes in math-

ematics and the natural sciences were sequence, but that students may per-
found to be more likely to receive lower ceive as having little to do with their
ratings than classes in other disciplines immediate academic interests or future
(Cashin, 1990; Feldman, 1978). The needs.
differences were not apparent for all Contrary to what one might other-
dimensions, however—the organization wise expect, studies have found that
of courses and the fairness of tests and instructors who received higher ratings
assignments were two areas in which did not assign less work or “water
students rated the disciplines similarly. down” their courses (Marsh, 1987;
Lower ratings for natural science and Marsh and Roche, 1993; Marsh and
mathematics classes in such dimensions Roche, 2000). Natural science courses
as faculty–student interaction, course not only were generally rated less
difficulty and pace, and presentation highly, but also were judged to be more
format (lecture versus discussion) difficult. In this particular case, stu-
suggested that these courses were less dents within those disciplines who gave
student-oriented, more difficult, faster- teachers high ratings also noted that
paced, and more likely to include those teachers assigned more work.
lecture presentations. What this ap- The student characteristics most
pears to indicate is that students did not frequently studied for their effects in
like these aspects of the courses and biasing evaluations of teaching include
may have learned less (Centra, 1993). grade point average, expected grade in
Student ratings can be influenced by the course, academic ability, and age.
many other variables that may interact According to most studies (e.g., Marsh
with or counteract the influence of and Roche, 2000; McKeachie, 1979,
discipline or course format. For ex- 1999), none of these characteristics
ample, studies have shown that students consistently affects student ratings.
tend to give slightly higher ratings to Despite this finding, some instructors
courses in their major field or to courses still firmly believe that students give
they chose to take, as opposed to those higher ratings to teachers from whom
they were required to take. The likely they expect to receive high grades.
reason is that students (and possibly Instructor characteristics that could
teachers as well) are generally less possibly influence ratings are gender,
interested in required courses. These race, and the students’ perception that
often include introductory or survey the faculty member is especially “enter-
courses that meet distribution require- taining” during instruction (Abrami et
ments in a college’s general education al., 1982). Several studies have analyzed

the effect of gender—of both the evalu- Graduating Seniors and Alumni
ating student and the teacher—on Evaluations of an instructor’s teach-
student evaluations. Most of these ing by graduating seniors and alumni
studies indicate there is no significant can be useful in providing information
difference in ratings given to male and about the effectiveness of both indi-
female instructors by students of the vidual teachers and the department’s
same or the opposite sex (Centra and overall curriculum. Current students
Gaubatz, 2000a; Feldman, 1993). In can comment on day-to-day aspects of
certain areas of the natural sciences and teaching effectiveness, such as the
engineering in which women faculty instructor’s ability to organize and
members are a distinct minority, female communicate ideas. Graduating seniors
teachers have been found to receive and alumni can make judgments from a
higher ratings than their male counter- broader, more mature perspective,
parts from both male and female stu- reflecting and reporting on the longer-
dents. Female teachers also were more term value and retention of what they
likely than male teachers to use discus- have learned from individual instructors
sion rather than lecturing as a primary and from departmental programs. They
method for teaching, which may help may be particularly effective contribu-
account for the higher ratings they tors to evaluations based on exit inter-
received (Centra and Gaubatz, 2000a). views (Light, 2001). There are, how-
The question of whether teachers ever, drawbacks to surveying seniors
who are highly entertaining or expres- and alumni, including difficulties in
sive receive higher ratings from stu- locating graduates and deciding which
dents has been examined in a series of students to survey (e.g., the percentage
“educational-seduction” studies (Abrami of students included in an evaluation
et al., 1982; Naftulin et al., 1973). In one process based on random surveys
study, researchers employed a profes- versus those recommended by the
sional actor to deliver a highly entertain- faculty member being evaluated), and
ing but inaccurate lecture. The actor the hazy memory alumni may have
received high ratings in this single about particular instructors (Centra,
lecture, particularly on his delivery of 1993).
content. A reasonable conclusion from
these studies is that by teaching more Teaching Assistants
enthusiastically, teachers will receive Teaching assistants are in a unique
higher ratings (Centra, 1993). position to provide information about

the teaching skills of the faculty mem- tions of teaching, few studies exist
bers with whom they work. They also concerning the efficacy of peer review,
can offer useful insight and perspective and those available tend to be limited in
on the collection of courses and cur- scope. Research has demonstrated that
ricula offered by their academic depart- extended direct observation of teaching
ment (Lambert and Tice, 1992; National by peers can be a highly effective means
Research Council [NRC], 1995b, 1997b, of evaluating the teaching of an indi-
2000b). Because teaching assistants vidual instructor (e.g., American Asso-
routinely observe classes and work with ciation for Higher Education [AAHE],
students throughout the term, they can 1995; Hutchings, 1996). However,
comment on course organization, the colleges and universities do not use
effectiveness of an instructor’s presenta- classroom observation widely in the
tions and interactions with students, the assessment of teaching.
fairness of examinations, and the like. A common but erroneous assumption
Teaching assistants also can assess how is that peer evaluations of teaching,
well the instructor guides, supervises, including evaluations by department
and contributes to the development and chairs, are best conducted through
enhancement of his or her own peda- classroom observation (Seldin, 1998).
gogical skills. As continuing graduate Even when peer evaluation does involve
students, however, teaching assistants extensive classroom observation,
may be vulnerable to pressures that problems can occur. For example, some
make it difficult to provide candid research has shown that when an
evaluations. Thus when they are asked instructor’s evaluation is based solely on
to evaluate their instructors, special classroom observation, the raters
precautions, such as ensuring confiden- exhibit low levels of concurrence in
tiality, must be taken. their ratings (Centra, 1975). This may
be because many faculty and administra-
Faculty Colleagues tors have had little experience in con-
Compared with the extensive re- ducting such reviews in ways that are
search on the utility8 of student evalua- fair and equitable to those being re-
viewed. Another reason may be that
such observation is not part of the
8
culture of teaching and learning within a
Utility denotes the extent to which using a
test to make or inform certain decisions is department. It may be possible to train
appropriate, economical, or otherwise feasible. faculty in observation analysis, provid-
The criterion of fairness is beginning to replace
utility in the scholarly literature on measurement. ing them with the skills, criteria, and

standards needed for consistent ratings ratings. The reliabilities of the evalua-
of a colleague’s classroom performance. tions (based on average intercorrela-
However, such efforts are time-consum- tions) were very high (above 0.90) for
ing and require more serious dedication each of the three performance areas. In
to the task than is usually given to fact, Root concluded that even a three-
teaching evaluations in higher educa- member committee working in similar
tion. fashion would be able to provide suffi-
Some studies have shown that faculty ciently reliable evaluations and in a very
believe they are better able to judge the short period of time—no more than an
research productivity of their colleagues hour or two. This study supports the
than their teaching effectiveness. use of colleague evaluations for
Kremer (1990) found that evaluations of summative decisions providing that the
research were more reliable than committee has previously discussed
evaluations of teaching or service. In evaluative criteria and expected stan-
that study, as is generally the case, dards of performance, and has a num-
faculty had access to more information ber of different sources of data on which
about their colleagues’ research than to base its evaluations.
about their teaching or service. Accord- This is a particularly critical point
ing to other studies, when faculty because at present, although tenure and
members have an extensive factual promotion committees at the college or
basis for their evaluations of teaching, university level always include faculty
there is higher reliability in their rat- representatives, such faculty usually do
ings. For example, Root (1987) studied not have the authority or the time
what happened when six elected faculty needed to make their own independent
members independently rated individual evaluation of a candidate’s performance
dossiers of other faculty. The dossiers in teaching, research, or service.
included course outlines, syllabi, teach- Instead they must rely almost entirely
ing materials, student evaluations, and on other sources, such as written or
documentation of curriculum develop- oral evaluations from colleagues in the
ment. The faculty members being candidate’s discipline or student evalua-
evaluated also submitted information tions.
about their scholarly and service activi- When conducted properly, review and
ties. Using cases that illustrated high evaluation by one’s colleagues can be an
and low ratings, the six-member com- effective means of improving teaching at
mittee reviewed and discussed criteria the college level, providing feedback for
for evaluation before making their ongoing professional development in

teaching, and enabling more informed different sections of the same course
personnel decisions (AAHE, 1993; (Bernstein and Quinlan, 1996; Edgerton
Chism, 1999; French-Lazovik, 1981; et al., 1991; Hutchings, 1995, 1996).
Hutchings, 1995, 1996; Keig and
Waggoner, 1994). AAHE recently Instructional Contributions
undertook an extensive, multiyear In addition to classroom observation,
initiative to examine ways of maximizing faculty colleagues can examine and
the effectiveness of peer review of comment on an instructor’s teaching-
teaching. A website describes the related activities. These kinds of evalua-
results and products of this initiative in tions might include examining syllabi,
detail.9 The ideas reviewed below distributed materials, or the content of
reflect the findings of the AAHE initia- tests and how well the tests align with
tive and other sources as cited. course goals. They might also address
the faculty member’s involvement with
Evaluation of Course Materials curriculum development, supervision of
Departments can obtain valuable student research, contributions to the
information about course offerings from professional development of colleagues
individual instructors by asking faculty and teaching assistants, publication of
to review and offer constructive criti- articles on teaching in disciplinary
cism of each other’s course materials journals, authorship of textbooks,
and approaches to teaching and learn- development of distance-learning or
ing. Faculty who teach comparable web-based materials, and related activi-
courses or different sections of the ties (Centra, 1993).
same course or who are particularly
knowledgeable about the subject matter Use of Students for
can conduct reviews of selected course Classroom Observation
materials. They can analyze those As noted above, peer observation can
materials with regard to such matters as be an effective evaluation technique if
the accuracy of information, approaches the observers are trained in the process.
to encouraging and assessing student Understandably, observation of col-
learning, and the consistency of expec- leagues remains a highly sensitive issue
tations among instructors who teach for some faculty members. In some
cases, the presence of the observer may
even affect the instructional dynamics of
9
Information about AAHE’s peer review of the course. For this reason, and also on
teaching initiative is available at <http://
www.aahe.org/teaching/Peer_Review.htm>. the grounds of fairness and balance, the

best use of peer observation may be as a members traditionally have provided a

voluntary and informal procedure that list of basic information about their
enables faculty members to gain insight teaching. These lists might include
on the strengths and weaknesses of details about instructional goals and
their teaching skills, rather than as a objectives, conduct and supervision of
basis for personnel decisions. In this laboratory instruction, teaching meth-
spirit, some institutions also are experi- ods, syllabi and other course materials,
menting with the use of student consult- websites, student supervision and
ants—students not enrolled in a particu- advising, and efforts at self-improve-
lar course—to assist faculty who have ment.
requested input on their teaching but In recent years, however, increasing
are reluctant to ask colleagues (e.g., numbers of faculty have elected to
Emerson et al., 2000).10 At a few institu- develop, or departments and institutions
tions, classroom teachers from local have required the submission of, teach-
secondary schools have volunteered or ing portfolios to be used for purposes of
are paid to provide such input. both formative and summative evalua-
tion (e.g., Anderson, 1993; Bernstein
Self-Evaluation by Faculty and Quinlan, 1996; Centra, 1994;
Reports on Teaching Activities and Edgerton et al., 1991; Hutchings, 1998;
Teaching Portfolios Seldin, 1991). Teaching portfolios have
the advantage of providing continuing
Most institutions require faculty to
documentation of teaching and advising;
describe their teaching, student advis-
that is, teachers can accumulate evi-
ing, scholarship, and service activities
dence of their effectiveness as it ap-
each year and in greater detail for
pears. Teachers’ personal reflections on
promotion or tenure and other person-
their teaching and evidence of student
nel decisions. In response, faculty
learning that is supported, perhaps, by
their own classroom research are key
10
For example, Worcester Polytechnic
components of a portfolio. Self-analysis
Institute and Brigham Young University are for formative evaluation of teaching
using such student consultants to provide
effectiveness—as opposed to quantified
instructors with “off-the-record” or private
midcourse feedback on such factors as what they self-evaluation for summative evalua-
gained from a particular class and how others in tion—gives faculty the opportunity to
the class responded to the material. For
additional information, see Greene (2000). See present their own best case for their
also <http://www.wpi.edu/Academics/CEDTA> success in achieving their teaching
and <http://www.byu.edu/fc/pages/
fchomepg.html>. goals (Centra, 1979; Hutchings, 1998).

Teaching portfolios pose opportuni- of AAHE’s ongoing project on teaching

ties and challenges to both those who portfolios is a series of papers (Cam-
are asked to create them and those who bridge, 2001) that provides guidance to
must review them. For example, be- faculty members, departments, and
cause they are more qualitative in institutions wishing to maintain elec-
nature than other sources of informa- tronic portfolios.
tion, teaching portfolios are likely to be
more difficult to evaluate objectively. Self-Review
When they are used for summative To supplement descriptive informa-
purposes, it may be difficult for commit- tion, faculty who engage in self-review
tees on promotion and tenure to com- reflect on their accomplishments,
pare the contents of one faculty strengths, and weaknesses as instruc-
member’s portfolio with those of tors. Research has shown that self-
another’s. Recognizing this challenge, evaluation can be helpful in summative
AAHE is now sponsoring a multiyear personnel decisions by providing
initiative to examine the most effective context for the interpretation of data
ways of developing and utilizing infor- from other sources. For example, a
mation in teaching portfolios for teacher faculty member may have a particularly
evaluation and ongoing professional difficult class or may be teaching a
development.11 In addition, AAHE course for the first time. Or she or he
recently acquired and posted on the may be experimenting with new teach-
World Wide Web “The Portfolio Clear- ing methods that may result in both
inghouse,” a database of some 30 improved student learning and retention
portfolio projects from a variety of types and lower student ratings (Hutchings,
of colleges and universities around the 1998).
world. This database provides informa- The committee found that much of
tion about portfolios as a means of the research on self-evaluation has
demonstrating student learning, effec- focused on instructors rating their
tive teaching, and institutional self- teaching performance rather than
assessment.12 Another recent product simply describing or reflecting on it.
One analysis indicated that self-evalua-
tions did not correlate with evaluations
11
by current students, colleagues, or
Additional information is available at <http:/
/www.aahe.org/teaching/portfolio_projects. administrators, although the latter three
htm>. groups agreed in high measure with one
12
This database is available at <http://
www.aahe.org/teaching/portfolio_db.htm>. another (Feldman, 1989). At the same

time, it was found that while teachers kind of evidence should be interpreted
tended to rate themselves higher than very cautiously since many factors other
their students did, they identified the than teaching effectiveness may account
same relative strengths and weaknesses for the findings. For example, recent
as did other evaluators (Centra, 1973, changes in an institution’s policy on
Feldman, 1989). Therefore, self-evalua- dropping courses may influence which
tions may be most useful in improving students decide to leave or remain in a
instruction, although corroborating course and when they elect to do so,
evidence from other sources may be independently of the instructor’s teach-
necessary to underscore needed ing effectiveness. If, however, records
changes. For summative purposes, show that a larger-than-normal fraction
however, most of the faculty queried in of the students in a professor’s course
one survey agreed with the findings of regularly drop out and repeat the class
research: self-evaluations lack validity at a later time, the attrition may be
and objectivity (Marsh, 1982). Although relevant to the quality of the instructor’s
quantifiable self-evaluations should thus teaching. Similarly, questions might be
probably not be used in summative raised about the quality of an
evaluations, teaching portfolios can be instructor’s teaching effectiveness
useful in improving instruction if they (especially in lower division courses) if a
are considered in conjunction with higher-than-normal fraction of students
independent evaluations from students, who have declared an interest in major-
colleagues, or teaching improvement ing in the subject area fails to enroll in
specialists. higher level courses within the depart-
ment (e.g., Seymour and Hewitt, 1997).
Institutional Data and Records In contrast, an unusual grade distribu-
tion may reflect some anomaly in a
Grade Distributions, Course Retention, particular class and should be consid-
and Subsequent Enrollment Figures ered in that light. For example, while
Historical records of grade distribu- the motives or competence of an in-
tions and enrollments within a depart- structor who consistently awards high
ment may provide supplemental infor- grades might be questioned, it is en-
mation about a faculty member’s teach- tirely possible that this individual has
ing when compared with data collected engaged his or her students in regular
from colleagues who have taught similar formative evaluations, which has helped
courses or are teaching different sec- them overcome academic problems and
tions of the same course. However, this learn more than might otherwise be

expected. These students’ performance to build a record of attracting high-

on standardized quizzes or examina- quality students. Strong indicators of
tions might therefore exceed that of how effective their mentoring has been
students being taught by other instruc- include the disseminated scholarly
tors, so that a skewed, high grade products or subsequent academic and
distribution would be entirely war- professional accomplishments of their
ranted. Similarly, if a large proportion former students in research as well as in
of students from a faculty member’s teaching. Again, it must be acknowl-
introductory class later enroll in the edged that many factors affect students’
instructor’s upper division advanced decisions to enroll in a particular aca-
elective course, one might reasonably demic program, and many factors affect
assume that this instructor has captured their subsequent achievements as well.
students’ interest in the subject matter. However, evidence, if any, that links a
particular faculty member to students’
Quality and Performance of selection of supervisors and their future
Undergraduate Research Students scholarly productivity and professional
Faculty members who have super- aspirations and accomplishments can be
vised independent undergraduate considered useful as supplemental
research will have had the opportunity evidence of teaching effectiveness.


PART II
Applying What Is Known:

Strategies for Evaluating
Teaching Effectiveness


Evaluation Methodologies
Part I of this report describes recent need to correct their learning deficien-
research on ways to rethink and restruc- cies and misconceptions. When such
ture teaching and learning, coupled with evaluation is properly employed, stu-
new approaches to evaluation and dents learn that they can engage in self-
professional development for faculty. assessment and continuous improve-
Those findings have the potential to ment of performance throughout their
reshape undergraduate education in lives.
science, technology, engineering, and Accordingly, this chapter offers
mathematics (STEM) for a much larger practical guidance to postsecondary
number of undergraduates. However, faculty and administrators on ways to
developing strategies for implementing institute a system of both evaluation and
and sustaining such changes requires professional development that can
the commitment of all members of a contribute to significant gains in teach-
college or university community. ing effectiveness for faculty who teach
In a teaching and learning commu- undergraduates. The chapter describes
nity, the most effective evaluation is that how input from students (undergradu-
which encourages and rewards effective ates and graduate teaching assistants),
teaching practices on the basis of colleagues, and faculty self-evaluation
student learning outcomes (Doherty et can be used for evaluating individual
al., 2002; Shapiro and Levine, 1999). instructors. It also describes the advan-
Assessment of student learning at its tages and disadvantages of these vari-
best enables students to identify their ous approaches.
own strengths and weaknesses and to As stated in Chapter 1, ongoing
determine the kinds of information they formative assessment of student learn-
71

ing can have powerful benefits both in learning and the use of that information
improving learning and in helping to improve teaching are considered first.
faculty improve their teaching on the Additional strategies and methods for
basis of the feedback they receive from formative evaluation follow. The chap-
a variety of sources. The information ter concludes with a series of sugges-
gathered during such assessments also tions for improving summative evalua-
can serve as a basis for more formal, tion of faculty. The committee empha-
summative evaluations that have an sizes that the approaches described in
impact on important personnel deci- this chapter are but a sampling of the
sions. techniques that appear in the research
The technique of outcomes assess- literature on improving the evaluation of
ment as a means of measuring student teaching and student learning. They are
Assessment Is More Than Grades
To many, the word “assessment” simply means the process by which we assign students
grades. Assessment is much more than this, however. Assessment is a mechanism for
providing instructors with data for improving their teaching methods and for guiding and
motivating students to be actively involved in their own learning. As such, assessment
provides important feedback to both instructors and students.
Assessment Is Feedback for Both Instructors and Students
Assessment gives us essential information about what our students are learning and about
the extent to which we are meeting our teaching goals. But the true power of assessment
comes in also using it to give feedback to our students. Improving the quality of learning in
our courses involves not just determining to what extent students have mastered course
content at the end of the course; improving the quality of learning also involves determining
to what extent students are mastering content throughout the course.
SOURCE: Excerpted from National Institute for Science Education (2001b).

included here on the basis of the and legislators that a program of study
committee’s analysis of the research produces competent graduates (Banta,
literature and the expertise of individual 2000).
committee members, and with the
expectation that each institution will Outcome Assessment Activities
adapt or modify these approaches Faculty members, both individually
according to its individual needs. and as colleagues examining their
department’s education programs, have
found the following activities helpful
IMPROVING TEACHING BY when undertaking outcome assessment:
EXAMINING STUDENT
• Developing expected student
LEARNING: OUTCOME
learning outcomes for an individual
ASSESSMENT
course of study, including laboratory
skills.
One approach to improving student
• Determining the point in a
learning is outcome assessment—the
student’s education (e.g., courses,
process of providing credible evidence
laboratories, and internships) at which
that an instructor’s objectives have been
he/she should develop the specified
obtained. Outcome assessment enables
knowledge and skills.
faculty to determine what students
• Incorporating the specified learn-
know and can do as a result of instruc-
ing outcomes in statements of objectives
tion in a course module, an entire
for the appropriate courses and experi-
course, or a sequence of courses. This
ences.
information can be used to indicate to
• Selecting or developing appropri-
students how successfully they have
ate assessment strategies to test student
mastered the course content they are
learning of the specified knowledge and
expected to assimilate. It can also be
skills.
used to provide faculty and academic
• Using the results from assessment
departments with guidance for improv-
to provide formative feedback to indi-
ing instruction, course content, and
vidual students and to improve curricu-
curricular structure. Moreover, faculty
lum and instruction.
and institutions can use secondary
• Adjusting expected learning
analysis of individual outcome assess-
outcomes if appropriate and assessing
ments to demonstrate to prospective
learning again. Such a process can lead
students, parents, college administra-
to continual improvement of curriculum
tors, employers, accreditation bodies,
and instruction.
E VA L U AT I O N M E T H O D O L O G I E S 73

Faculty in STEM are challenged in Scoring of Outcome

their teaching by a set of circumstances Assessments: Primary
that most faculty in other disciplines do Trait Analysis
not encounter, such as designing labora- Increasingly, primary trait analysis
tory and field components of courses, (Lloyd-Jones, 1977) is being used as a
incorporating modern technology into scoring mechanism in outcome assess-
courses, or supervising students in- ment (Walvoord and Anderson, 1998).
volved with original research (see Primary trait analysis is a technique
Chapter 2 for additional detail). How- whereby faculty members consider an
ever, faculty in these disciplines also assignment or test and decide what
have an array of assessment methodolo- traits or characteristics of student
gies from which to choose that address performance are most important in the
particular learning outcomes (e.g., see exercise. They then develop a scoring
Doherty et al., 2002). Student re- rubric (Freedman, 1994) for these traits
sponses in each of the following formats and use it to score each student’s
can first be studied for the information performance.
they provide about individual student For example, Emert and Parish
learning and performance, and then (1996) developed multiple-choice and
compared across students and classes short-answer tests for undergraduate
for clues about the strengths and students enrolled in courses in algebra,
weaknesses of curriculum and instruc- discrete mathematics, and statistics.
tion: Students were asked to submit support-
ing work to provide additional insight
• Classroom quizzes and exams into their thought processes and the
• Projects extent to which they had developed an
• Poster presentations of library or understanding of mathematical con-
laboratory research cepts. Emert and Parish developed the
• Cooperative experiences following scoring rubric to assess
• Portfolios (collections of work) performance on each item their stu-
• Standardized tests both within and dents provided:
across disciplines
• Student journals Score Criterion
• Questionnaires 3 Conceptual understanding
• Interviews apparent; consistent notation,
• Focus groups with only an occasional error;
logical formulation; complete

or near-complete solution/ Benefits of Outcome Assessment

response It can be difficult and time-consuming
2 Conceptual understanding for faculty to redesign course objectives
only adequate; careless math- to focus on student learning outcomes,
ematical errors present (for to agree with colleagues on comprehen-
example, algebra, arithmetic); sive learning outcomes for the entire
some logical steps lacking; curriculum, and to select or develop
incomplete solution/response appropriate assessment tools. It can be
1 Conceptual understanding not equally or more difficult for faculty to
adequate; procedural errors; adopt a routine of systematically collect-
logical or relational steps ing and studying assessment data and
missing; poor or no response then making improvements based on
to the question posed that feedback. However, some ex-
0 Does not attempt problem, or amples of positive, multidimensional
conceptual understanding change have been documented from
totally lacking departments that have taken assess-
ment seriously. These departments
By studying the aggregate scores for update curricula continuously. They
each item, Emert and Parish and their develop new courses and phase out
colleagues discovered that students others as needs change. And they can
missed most items because they lacked document improvement in student
the conceptual understanding to ad- learning (Wergin, 1995; Wergin and
dress the problem appropriately (as Swingen, 2000).
opposed to making careless errors). By Other changes that have been
inspecting the items missed by large prompted by outcome assessment
numbers of students, faculty discovered include faculty employing more active
which concepts needed to be addressed learning strategies that enable students
through instruction again, perhaps in to practice the concepts they are learn-
alternative ways. Understanding such ing in class. Alumni and employers are
misconceptions by students can provide being asked to comment on curriculum
instructors with valuable insights into and instruction and even to serve as
how they might adjust their teaching evaluators of teaching and learning. For
techniques or emphases to address example, at Virginia Polytechnic Insti-
these kinds of problems (see, e.g., tute and State University, the Depart-
National Research Council [NRC], ment of Civil Engineering created an
1997a, 1999b). alumni advisory board and asked its

members to debrief a group of juniors vidual faculty members, particularly in

and seniors regarding the department’s their first years of teaching, with ongo-
curriculum. The students discussed ing individualized formative feedback
such issues as overcrowding due to from students and colleagues (Brinko,
space limitations. In response, the soil 1993; Cambridge, 1996; Centra, 1993;
mechanics laboratory was expanded Hutchings, 1996). Instructors are best
through privately sponsored renovation. served by informal evaluation activities
In addition, students’ concerns about that take place throughout a course,
opportunities to learn to use the latest especially when coupled with consulta-
software led to the development of a tions with learning experts.1 Such
new computer laboratory. And a per- informal activities can help instructors
ceived need for improved communica- identify what is working and what needs
tion skills encouraged faculty to develop to be improved while the course is still
new writing-intensive courses and in progress.
introduce them into the civil engineer- For example, helpful and regular
ing curriculum (Banta et al., 1996). feedback from students allows
Outcome assessment can be difficult midcourse corrections in such areas as
to implement because it requires that organization, methods of teaching, and
faculty reorient their course and cur- the introduction or modification of
riculum objectives to focus on what activities designed to enhance learning.
students learn rather than what faculty Many institutions have already recog-
teach. Nonetheless, the committee has nized the benefits of such midcourse
concluded that outcome assessment can corrections and offer faculty guidance
be an important approach to emphasiz- and appropriate forms for conducting
ing and focusing on what and how various levels of student surveys (see
students learn. Appendix B). The National Institute for
Science Education (NISE) provides a
“Student Assessment of Learning
OTHER STRATEGIES AND Gains” website where faculty can use
METHODS FOR and modify questionnaires designed to
FORMATIVE EVALUATION
Formative Evaluation by 1
In contrast, Marsh and Roche (1993) report
Undergraduate Students that feedback gathered at the end of a course had
significantly greater long-term impact on the
Research has shown that the best way improvement of teaching than midcourse
evaluations.
to improve teaching is to provide indi-

offer both formative and summative can also be used to assess their effec-
feedback from their students about how tiveness as a teacher summatively.
various elements of their courses are However, many factors other than the
helping the students learn. This innova- instructor’s teaching competence can
tive website also allows students to affect examination results, including
complete the survey form on line and prior knowledge; students’ preconcep-
provides instructors with a statistical tions; and their ability, interest, and
analysis of the students’ responses.2 skills in the subject area (Centra, 1993).
The results of studies on formative Another factor is student effort. Even
evaluations of student learning indicate the most effective teachers can do only
that the techniques described below so much to motivate students. Although
require modest effort, are easy to carry most college teachers try to motivate
out, and consume very little class time. students to learn, in the end students
In addition, faculty can obtain regular must take responsibility for their own
feedback from their students through learning and academic achievement.
the use of course listservs, electronic For the past three years, the Carnegie
mail, or a website for student feedback Foundation for the Advancement of
connected to a course’s website. Teaching and Pew Forum on Under-
graduate Learning (2002) have pub-
Repeated Measurements of lished annually the National Survey of
Student Learning and Student Engagement: The College
Teaching Effectiveness Student Report. Each of these reports is
The typical end-of-course student compiled from responses to a question-
evaluation form is an indirect assess- naire whose respondents consist of
ment tool that can help an instructor thousands of first-year and senior
understand what worked to assist undergraduates at 4-year colleges and
learning in a course and what did not. universities.3 The students are asked
Instructors may feel that students’ about the extent to which they partici-
scores on final examinations in their pate in classroom and campus activities
courses provide a valid measure of shown by research studies to be impor-
student learning and that this measure tant to learning. Questions from the
2 3
Additional information and links to the The list of institutions that participated in this
survey forms are available at <http://www. project is available at <http://www.indiana.edu/
wcer.wisc.edu/salgains/instructor/>. ~nsse/>.

2001 survey instrument are provided in that lists the learning outcomes for a
Appendix B.4 This instrument and its course or series of courses. Students
parent, the College Student Experiences may be asked to indicate how much the
Questionnaire (Indiana University, course or the entire curriculum in-
2000), can provide important informa- creased their knowledge and skills in
tion about the quality of effort students the specified areas.
are committing to their work. For maximum usefulness, teachers
If a teaching evaluation form is may want to add their own course-
distributed only at the end of a course, it related items to student evaluation
cannot help the instructor make useful forms, as well as encourage written or
modifications for students who are oral communication from students,
currently enrolled. A better way to including computer-assisted feedback.
assess student learning and teaching Evaluations of laboratory, field, and
effectiveness is to test students at the extra clinical or discussion sections
beginning and then again at the end of a require special questions, as do evalua-
course and inspect the “gain scores.” tions of student advising (NISE, 2001a).
An instructor’s willingness and ability to
use gain scores to improve a course Direct Questioning of Students
may be considered favorably during a The easiest way to find out whether
summative evaluation of teaching. At students understand what is being said
the same time, gain scores are easily is to ask them directly. But unless
misinterpreted and manipulated and instructors have developed sufficient
may not be statistically reliable (both rapport and mutual respect among the
pre- and post-tests are characterized by students in their class, they should
unreliability that is compounded when avoid questions or situations that could
the two are used together). Therefore, make it awkward for students to re-
they should not be used exclusively to spond (“Who is lost?”) or are so generic
examine student learning for purposes as to lead to nonresponses (“Are there
of summative evaluation. any questions?”). Instead, instructors
Another indirect measure of student should pose questions that encourage
learning that some faculty have found more specific responses, (e.g., “How
particularly useful is a questionnaire many of you are understanding what we
are talking about?”). Various forms of
information technology, such as in-class
4
The survey instruments for both 2000 and response keypads, can facilitate asking
2001 are also available at <http://www.indiana. such questions, allowing students to
edu/~nsse/>.

answer without fearing that they will be A similar approach, developed by the
singled out or ridiculed by their peers if physics education community, is “just-in-
they indicate their lack of understand- time” teaching (Dougherty, 1999).
ing. Students are asked to respond to one or
Even better, instructors can ask two short questions posed by the
students to paraphrase briefly the key instructor the day before a subject is to
points or essence of a discussion or be taught. They submit their responses
lecture. At the end of a class session, via e-mail or to a website. These re-
students can be asked individually or in sponses give the instructor a good idea
pairs to write a brief summary of the of what the students do and do not
main ideas presented and submit it to understand about the concepts to be
the instructor (anonymously). If this considered. The instructor can then
method is used, students should clearly adjust the amount of time spent on
understand that the written summary is explaining the concepts, working
not a quiz and will not be graded. through problems, or providing ex-
amples that will help the students learn
Minute Papers and and understand the concepts.
Just-in-Time Teaching
At the end of a class, instructors can Student Teams
ask students to write for a minute or two Another documented approach
on one of the following kinds of ques- involves asking a team of students to
tions: “What is the most significant work throughout the term on continu-
thing you’ve learned today?” “What ous course evaluation (Baugher, 1992;
points are still not clear?” or “What Greene, 2000; Wright et al., 1998). The
question is uppermost in your mind at team members are encouraged to
the end of today’s class?” Responses administer questionnaires and interview
can help instructors evaluate how well their peers about how the instructor is
students are learning the material. or is not promoting learning.
Student responses to the second and For larger classes, a liaison commit-
third questions also can help instructors tee of two to four students can be
select and structure topics for the next established that meets periodically with
class meeting. Large numbers of such the instructor to discuss difficulties or
short papers can be read quickly, and a dissatisfactions. Membership on the
review of unclear concepts can take committee can be rotated from a list of
place at the next class meeting (Angelo volunteers as long as the entire class
and Cross, 1993; Schwartz, 1983). knows who the liaisons are at any given

time. Alternatively, students who are groups provide students with opportuni-
not enrolled in a course can be hired to ties to learn from one another, and a
attend the class and offer ongoing group may find it easier to seek assis-
feedback to the instructor (e.g., Greene, tance from the instructor. In turn,
2000). having group representatives rather
than individual students approach the
Students’ Course Notes instructor can reduce the amount of
With students’ permission, instruc- time required to answer repetitive
tors can ask to borrow a set of notes. questions, especially in larger classes.
This technique allows teachers to see
what students consider to be the main Informal Conversations
points presented and whether there is Instructors can seek feedback
misinformation or confusion about through informal conversations with
various topics. Alternatively, to ensure students during office hours, before or
student anonymity, students can be after class, or through e-mail. They can
asked to photocopy selected portions of ask students about what has been
their notes and submit them to the working well or what is problematic.
instructor without identifying informa- Instructors should not pose these
tion (Davis, 1993). questions to students in ways or at times
that might force them to answer quickly.
Chain Notes Questions should be directed to those
In small classes, it may be possible to students the teacher thinks would be
pass around a piece of paper midway most likely to respond candidly. When-
through a session and ask students to ever this kind of feedback is solicited,
jot down the main point of what is being instructors should keep in mind that
discussed at that moment. The instruc- such evidence is anecdotal and may not
tor then has a listing of what students be representative of the entire class.
consider to be the key concepts dis- However, informal responses from
cussed in that class period, which can individual students can serve as the
be used (Angelo and Cross, 1993). basis for index card questions to the
entire class (discussed next). Asking
Student Study Groups such questions based on informal
Students can be encouraged to form conversations with students can also
small study groups and to send repre- reinforce the message that the instruc-
sentatives to discuss any difficulties or tor is listening to students and takes
questions with the instructor. Study input from them seriously.

Index Cards to (1) select a spokesperson who will

Several times during the term, an write down the group’s comments, (2)
instructor can pass out index cards to name something in the course they find
students and ask them to respond to two very helpful or worthwhile, (3) name
questions, one on the front of the card, something they would like to see
the other on the back. General ques- changed, and (4) suggest how the
tions can be posed, such as “What are course could be improved. After the
your overall impressions of the course?” groups have completed their work, the
“What’s good about the course?” “Do evaluator asks the spokesperson from
you have any suggestions for changing each group to report. The evaluator
the course?” or “Are there any prob- summarizes the points of consensus for
lems?” Alternatively, the instructor can the entire class and also clarifies points
ask more specific questions about of disagreement. The evaluator then
aspects of the course, such as “Are the provides an oral or written summary for
problem sets too difficult?” or “Is the the instructor (Clark and Redmond,
laboratory section well connected to 1982).
other aspects of the course?” Providing
Small Group
prompts (such as “I would like you to do
Instruction Diagnosis5
more . . . ” or “I would like you to do less
. . . ”) and asking students to complete This technique (also known by its
the sentence is another useful technique abbreviation, SGID) originated at the
(Davis, 1993). University of Washington and is now
promoted by teaching and learning
Outside Evaluators centers on a variety of types of cam-
Midway through the term, an instruc-
tor can invite an instructional improve-
ment specialist from the campus-wide or 5
The description of small group instruction
diagnosis presented here is based on information
discipline-based teaching and learning taken from the websites of several campus
center or a departmental colleague to centers for teaching and learning. A more
detailed description of this approach, along with
conduct an oral evaluation with his or
links to other websites and resources on the
her students. At the beginning of the subject, is available from a website at Miracosta
class, the teacher introduces the guest Community College, <http://www.miracosta.cc.
ca.us/home/gfloren/sgid.htm>. Small Group
evaluator and then leaves the room for Instructional Diagnosis, an online journal from
20 minutes. During that time, the the National Teaching and Learning Forum that
publishes research on the uses of the method, is
evaluator asks students to cluster into available at <http://www.ntlf.com/html/pi/9705/
groups of five or six and take 10 minutes sgid.htm>.

puses. The goal of SGID is to align Instructors should emphasize to stu-

expectations for improving teaching and dents that they would like to receive
learning. Consultants are employed to candid, constructive responses that will
gather information directly from stu- help them improve the course.
dents and instructors. The technique It also is important for the instructor
provides feedback to instructors, includ- to respond quickly and candidly to
ing suggestions for strengthening their students’ comments and concerns.
courses, and generally increases com- Davis (1993) discusses strategies for
munication between students and responding to student feedback. She
instructors. The consultative process sorts student suggestions for improve-
takes anywhere from 15 to 60 minutes ment into three categories: (1) improve-
and is most effective when conducted ments that can be instituted immedi-
near midsemester so the faculty mem- ately during the current semester (e.g.,
ber will have sufficient time to amend the turnaround time on grading home-
the course. work assignments); (2) those that must
Classroom interviews involve the wait until the next time the course is
consultant interviewing students, in the offered (e.g., the textbook or readings
instructor’s absence, to identify course assigned); and (3) those that the in-
strengths and areas for change. The structor either cannot or, for pedagogi-
consultant summarizes this information cal or curricular reasons, will not
and meets with the instructor to discuss change (e.g., the number of tests,
students’ perceptions and pedagogical specific content).
options. Research at the University of At the class meeting after an evalua-
Washington on the use of class inter- tion exercise, the instructor should
views indicates that students appreciate thank students for their comments and
the opportunity to provide feedback to clarify any confusion or misunderstand-
their instructor before the end of the ings noted in those comments about the
quarter. instructor’s goals or the students’
expectations. The instructor should
Response to Students’ Concerns then indicate which suggestions would
It is important that the issues posed be implemented this term, those that
in a midsemester evaluation be ones to must wait until the course is next
which the instructor will be able to offered, and those on which action
respond during the term. Otherwise, cannot or will not be taken. In the third
students may develop false expectations case, it would be helpful to explain
about the remainder of the course. briefly the reasons for this decision.

Formative Evaluation by • Asking teaching assistants to

Graduate Teaching Assistants review examinations and quizzes before
Teaching assistants can be an invalu- they are given to students. Having
able source of feedback for faculty participated in the course, teaching
members about successes and problems assistants can identify ambiguous or
that are occurring in classes, discussion unclear exam items before the tests are
sections, and teaching laboratories. administered. After midterms or quiz-
Such feedback can be especially illumi- zes have been graded, teaching assis-
nating if teaching assistants are encour- tants can provide detailed information
aged to attend class sessions regularly about patterns of error or misunder-
and to meet with the faculty member in standing. Collecting this kind of infor-
charge of the course and with each mation from a number of teaching
other. Ways in which teaching assis- assistants from different courses, from
tants can provide appropriate feedback sections within a course, and over an
to individual faculty and to their aca- extended period of time can also enable
demic department include the following: departments to determine which con-
cepts need to be reinforced in several
• Encouraging teaching assistants to courses or which misconceptions
provide information throughout the persist as students advance through the
term about the difficulties students may curriculum.
be having in the courses with which the • Soliciting from teaching assistants
teaching assistants are involved. constructive suggestions on aspects of a
Through conversations with and direct course or the department’s programs,
observation of students in the course, such as websites, laboratory offerings,
teaching assistants can tell an instructor and similarities and differences in
what aspects of the course readings, approaches to teaching and assessing
assignments, and presentations are student learning in different sections of
causing problems for students. Such the same course.
information is more likely to be offered
Formative Evaluation by
if instructors make it clear that identify-
Faculty Colleagues
ing students’ difficulties is a normal and
expected part of a teaching assistant’s Traditionally, faculty members have
responsibilities. Some faculty ask willingly and candidly judged their
teaching assistants to give them brief colleagues’ scholarly work through a
weekly reports on the one or two things variety of means (see Chapter 3) but
that cause students the most difficulty. have hesitated when asked to judge

their colleagues’ teaching effectiveness. of such critical aspects of teaching as

Yet many senior faculty have the back- course organization, teaching methods,
ground and perspective needed to and the instructor’s choice of appropri-
provide judgments about such matters ate assessment practices.
as the candidate’s knowledge of the
subject, course content, appropriateness Observation
of course objectives and instructional Instructors who are being evaluated
materials, examination skills, testing can ask a mentor, colleague, or instruc-
proficiency, and breadth and depth of tional improvement specialist at the
student learning. Under the right campus or discipline-based teaching and
circumstances, these judgments can be learning center to visit their classes and
used to assist in summative evaluations provide feedback on their teaching.
of faculty (see also Chapter 4). Similar Prior to each visit, instructors can
judgments from colleagues also can be discuss with observers the specific
useful in formative evaluations for classroom issues or techniques on
professional development of faculty. which the observers should focus (e.g.,
At small institutions or in very small student–teacher interaction, the nature
departments, a lack of resources or of questions posed, use of class time,
limited numbers of faculty may make and other issues important to the
faculty input more difficult to obtain instructor).
than in larger institutions or depart- Faculty also can ask colleagues,
ments. In addition, friendships or particularly those known to be excellent
rivalries that arise within any depart- teachers, for permission to visit their
ment may be amplified in smaller courses. Visitors can note the specific
departments. In such cases, balanced techniques used by the colleague in
and objective evaluations of teaching leading discussions, conducting teach-
colleagues may be achieved only by ing laboratories, and so on. If time
including in the evaluation process permits after class, the observing and
additional faculty from outside the observed faculty members can discuss
academic unit of the person being their respective teaching philosophies,
evaluated. Even when these issues do goals, instructional methods, out-of-
not surface, engaging faculty from class preparation, and similar matters.
outside the department, particularly It is usually most helpful for a faculty
those who are knowledgeable about member to attend a series of classes
effective pedagogies for promoting (say, all classes dealing with a specific
student learning, should enable review topic or issue) to obtain a broad per-

spective on the range of pedagogical students learn and how to improve their
approaches used by the colleague in his teaching. Referred to as the Master
or her teaching. Faculty Program, this initiative involves
faculty working together in pairs or in
Role of Colleagues in “Formal” triads. Faculty members observe each
Formative Evaluation other’s classes and interview each
Informal discussions and efforts to other’s students several times during
improve instruction among faculty the semester. Interviewers’ questions
members take place daily, but some emphasize student learning in the
departments and institutions employ course (for example, topics that may be
more systematic and formal efforts to difficult or reactions to specific class
assist in the improvement of teaching sessions). With these observations in
through formative evaluation. In addi- hand, the faculty participating in the
tion to the evaluation questionnaires program meet periodically to discuss
reprinted in Appendix C, the following candidly, and confidentially, how each
approaches to formative evaluation can participant has or has not fostered
be especially useful for the purposes of student learning. Chandler (1991) has
faculty professional development. documented the generally positive
results of this type of program involving
Faculty mentoring faculty. Increasingly, some 300 faculty at 21 different colleges
departments are assigning senior and universities.
faculty as mentors to untenured faculty.
Boice (1992) found that it was not Formative evaluation by faculty col-
necessary for successful mentors to be leagues from other institutions. Faculty
from the same department. Whether at higher education institutions across
from within or outside of the faculty the country and around the world can
member’s department, the ideal faculty provide formative evaluation to col-
mentor appears to play four major roles: leagues via the Internet. They can
friend, source of information, and career comment on the content of a faculty
and intellectual guide (NRC, 1997b; member’s websites for courses, old
Sands et al., 1991). examination questions, assignments,
At a variety of higher education and student responses to questions
institutions, Katz and Henry (1988) posed by the faculty member. This kind
developed a strategy of transdisciplinary of input from colleagues at other institu-
mentoring based on faculty working tions could be included as part of a
together to understand both how teaching portfolio or dossier for

summative evaluation, but also has teaching and bring to the scholarship of
great potential for ongoing formative teaching the same kinds of recognition
feedback. and reward afforded for other forms of
scholarly work (Hutchings, 2000).6
Projects of the American Association for Examples of the criteria being advanced
Higher Education. The American Asso- for evaluating a faculty member’s
ciation for Higher Education (AAHE) scholarship in teaching are presented in
has promoted collaboration in assessing Box 5-1, excerpted from Glassick et al.
and improving teaching through a (1997, p. 36). Centra (2001) has ex-
variety of projects. One such project, tended these criteria to allow for evalua-
conducted in the mid-1990s, involved 12 tion of the scholarship of teaching and
universities and stressed peer review as learning as practiced by academic
a means of formative evaluation. In this departments and institutions (see Box 5-
project, participants monitored their 2).
progress in improving student learning.
AAHE’s (1993) Making Teaching Com- Self-Evaluation
munity Property: A Menu for Peer Self-reports and self-reflections on an
Collaboration and Peer Review provides instructor’s teaching and promotion of
many other examples of peer review student learning can be important
efforts that contribute to formative sources of information for evaluating a
evaluation and improved professional teacher’s effectiveness (Hutchings,
development in teaching for faculty. 1998). These self-reports, which may be
More recently, AAHE, the Carnegie part of a required annual report or a
Foundation for the Advancement of teaching portfolio, are more useful and
Teaching, and the Carnegie Academy appropriate for formative or professional
for the Scholarship of Teaching and development purposes than for
Learning jointly developed a program summative personnel decisions. Faculty
for peer collaboration based on ideas who have not previously performed self-
and criteria advanced by Boyer (1990) evaluation may require assistance from
and Glassick and colleagues (1997). teaching and learning centers.
The goals of the program are to support As a summary of a professor’s major
the development of a scholarship of teaching accomplishments and
teaching and learning that will foster
significant, long-lasting learning for all
6
students. The program also seeks to Additional information about this program is
available at <http://www.carnegiefoundation.
enhance the practice and profession of org/CASTL/index.htm>.

Box 5-1. Evaluating the Scholarship of Teaching
Clear Goals: Does the scholar state the basic purposes of his or her work
clearly? Does the scholar define objectives that are realistic and achievable? Does
the scholar identify important questions in the field?
Adequate Preparation: Does the scholar show an understanding of existing
scholarship in the field? Does the scholar bring the necessary skills to his or her
work? Does the scholar bring together the resources necessary to move the project
forward?
Appropriate Methods: Does the scholar use methods appropriate to the
goals? Does the scholar apply effectively the methods selected? Does the scholar
modify procedures in response to changing circumstances?
Significant Results: Does the scholar achieve the goals? Does the scholar’s
work add consequentially to the field? Does the scholar’s work open additional areas
for further exploration?
Effective Presentation: Does the scholar use a suitable style and effective
organization to present his or her work? Does the scholar use appropriate forums for
communicating work to its intended audiences? Does the scholar present his or her
message with clarity and integrity?
Reflective Critique: Does the scholar critically evaluate his or her own work?
Does the scholar bring an appropriate breadth of evidence to his or her critique?
Does the scholar use evaluation to improve the quality of future work?
SOURCE: Glassick et al. (1997, p. 36).
strengths (Shore et al., 1986), the how various materials were used in
teaching portfolio may include the teaching, innovations the instructor has
following kinds of evidence of teaching attempted and an evaluation of their
effectiveness: success, videotapes of teaching)
• Material or assessments from
• Development of new courses
others (student work and evaluations,
• Products of good teaching (for
input from colleagues or alumni)
example, student workbooks or logs,
• Descriptions of how the individual
student pre- and post-examination
has remained current in the field, such
results, graded student essays)
as using knowledge gained from attend-
• Material developed by the indi-
ing professional conferences (Edgerton
vidual (course and curriculum develop-
et al., 1991; Shore et al., 1986)
ment materials, syllabi, descriptions of

Box 5-2. Framework and Examples of Practices/Policies for

Evaluating the Scholarship of Teaching
Dimensions of the Scholarship of Teaching
Focusing on Teaching
Making Teaching Practices and Learning Having Content and
Public Outcomes Pedagogical Knowledge
Departments That Practice the Scholarship of Teaching
• Have a system of • Administer major • Sponsor seminars

peer review of field-level exams or workshops on
teaching. or other teaching in the
• Discuss teaching assessments. discipline.
and subject • Encourage team • Encourage diverse
content topics at teaching or approaches to
department interdisciplinary teaching.
meetings. courses. • Reward staff who
• Encourage • Encourage publish or give
members to teaching conference papers
prepare teaching innovations. on teaching.
portfolios. • Encourage
• Have a mentoring research on
system for teaching and
teaching. learning.
• Encourage • Seek student
classroom visits perceptions on
and discussions of teaching
teaching. practices, learning
• Support practices, and
attendance at learning
conferences and outcomes.
workshops on
teaching.

Institutions That Practice the Scholarship of Teaching
• Encourage student • Have active • Reward teachers/

evaluations of programs or departments that
teaching. centers to support promote the use of
• Support a teaching and means by which
mentoring learning. discipline
program for • Have training knowledge can be
teachers. program for related to
• Sponsor seminars, teaching students.
workshops, or assistants. • Have staff
conferences on • Provide grants to development
teaching and support research programs that
learning. on teaching and emphasize diverse
• Require/ learning. teaching
encourage faculty • Have a plan for practices.
to prepare assessing student-
teaching portfolios learning
or detailed reports outcomes.
on teaching. • Survey students
• Periodically and graduates on
review teaching. learning
• Publish results of experiences.
learning outcome • Use evidence of
and teaching student learning in
environment hiring and
surveys. promotion
• Weigh teaching decisions.
performance • Reward the use
heavily in hiring and development
and promotion of effective
decisions. teaching
• Encourage a peer practices.
review program.
SOURCE: Centra (2001, pp. 8–9).

• External support obtained for such semester has been completed, the
purposes as improving teaching or instructor prepares a similar brief
purchasing instrumentation for teaching description of the types of students who
laboratories actually enrolled, the instructional
methods that were used, and how the
Videotaping students’ achievement of major goals
Videotaping is a useful strategy that was measured. The evaluation should
enables instructors to see what they do address (1) goals the instructor believes
well and what needs to be improved. In were met and evidence of student
consultation with an expert from the learning and academic achievement, (2)
campus’s teaching and learning center, goals that were not realized, (3) the
instructors can determine whether they nature of and possible reasons for
exhibit such classroom behaviors as discrepancies between the instructor’s
dominating a discussion, allowing original intentions and actual outcomes,
students enough time to think through and (4) how the instructor might modify
questions, or encouraging all students the course in the future to achieve more
to participate in discussions. Faculty of the intended goals. These self-
who have been videotaped find the assessments can become part of a
experience extremely helpful, especially teaching portfolio that can later be used
if they discuss the analysis with some- for more summative types of evaluation.
one having expertise in classroom Another form of before-and-after
behavior. Videotaping is best used for assessment may help instructors who
formative evaluation. are interested in examining their teach-
ing behaviors and effectiveness rather
Before-and-After Self-Assessment than course outcomes. For this tech-
Faculty members can use before-and- nique, instructors use the end-of-course
after self-assessment to determine evaluation form, but complete the
whether course outcomes meet their questionnaire before their course
expectations. Before a course begins, begins (predicting how they think they
the instructor writes brief comments will do) and again at the end of the
about the types of students for whom semester (how they believe they did).
the course is intended. Given that They also may wish to fill out a ques-
audience, the instructor lists the most tionnaire at the end of the term based
important course and learning goals and on what they expect, on average, their
the teaching strategies she or he will students will say about their teaching.
design to achieve them. Once the In most cases, such self-evaluations are

likely to be more positive than student • workload assigned and perceived

ratings (Centra, 1973; Feldman, 1989). difficulty of a course;
In looking at the results, instructors • quality and fairness of grading,
may wish to focus on any deficiencies assignments, and examinations;
noted in the self-evaluation or on dis- • students’ ratings of their own
crepancies between their own evalua- learning and progress; and
tions and those of their students. • students’ ratings of their level of
effort, attendance, and participation in
the course, completion of assignments,
SUMMATIVE EVALUATION and motivation.
OF TEACHING
Questionnaires used for student
evaluations sometimes address aspects
Evaluations from of a faculty member’s teaching style that
Undergraduate Students may or may not contribute to student
Questionnaires are most commonly learning. For example, they may ask
used for summative student evaluations whether the faculty member makes eye
of teaching. The questionnaires can be contact with students during discus-
machine-scored and fall into two catego- sions, how many questions the instruc-
ries: those developed locally by campus tor poses during class (as compared
teaching and learning centers by con- with the nature of the questions), or
sulting the literature or adapting forms how often students may be assigned to
used elsewhere, and those developed by work in groups rather than work alone.
other institutions or organizations and Such questions are appropriate only if
made available for a fee. they are explicitly intended to provide
Questionnaires vary somewhat in the formative feedback for the instructor,
characteristics of teachers and courses but should not be used for summative
covered, as well as in the quality and purposes. Each instructor has a unique
usefulness of the scores generated for personality, persona, and approach to
the instructor. Typically, student teaching. The primary concern when
evaluation instruments have attempted developing or analyzing questions on
to identify strengths and weaknesses of student questionnaires for purposes of
instructors in the following areas: summative evaluation should be
whether the students are actually
• organization or planning;
learning at the desired level and in ways
• teacher–student interactions;
that are consistent with the course goals
• clarity and communication skills;
(Rosenthal, 1976).

Global ratings of the course overall or dent evaluations for summative pur-
the teacher’s instructional effectiveness poses is provided in Appendix A.
also are common to most student
questionnaires. For courses in science Summative Evaluation by
and engineering, special questions Graduate Teaching Assistants
about the efficacy of laboratories, If a department wishes to involve
fieldwork, and research experiences teaching assistants in performing
also can be included as part of the summative evaluations of faculty or
standardized form or posed in a sepa- improving a department’s educational
rate questionnaire. For example, the offerings and approaches to teaching
University of Washington provides and learning, both the teaching assis-
separate evaluation forms for laborato- tants and faculty must feel confident that
ries, as well as for clinics and seminars the procedures for gathering informa-
(e.g., University of Washington Educa- tion will preserve the assistants’ ano-
tion Office of Educational Assessment7 ). nymity. Teaching assistants need to
Appendix B provides more specific know before participating how the
information about and several examples information will be used and who will
of student questionnaires for evaluating see the data.
undergraduate teaching. See also Davis When evaluations from teaching
(1988) for compilation of questions that assistants are to be used for personnel
can be used on an end-of-course ques- decisions, the department might con-
tionnaire. sider asking for written assessments.
It is important to note that question- Alternatively, a system might be estab-
naires usually do not permit students to lished whereby teaching assistants
assess such characteristics as an would be interviewed informally by a
instructor’s level of knowledge of member of the evaluation committee
subject matter. Students cannot and and their comments recorded and
should not evaluate instructors in this submitted collectively. In either case,
regard. Instead, faculty peers and teaching assistants should be asked to
colleagues should assess these charac- indicate the basis for their assessment.
teristics of an instructor’s teaching. Such information might include the
Additional detail about the use of stu- number of courses they have taught
with the instructor, descriptions of their
training and supervisory activities, the
7
nature and amount of their contact with
www.washington.edu/oea/>. undergraduate students, whether they

were allowed to obtain informal student Summative Evaluation by

opinions about the course, and the Faculty Colleagues
extent to which they observed each The following approaches might help
major aspect of the course (e.g., lecture, some institutions obtain more system-
laboratory). atic and complete information on teach-
Teacher assistants can be asked for ing performance for purposes of
the following kinds of information: summative evaluation. When these
• An overall judgment of the effec- approaches could also be useful for
tiveness of the faculty member’s teach- formative evaluation, this is noted.
ing.
Ad Hoc Committees on
• An analysis of the particular
Teaching Effectiveness
strengths and weaknesses of the teach-
ing as reflected in the design, prepara- The department might appoint an ad
tion, and conduct of the course. If the hoc committee on teaching to evaluate
department wants specific comments on each faculty member who is being
particular aspects of teaching, the considered for tenure or promotion. At
instructions to the teaching assistants smaller institutions, where final deci-
should emphasize the need for support- sions for promotion and tenure may rest
ing evidence. with an institution-wide committee
• The extent to which working with rather than individual departmental
the instructor contributed to the teach- committees, a similar panel separate
ing assistant’s own professional develop- from the committee on tenure and
ment in teaching. promotion could be established regu-
• The appropriateness of the larly to review the institution’s policies
instructor’s assignments and expecta- with regard to the process and use of
tions of the teaching assistants. summative evaluations for teaching.
The only responsibility of such ad hoc
For each question posed, the teaching committees would be to evaluate teach-
assistants should be encouraged to ing performance. The committee could
supply specific examples. If their consist of senior faculty members, one
responses are summarized for person- or two junior faculty members, and one
nel decisions, the summary must or more graduate or senior-level under-
indicate the number of teaching assis- graduate students. One or more of
tants who worked with the faculty these ad hoc committee members
member and the number from whom should be from outside the candidate’s
information was obtained. department.

The materials to be considered by the What were the strong and weak points
committee could include a variety of of your teaching? What would you
teaching-related materials, all of which change or do differently the next time
would be supplied by the candidate: you teach the course? What did you
course syllabi and examinations, teach- find most interesting and most frustrat-
ing and learning aids, and evidence of ing about the course?
the impact of the candidate’s teaching The candidate’s department chair also
on students’ learning and intellectual could provide the committee with
growth. The faculty member also could student evaluations from courses taught
be asked to submit documentation for previously, names and addresses of
the following: currency of course student advisees, dissertation advisees,
content, participation in the design of enrollees in past and current courses,
courses, contributions to curriculum and the candidate’s cumulative teaching
and instruction, supervision of student portfolio if one has been prepared. The
research, advising duties, preparation of candidate should see the list of materi-
teaching assistants (if appropriate), and als submitted to the committee and be
individual and collaborative efforts to given the opportunity to supplement it.
improve teaching effectiveness. Through brief interviews, telephone
Candidates should also prepare and calls, letters, or brief survey question-
submit a self-assessment of their teach- naires issued to the candidate’s current
ing effectiveness. The self-assessment and former students from a variety of
could address questions such as the courses, the committee could compile a
following: What are the goals of your picture of students’ views of the teacher
teaching? Why were these goals se- that would supplement the written
lected? How did you know whether evaluation reports from past courses. In
students were gaining competence and addition, each committee member could
learning the material? How well did the observe and evaluate at least two of the
courses meet your learning goals for candidate’s classes.
your students, and how do you know? Studies of such ad hoc committees
What problems, if any, did you encoun- revealed that members met several
ter in attempting to meet these goals? times to discuss their individual find-
How did you conduct the course and ings, used a rating form, and prepared a
challenge and engage students? How report, which was then submitted to a
did your methods take into account the departmental tenure and promotion
levels and abilities of students? How committee (see Centra, 1993, pp. 129–
satisfied were you with the course? 131 for details). Given the highly

positive reliability coefficients reported A form designed by French-Lazovik

by Root (1987) for colleague evaluations (1981) is also provided in Appendix C.
when the colleagues are properly This form offers five broad questions
prepared, one can conclude that the with which faculty peers can evaluate
assessments of a faculty member’s such dimensions as the quality of
teaching effectiveness thus provided are materials used in teaching. The form
reliable. also lists which portfolio materials
should be reviewed and suggests a
Colleagues’ Evaluation Questionnaires focus for colleagues when examining
Several questionnaires have been these materials. Other institutions have
designed to elicit colleagues’ evaluation developed more extensive guides to
of a candidate’s teaching effectiveness help candidates prepare for peer evalua-
for summative evaluation purposes, tion and to assist faculty colleagues in
although they may also be used for conducting such evaluations effectively
formative evaluation. Two forms devel- (e.g., the University of Texas’s Prepar-
oped at Syracuse University and the ing for Peer Evaluation;8 see also the
University of Texas at Austin provide many resources available through the
scaled-response items and open-ended websites of college and university
questions that faculty colleagues and teaching and learning centers through-
department chairs can use to guide out the United States and in other
their analysis of a candidate’s chosen countries).9
instructional materials, as well as While the kinds of forms included in
teaching behaviors they observe during Appendix C have proven helpful to
classroom visits. These forms, printed faculty in identifying what materials and
in their entirety in Appendix C, cover characteristics of a candidate’s teaching
questions grouped under the following to assess, the reliability and validity of
five characteristics of good teaching: their evaluations depend on the use of
• organization of subject matter and

course,
• effective communication,
8
• knowledge of and enthusiasm for www.utexas.edu/academic/cte/PeerObserve.
subject matter and teaching, html>.
9
A list of websites for teaching and learning
• fairness in examinations and
centers of colleges and universities in Asia,
grading, and Australia and New Zealand, Europe, and North
• flexibility in approaches to teach- America is available at <http://eagle.cc.ukans.
edu/~cte/resources/websites.html>.
ing.

appropriate procedures. For example, drafting the policies and procedures to

as noted above, Root’s (1987) study be implemented at the departmental
indicated that a minimum of three and institutional levels, and determining
departmental colleague evaluators the procedures to be used for analyzing
should use the form. They should and reviewing the results of summative
discuss the evaluation criteria before evaluations of teaching.
reviewing materials and making class- • Recognizing and addressing as
room visits. Evaluators also should be part of the system of evaluation the full
provided with examples of evaluations range of teaching styles and activities,
from other candidates, both internal and both in and out of class. Effective
external, that illustrate high and low evaluation systems should be able to
ratings. assess a broad range of teaching styles
and approaches.
• Making evaluation forms and
EFFECTIVE IMPLEMENTATION OF supporting documents freely available
EVALUATION METHODOLOGIES to faculty so they understand what
information will be considered legiti-
Before revising and implementing mate and relevant in the evaluation of
policies and procedures for evaluating their teaching performance.
teaching, especially for summative • Establishing uniform procedures
evaluation, stakeholders should proceed for collecting and using information
in ways that will confer maximum from students. For example, institution-
credibility on the results of their efforts. wide procedures should be defined that
Depending on the institution in ques- protect the anonymity of respondents
tion, administrators, the academic and ensure that instructors do not see
senate or committee on tenure and end-of-semester student evaluations
promotion, and faculty must accept that until after they have submitted their
the results of evaluation efforts will be grade reports.
helpful both in personnel decisions and • Establishing a uniform and equi-
in improving the teaching effectiveness table system for the analysis and review
of faculty. Policies and procedures that of evaluation data, including appropriate
could assist in the process include the response rates for end-of-course student
following: questionnaires.
• Making clear which letters and
• Closely involving the institution’s surveys will be kept confidential; which
faculty in selecting evaluation methods, can be seen by the faculty under review;

and which information, if any, will be Formative Discussions Between

shared with students for purposes of the Department Chair and
selecting future courses. Individual Faculty Members
Optimally, department chairs should
In addition, the following procedures meet at least annually with each mem-
could make any evaluation process ber of the department to discuss teach-
more equitable and more easily ac- ing accomplishments and issues. Such
cepted. meetings are especially critical for any
faculty member whose teaching evalua-
Regular Meetings Between New
tions are substantially below the
Faculty Members and the
department’s expectations or those of
Department Chair
other departmental colleagues. These
The department chair should meet meetings should occur well before
with each new faculty member and summative decisions are to be made so
make clear the department’s and the that candidates have ample opportuni-
institution’s general expectations and ties to develop a plan for improving their
policies regarding teaching. Norms of teaching. Additional meetings at regu-
grading for assigned courses should be lar intervals should be scheduled to
described. The chair also should assess progress in addressing concerns.
encourage the new faculty member to
consult with other department col- Sharing of Faculty-Generated
leagues who teach the same or related Teaching Portfolios
courses to develop policies and proce- The department’s academic person-
dures for establishing desired learning nel files could include a teaching portfo-
outcomes, pedagogical approaches, and lio for each faculty member. Faculty
methods for assessing learning (see, members could place in the portfolio
e.g., Annex Box 1-2 in Chapter 1). New copies of their course materials (includ-
faculty members should be encouraged ing learning objectives and expected
from the beginning of their employment outcomes), syllabi, reading lists, assign-
to contribute actively to such discus- ments, examinations, and instructional
sions. The chair also should encourage materials. A website also could be
and assist new faculty members to work established for this purpose. Depending
with faculty colleagues both within and on institutional policy, student evalua-
outside the department on improving tion forms or summaries of students’
their teaching, and possibly assign a course evaluations also could be in-
senior mentor to assist them. cluded in the portfolio. It should be

assumed that a faculty member would institution for improving their teaching.
continue to have access to all materials Members of such a panel might include
in his or her portfolio, unless letters a former recipient of the campus teach-
solicited or submitted in confidence ing award, a respected senior faculty
were protected under rules of the member who teaches introductory and
university. lower division courses, and a newly
tenured associate professor.
Feedback from Graduating
Seniors and Alumni Oversight Committee to Monitor
As part of the department’s regular Departmental Curriculum
academic program review, graduating and Instruction
seniors and alumni could be surveyed. The department chair could establish
Relevant survey information about an a permanent faculty committee to
individual instructor’s teaching effec- monitor the quality and effectiveness of
tiveness would be placed in his or her instruction by all members of the
teaching portfolio. Instructors should department. This committee would also
be made aware that such information oversee all evaluations of curriculum,
will be included in their portfolios and teaching, and student learning and,
be allowed to provide written comments where appropriate, nominate faculty for
or responses, where permissible. the campus’s or college’s teaching
awards.
Departmental Panel on Teaching
Effectiveness and Expectations Legal Considerations
In addition to an ad hoc department All stakeholders who are involved
committee to monitor candidates’ with the evaluation of teaching must act
progress in teaching, as discussed in accordance with institutional policies
above, the department as a whole could that have been designed to ensure
establish a faculty panel that would legally equitable and fair treatment of all
summarize the department’s policies involved parties. Such policies might
and procedures regarding expectations require, for example, that:
for teaching effectiveness, the methods
and criteria used to judge that effective- • The faculty be involved in the
ness, and the role of evaluation in design of an evaluation system, as well
academic personnel decisions. The as in evaluations of colleagues.
panel would remind faculty of the • The institution complies with all
resources available to them through the procedures specified in contracts or

handbooks, as both are legal docu- respond to individual evaluation reports

ments. or to clarify information in their dossiers
• The evidence that is used for or portfolios.
personnel decisions be job-related and • The procedures used in internal
nondiscriminatory. review of decisions be clearly elucidated
• The faculty members be allowed to and made available to all faculty.

Evaluation of Individual Faculty:

Criteria and Benchmarks
This report thus far has synthesized might be rendered by the department
the findings of research on evaluating or institution. This chapter presents
effective teaching, and has offered specific criteria that can be used when
specific recommendations to leaders in summative evaluations are undertaken.
the higher education community for These criteria are organized according
changing the climate and culture on to the five characteristics of effective
their campuses such that the evaluation teaching outlined in Chapter 2. It
of teaching will be valued, respected, should be emphasized that the criteria
and incorporated in the fabric of the suggested below are based on the
institution. The report also has empha- committee’s identification of best
sized that any system of teaching practices from an examination of the
evaluation should serve as a critical scholarly literature, but they are not
basis for improving student learning. exhaustive. Each evaluating depart-
The previous chapter provides a ment or institution is encouraged to
framework that departments and institu- select and, if necessary, modify those
tions can apply to evaluate the teaching criteria from the compendium presented
of individual faculty. It emphasizes the below that best suit its specific circum-
need for ongoing formative evaluation stances. As emphasized in Chapter 5,
that offers faculty members ample those who evaluate faculty teaching
opportunities, resources, and support should be careful to use multiple—and
systems for improving their teaching defensible—sources of evaluation,
prior to any summative evaluations that particularly for summative purposes.
100

1. KNOWLEDGE OF AND design of new products and processes.

ENTHUSIASM FOR Conveying that sense of excitement is
SUBJECT MATTER equally important in helping students
appreciate more fully the subject matter
Summarizing the discussion of this being taught.
characteristic from Chapter 2, effective
teachers: The following questions might be
posed for evaluation for this characteris-
• Understand and can help students tic:
learn and understand the general
principles of their discipline (e.g., the • Does the instructor exhibit an
processes and limits of the scientific appropriate depth and breadth of knowl-
method). edge?
• Provide students with an overview • Is the instructor’s information
of the whole domain of the discipline. current and relevant?
• Possess sufficient knowledge and • Does the instructor show continu-
understanding of their own and related ous growth in the field?
subdisciplines that they can answer
most students’ questions or know how Data sources and forms of evaluation
to help students find appropriate infor- for this characteristic are shown in
mation. Table 6-1.
• Keep their knowledge about a field
of study current through an active
research program or through scholarly 2. SKILL, EXPERIENCE, AND
reading and other types of professional CREATIVITY WITH A RANGE OF
engagement with others in their imme- APPROPRIATE PEDAGOGIES
diate and related disciplines (e.g., AND TECHNOLOGIES
participation in professional meetings
and workshops). Summarizing the discussion of this
• Are genuinely interested in—and characteristic in Chapter 2, effective
passionate about—the course materials teachers:
they are teaching. Practicing scientists,
mathematicians, and engineers under- • Have knowledge of and select and
stand and appreciate the infectious use a range of strategies that offer
enthusiasm that accompanies original opportunities for students with different
discovery, application of theory, and learning styles to achieve.
E VA L U AT I O N O F I N D I V I D U A L F A C U L T Y 101

TABLE 6-1 Data Sources and Forms of Evaluation for Evaluating Knowledge and Enthusiasm
for Subject Matter
How Evaluation Data Discussed in

Source Form of Can Be Used (formatively, Report Beginning
of Data Evaluation summatively, or both) on Page(s)
Students • Student evaluations • Both 76

• Interviews • Both 80
Faculty • Review of course materials • Both 63

Colleagues and other products
• Observation • Both 45, 51
Instructor • Written self-appraisal • Both 65

Under Review
• Are organized and clearly commu- fullest potential and then employ the
nicate to students their expectations for professional knowledge and skill neces-
learning and academic achievement. sary to assist them in overcoming
• Focus on whether students are academic difficulties.
learning what is being taught and view
the learning process as a joint venture The following questions might be
between themselves and their students. posed for evaluation for this characteris-
• Give students adequate opportu- tic:
nity to build confidence by practicing • Does the instructor clearly commu-
skills. nicate the goals of the course to stu-
• Ask interesting and challenging dents?
questions. • Is the instructor aware of alterna-
• Encourage discussion and promote tive instructional methods or teaching
active learning strategies. strategies and able to select methods of
• Persistently monitor students’ instruction that are most effective in
progress toward achieving learning helping students learn (pedagogical
goals through discussions in class, out- content knowledge)?
of-class assignments, and other forms of • To what extent does the instructor
assessment. set explicit goals for student learning
• Have the ability to recognize those and persist in monitoring students’
students who are not achieving to their progress toward achieving those goals?

TABLE 6-2 Data Sources and Forms of Evaluation for Evaluating Skill in and Experience with
Appropriate Pedagogies and Technologies

Current Students • Student ratings • Both 139

• Outcome assessment • Both 73
of learning

Colleagues and other evidence of
teaching effectiveness

Under Review
Data sources and forms of evaluation • Know whether students are truly
for this characteristic are shown in learning what is being taught.
Table 6-2. • Determine accurately and fairly
students’ knowledge of the subject
matter and the extent to which learning
3. UNDERSTANDING OF AND has occurred throughout the term (not
SKILL IN USING APPROPRIATE just at the end of the course).
TESTING PRACTICES
The following questions might be
Summarizing the discussion of this posed for evaluation for this characteris-
characteristic in Chapter 2, effective tic:
teachers:
• Is the instructor aware of a range
of tools that can be used to assess
• Assess learning in ways that are
student learning?
consistent with the learning objectives
• Does the instructor select assess-
of a course and integrate stated course
ment techniques that are valid, reliable,
objectives with long-range curricular
and consistent with the goals and
goals.
learning outcomes of the course?

• Are students involved in contribut- 4. PROFESSIONAL

ing to the development of the assess- INTERACTIONS WITH STUDENTS
ment tools used? WITHIN AND BEYOND
• Are assignments and tests graded THE CLASSROOM
carefully and fairly using criteria that
are communicated to students before Summarizing the discussion of this
they begin a task? characteristic in Chapter 2, effective
• Do students receive prompt and instructors:
accurate feedback about their perfor-
mance at regular intervals throughout • Meet with all classes and assigned
the term? teaching laboratories, post and keep
• Do students receive constructive regular office hours, and hold exams as
suggestions on how to improve their scheduled.
course performance? • Demonstrate respect for students
as individuals; this includes respecting
Data sources and forms of evaluation the confidentiality of information
for this characteristic are shown in gleaned from advising or student
Table 6-3. conferences.
TABLE 6-3 Data Sources and Forms of Evaluation for Evaluating Proficiency in Assessment


• Interviews with selected • Both 59
students

Colleagues and other evidence of
teaching effectiveness

Under Review
Institutional • Grade distribution • Summative 66

Records

• Encourage the free pursuit of • Taking into account differences in

learning and protect students’ academic the difficulty and cost of undertaking
freedom. research in various disciplines, under-
• Address sensitive subjects or graduate research experiences should
issues in ways that help students deal engage students in interesting and
with them maturely. challenging projects that help them
• Contribute to the ongoing intellec- develop additional insight into and
tual development of individual students understanding of science, as well as the
and foster confidence in their ability to specific topic on which they are work-
learn and discover on their own. ing. How active has the instructor been
• Act as an advisor to students who in directing student research projects
are having problems with course mate- and independent studies? What is the
rial and know how to work with such caliber of these student projects? To
students in other venues besides the what extent has the instructor fostered
classroom to help them achieve. When independent and original thinking by
a student clearly is not prepared to students and inspired them to develop
undertake the challenges of a particular sufficient independence to pursue the
course, the effective instructor may subject on their own? Have students
counsel that student out of the course or been encouraged to participate in
suggest alternative, individualized professional meetings? Has student
approaches for the student to learn the work led to professional publications or
subject matter that is prerequisite for acknowledgments?
the course. • Does the instructor take an active
• Uphold and model for students the interest in advisees’ individual academic
best scholarly and ethical standards and career choices? How well informed
(e.g., University of California Faculty is the instructor about department and
Code of Conduct).1 university policies and procedures that
concern advisees? Does the instructor
The following questions might be provide sufficient office time for stu-
posed for evaluation for this characteris- dents to obtain clarification and guid-
tic: ance?
• How effectively does the instructor
train and supervise teaching assistants
assigned to his or her courses? How
1
The University of California System’s Faculty does the instructor contribute to the
Code of Conduct Manual is available at <http://
www.ucop.edu/acadadv/acadpers/apm/>. professional development of teaching

TABLE 6-4 Data Sources and Forms of Evaluation for Evaluating Professionalism with
Students Within and Beyond the Classroom


• Interviews • Summative 139
• Special surveys • Summative 93
Former Students • Retrospective assessment • Both 60
Teaching • Written appraisal • Both 60

Assistants
Faculty • Review of instructor’s • Both 63

Colleagues contributions to curriculum
design and development
Instructor • Written self-appraisal • Summative 65, 93

Under Review
assistants? Does the instructor treat his needs to be seen as a scholarly pursuit
or her assistants with courtesy and as that takes place in collaboration with
professional colleagues? departmental colleagues, faculty in
other departments in the sciences and
Data sources and forms of evaluation engineering, and even more broadly
for this characteristic are shown in across disciplines. Such conversations
Table 6-4. enable faculty to better integrate the
course materials they present in their
courses with what is being taught in
5. INVOLVEMENT WITH AND other courses.
CONTRIBUTIONS TO ONE’S Summarizing the discussion of this
PROFESSION IN ENHANCING characteristic in Chapter 2, effective
TEACHING AND LEARNING teachers:
Much can be learned from teachers • Work with colleagues both on and
who work with colleagues both on and beyond campus, collaborating with
beyond the campus. Effective teaching departmental colleagues; faculty in

other departments in the sciences, • Has the instructor made notewor-

mathematics, and engineering. thy contributions to the design and
• Work to better integrate the materi- development of the department’s
als they present in their courses with curriculum? Has the instructor pro-
what is being taught in other courses. duced valuable instructional materials
or publications related to teaching
The following questions might be effectiveness or classroom activities?
posed for evaluation for this characteris- Has the instructor been involved in
tic: efforts to improve education or teaching
within the discipline or across disci-
• During the term, has the instructor plines? Has the instructor participated
specifically elicited feedback from in seeking external support for instru-
students, colleagues, or instructional mentation or education research
experts (e.g., from the campus teaching projects?
and learning center) about the quality of
his or her teaching? Data sources and forms of evaluation
• To what extent does the instructor for this characteristic are shown in
meet his or her teaching obligations and Table 6-5.
responsibilities?
TABLE 6-5 Data Sources and Forms of Evaluation for Evaluating Professional Involvement and
Contributions


• Formative procedures • Formative 61

Under Review • Grant applications 46
• Publications 48
Colleagues from • Written reviews of work • Both 79

Within and
Outside the
Institution

Evaluation of Departmental
Undergraduate Programs
The discussion in this report thus far forces effective teaching practices and
has focused primarily on attributes of student learning. This position is
effective teaching by individual faculty consistent with the National Science
members. The central theme has been Foundation’s report Shaping the Future
that evidence of high-quality student (NSF, 1996, pp. 63–64), which recom-
learning should be the major criterion mends that college and university
for measuring a faculty member’s governing boards and academic admin-
teaching effectiveness. The report has istrators:
also emphasized the importance of
• Accept responsibility for the
using multiple indicators and different
learning of all students and make that
kinds of evaluators (e.g., students,
clear not only by what the institution
alumni, graduate assistants, colleagues),
says but also by putting in place
as well as increasing reliance on ongo-
mechanisms to discharge that respon-
ing formative evaluation, to provide a
sibility at the institutional and depart-
more holistic view of an individual’s
mental levels.
teaching effectiveness.
• Hold accountable and develop
The committee believes that similar
reward systems for departments and
expectations can and should apply to
programs, not just individuals, so that
academic departments and colleges.
the entire group feels responsible for
Departments should regularly evaluate
effective STEM (science, technology,
their current undergraduate programs
engineering, and mathematics)
and their commitment to fostering an
learning for all students.
environment that recognizes and rein-
108

• Provide resources to ensure that [AAHE], 1993; Wergin, 1994; Wergin

faculty, particularly new faculty, have and Swingen, 2000; Wyckoff, 2001).
the opportunity to both learn how to Because the organization and roles of
and have the time to design effective academic departments vary so widely
instruction, use technology appropri- within and among institutions, and
ately, foster inquiry-based and col- especially among disciplines (Diamond
laborative learning, and assess and Adams, 1995, 2000), the task of
learning achieved. performing any kind of systematic
• Make sure that the faculty evaluation of these entities would appear
reward system, in practice as well as to be nearly insurmountable. However,
in theory, supports faculty who a number of reports have suggested
effectively help students learning in how members of academic departments
hospitable environments that recog- might assume collective responsibility
nize individual students’ differences for developing a coherent set of courses,
and that provide reasonable opportu- programs, and other educational experi-
nities to address those differences. ences that can enable all participating
Academic departments serve many students to maximize their opportuni-
roles, including general education of ties to learn (e.g., Shulman, 1993;
nonmajors, professional preparation of Wergin, 1994; Wergin and Swingen,
majors, contributions to interdiscipli- 2000). In addition, some disciplines
nary or honors programs, and profes- have developed guidelines for evaluat-
sional preparation of teachers and ing undergraduate programs (e.g., for
health professionals. Departments can chemistry, American Chemical Society,
encourage and support their members 1992; for earth sciences, Ireton et al.,
to work collectively to integrate courses 1996; for engineering, Accreditation
and curricula and improve teaching and Board for Engineering and Technology,
learning. They also can redirect their 1998; for mathematics, Mathematical
physical and financial resources to Association of America, 2000). How-
encourage continual improvement in ever, many of these guidelines focus
teaching and learning. In summary, primarily on defining what is expected
academic departments can become of students who will major in those
both the primary units for catalyzing subjects. Little attention has been paid
change in undergraduate education and to defining a quality education for other
true learning communities (American students who enroll in courses primarily
Association for Higher Education to fulfill graduation requirements for
E VA L U AT I O N O F D E P A R T M E N T A L U N D E R G R A D U AT E P R O G R A M S 109

future teachers (see McNeal and learning. Other issues on which depart-
D’Avanzo, 1997). mental members might focus include
There is growing consensus on the classroom teaching, academic advising
characteristics of effective undergradu- for students, and the roles of teaching
ate programs in STEM but too little laboratories and independent research
effort has been expended to date on opportunities in enhancing student
determining how measures of quality learning. Faculty and administrators
might be made more consonant and also need to reach consensus on the
consistent with national efforts to underlying assumptions, guidelines, and
improve undergraduate STEM educa- metrics they will use to improve under-
tion (e.g., Boyer Commission, 1998; graduate programs.
National Research Council [NRC], Many of the issues surrounding the
1995a, 1996a, 1999a; NSF, 1996, 1998; evaluation of teaching for individual
Rothman and Narum, 1999) or to align faculty also apply to the collective
such programs more closely with performance of academic departments.
national standards and benchmarks in The principles set forth in this report for
these disciplines for grades K–12 evaluating the teaching effectiveness of
(American Association for the Advance- individuals can easily be reshaped to
ment of Science [AAAS], 1993; Interna- apply to academic departments. This
tional Technology Education Associa- chapter lays a foundation for such
tion [ITEA], 2000; National Council of discussions.
Teachers of Mathematics [NCTM], Unlike the rest of the report, this
1989, 2000; NRC, 1996b). chapter offers no findings or recommen-
Members of academic departments, dations. Instead, it articulates a series
in conjunction with the principal aca- of questions that members of depart-
demic and executive officers on their ments might ask themselves and each
campuses, need to examine critically the other as they examine their unit’s role in
criteria they currently use to evaluate fostering the improvement of under-
the efficacy of their approaches to graduate education. These questions
undergraduate education. The first step are organized in accordance with the
in accomplishing this task is for each major responsibilities of departments in
department to adopt a mission state- the STEM disciplines.
ment on improving teaching and student

EVALUATING A DEPARTMENT’S to help faculty members become famil-

ABILITY TO ENHANCE TEACHING iar with this research and its implica-
AND LEARNING IN CLASSROOMS tions for improving teaching and learn-
AND OTHER VENUES ing?
Engaging student interest in the Employing effective pedagogy:

department’s curricular offerings:
• Has the department examined
• Does the department encourage ways in which teaching effectiveness
faculty members to discuss how to and student learning can be enhanced
employ the most effective teaching in large classes, especially large sec-
techniques and educational experiences tions of introductory courses?
for students with various educational • Has the department established
backgrounds and aspirations? Are the protocols for evaluating teaching based
department’s programs designed to on the kinds of criteria described in this
engage and excite students about the report? Have members of the depart-
discipline specifically and about STEM ment been trained to undertake evalua-
generally? tive procedures such as peer review of
• Does the department evaluate the teaching (e.g., Bernstein and Quinlan,
effectiveness of courses for nonmajors 1996; Huber, 1999)?
and for preparation of students pursuing • Has the department developed
other science or engineering majors, expectations regarding the teaching
especially for prospective elementary expertise of new hires?
and secondary teachers? • Does the department support
faculty who become engaged in active
Applying research on human scholarship on teaching and learning?
cognition and learning: Have guidelines been established for
evaluating such work for personnel
• Does the department encourage decisions? Does the department en-
faculty to base instructional techniques courage and support graduate students
on modern research on human cogni- to pursue future faculty programs
tion and learning (e.g., NRC, 1997a, designed to introduce them to issues
2000e)? and scholarship in teaching and learn-
• Does the department sponsor ing?
seminars, workshops, or other activities

Assessing student learning: tional programs incorporate cutting-

edge topics and skills of the discipline
• Does the department encourage and present them to students in ways
faculty to discuss ways of optimizing the that are pedagogically appropriate?
assessment of student learning and • Does the department encourage
provide sufficient time and resources to colleagues to focus some of the
support such efforts? coursework at both the introductory
• Are student learning outcomes and upper levels on real-world applica-
considered a primary criterion when tions and on connections between
assessing the success of the STEM and other disciplines?
department’s curriculum and programs? • Do members of the department
seek ways to provide students who will
Emphasis on improving teaching never again have formal exposure to the
and learning in introductor y and sciences, mathematics, or engineering
lower division courses: with the intellectual skills and back-
ground needed to appreciate and en-
• Have members of the department gage in lifelong learning in these disci-
agreed on the role and mission of plines?
introductory courses for both majors • Does the department offer encour-
and nonmajors? agement and funding to purchase,
• Does the department encourage maintain, and integrate into undergradu-
faculty members to work together in ate courses cutting-edge tools and
structuring the subject matter of and technologies (e.g., information technol-
approaches to teaching introductory ogy, real-time data acquisition and
courses? processing, remote sensing) so that
• Do introductory courses meet the students can better appreciate and
educational needs of those who will experience how advances in the disci-
become the next generation of students pline are achieved?
in STEM, future teachers, and • Given the increasing proliferation
nonmajors in the discipline? of always-available databases, real-time
data acquisition through remote sensing
Incorporating advances in the and instrumentation, and similar ad-
discipline and related subject areas: vances, is the department finding ways
to extend the teaching and learning of
• Do the department’s introductory STEM beyond traditional classroom and
and advanced courses and other educa- laboratory settings?

• Does the department encourage assessing the role and nature of teach-
faculty members to integrate the cur- ing laboratories in the department’s
riculum of lower and upper division curriculum? For example, is there
courses? general agreement on whether labora-
tory exercises should parallel
Providing academic advising and coursework, provide students with
career planning: learning experiences not directly related
to work in classrooms, or some combi-
• Does the department view aca- nation of the two?
demic and career advising as central to • Does the department encourage
its mission? faculty to develop inquiry-based labora-
• Does the department encourage tory exercises that encourage students
faculty members to become more to develop their own hypotheses, design
effective academic and career advisors original experiments, and analyze data?
and provide the necessary resources • Have members of the department
and time for the purpose? discussed the criteria for assessing
• Does the department encourage students’ work in laboratories?
undergraduate students to undertake • Is the department familiar with the
real-world work and academic experi- use of virtual laboratories and the
ences through summer and academic- current status of research comparing
year internships? real and simulated approaches to
• Does the department bring people laboratory teaching and learning?
to campus for presentations to students
about career options and opportunities? Encouraging students to engage in
independent research:
EVALUATING DEPARTMENTAL • Does the department encourage

EFFORTS TO IMPROVE TEACHING faculty to oversee and support students
LABORATORIES AND OTHER who wish to engage in independent,
UNDERGRADUATE RESEARCH original research either on campus or
EXPERIENCES off site (e.g., cooperative arrangements
with other universities, private and
Emphasizing the role and impor- government research establishments, or
tance of teaching laboratories: industry)? Does the department take
advantage of undergraduate research as
• Have members of the department a way for its graduate students to grow
collectively established criteria for

professionally by helping to supervise • Is the department’s curriculum

such work? structured in ways that offer gateways
• Are venues available for providing for students from other departments,
academic credit or financial compensa- including those who will not major in
tion to students and teaching credits, the sciences, to continue studies within
time, equipment, and rewards to faculty the discipline?
who undertake such supervisory • Has the department worked with
responsibilities? other STEM departments to discuss
• Has the department discussed ways in which the presentation of topics
what the role of undergraduate research common to courses in several disci-
should be in relation to advancing its plines (e.g., energy) might be better
mission of teaching, research, and coordinated and the connections be-
service? tween disciplines emphasized (see NRC,
• Has the department considered 1999a, p. 36)?
how it might offer opportunities to • Has the department worked
engage in short- or long-term research recently with other STEM departments
experiences to both current and pro- and the institution’s college of education
spective teachers (especially those who to improve the preparation and continu-
will teach in the primary grades) and ing professional development of K–12
students who will not major in STEM? teachers in STEM (especially those
students who plan to teach in the
primary and middle grades)?
EVALUATING • Given the recent national emphasis
INTERDEPARTMENTAL on partnerships between higher educa-
COOPERATION IN IMPROVING tion and local schools, has the depart-
UNDERGRADUATE SCIENCE, ment discussed with other STEM
TECHNOLOGY, ENGINEERING, departments and local schools ways to
AND MATHEMATICS EDUCATION establish such partnership programs
and to recognize and reward faculty
• Has the department established colleagues who undertake such efforts?
dialogues with other departments about • Has the department worked with
the suitability and usefulness of its counterparts in local community col-
introductory courses as prerequisite or leges and 4-year institutions to establish
corequisite requirements for other policies and agreements that allow
STEM disciplines? students to move more seamlessly
between institutions?

Recommendations
In formulating its recommendations,

A major transformation is coming in the
the committee was struck by the diver-
sity of educational institutions in the American professoriate, if for no other
United States, the types and numbers of reason than we are on the verge of a
students they serve, and the educational
generational changing of the guard. Our
traditions they represent. Two-year
senior faculty, appointed in large numbers
community colleges, small liberal arts
colleges, and public and private re- during higher education’s expansionist
search universities offer different period circa 1957–1974, have begun to
educational experiences and represent
make choices about their retirement and
different scholarly environments.
later-life careers. And the next generation
Average class size, age or preparation of
students, the frequently conflicting of faculty is already beginning to succeed
demands of teaching and research, and them…. Leaders among the faculty and
the degree of collective (as opposed to
administration now in our colleges and
individual) faculty commitment to
universities have a time-limited window of
teaching can vary greatly among institu-
tions. A problem in one setting may not opportunity to influence this transformation,
be an issue in another. and in so doing to contribute to setting
The recommendations presented
future course of higher learning.
below are based on the four fundamen-
SOURCE: Rice et al. (2000, p. 1).
tal premises stated in Chapter 1 (and
reiterated in Box 8-1). All of them are
115

Box 8-1. Four Fundamen- To be useful, teaching evaluation has to

tal Premises become a feedback process. Given the
• Effective postsecondary aging of faculty and the public demand for

teaching in science, technology, better teaching, we need to consider ways
engineering, and mathematics
(STEM) should be available to all of using comprehensive evaluation systems
students, regardless of their major. to provide faculty with feedback or informa-
• The design of curricula and the
evaluation of teaching and learning tion about their performance that includes
should be collective responsibilities recommendations for future improvement.
of faculty in individual departments
or, where appropriate, performed
through other interdepartmental SOURCE: Ory (2000, p. 13).
arrangements.
• Scholarly activities that focus
on improving teaching and learning
should be recognized as bona fide implications for how teaching is hon-
endeavors that are equivalent to ored and supported by educational
other scholarly pursuits. Scholarship institutions.
devoted to improving teaching The kind of evaluation being recom-
effectiveness and learning should be
accorded the same administrative mended here requires the collection of
and collegial support that is avail- different kinds of evidence that can be
able for efforts to improve other used to determine whether faculty and
research and service endeavors. departments are indeed promoting
• Faculty who are expected to
student learning. Thus, if tenure review
work with undergraduates should be
given support and mentoring in committees of senior faculty rely exclu-
teaching throughout their careers; sively on outside letters evaluating
hiring practices should provide a first research and teaching accomplish-
opportunity to signal institutions’
ments, if they have had no personal
teaching values and expectations of
faculty. involvement with methods of evaluating
teaching or understanding how students
learn, and if their teaching experience
based on the premise that evidence of has been bounded only by the lecture
student learning should be an important hall, the messages they send about the
criterion for evaluating teaching. In importance of formative evaluation of
turn, evaluation of teaching that is teaching will be crystal clear and will not
predicated on learning outcomes has contribute to more effective teaching.

Another major difficulty in imple- instructional plans that embrace the

menting more effective evaluation of needs of both majors and nonmajors.
teaching is that these activities take time Departments that take the time to
from other commitments. Unless there examine their educational goals and the
are incentives to undertake evaluation effectiveness of their curricula should
of teaching in ways that truly enhance be more deserving of institutional
student learning, little change is likely support than those that maintain the
to occur. As discussed in Chapter 4, status quo. But there is a broader
Root (1987) has shown that, with appro- purpose to this exercise than depart-
priate training and motivation, improved mental self-interest in slot allocation.
evaluation can be accomplished with a Faculties that explore their goals for
manageable investment of time by the student learning and reach consensus
instructor’s colleagues. on how best to accomplish those goals
In addition to the priorities they set will have created the basis for a culture
through their leadership, deans and and community of teaching and learning
presidents have some budgetary author- that now characterizes the research
ity that can be used to improve teaching domain.
and its evaluation. For example, the Investments of time and funds un-
number of faculty teaching positions in a doubtedly will be required initially for
department is generally influenced by such efforts. The costs of these invest-
several factors: numbers of students ments will vary greatly, depending on
taught, institutional budgetary con- the kinds and levels of commitment and
straints, and decisions to have faculty in resources a department or institution
particular areas of expertise. In re- has already expended to improve its
search universities and growing num- system of evaluating teaching effective-
bers of other types of institutions, the ness. For all of the reasons highlighted
last criterion can be the most important, in this report, however, the committee is
driven by external forces at work in the convinced that such investment is
discipline as perceived and advocated by essential to improving teaching, learn-
the resident faculty. Sometimes depart- ing, and curriculum, and will provide
ments are asked to develop plans for ample rewards through improved
new appointments that are based on efficacy of teaching and student learn-
these disciplinary issues. Less often, ing.
departments are also challenged by Faculty acceptance and ownership of
administrations to prepare overarching any process for evaluating teaching
R E C O M M E N D AT I O N S 117

effectiveness is central to the success of 1. Overall Recommendations

that process on any campus. This
report focuses on helping faculty under- (1.1) Teaching effectiveness
stand the roles that various kinds of should be judged by the quality and
teaching evaluation can play in making extent of student learning. Many
them more effective instructors. Chap- different teaching styles and meth-
ters 5 and 6 provide specific advice and ods are likely to be effective.
recommendations for improving teach-
ing through appropriate evaluation Although many factors are involved in
procedures. Chapter 6 also provides judging effective teaching, evidence of
specific cross-references to other parts student learning should be foremost
of the report where faculty can find among them. Reaching institution-wide
discussion of evidence that supports the consensus on this principle is a critical
efficacy of various approaches to collect- step that will require consideration of
ing and analyzing data used for forma- such questions as what different kinds
tive and summative evaluation of teach- of students (STEM majors,
ing. Therefore, this chapter does not preprofessionals, and nonmajors)
contain a separate set of recommenda- should be learning in each discipline
tions for faculty. Instead, the recom- and how that learning can best be
mendations listed below are directed fostered. Definitions of effective teach-
primarily to policy makers, administra- ing in STEM courses in the institution
tors, and leaders of organizations should take into account what is known
associated with higher education. It is about student learning and academic
they who must become deeply involved achievement (e.g., Coppola and Jacobs,
with promoting and supporting the 2002; Huber and Morreale; 2002).
kinds of evaluations of teaching that can
lead to improved student learning. (1.2) Scholarly activities that
They also must establish opportunities focus on improving teaching and
for faculty to engage in ongoing profes- learning should be recognized and
sional and leadership development rewarded as a bona fide scholarly
directed at the improvement of teaching endeavor and accorded the types of
and learning as a scholarly endeavor, institutional supports aimed at
and reward them for doing so in ways improving scholarship generally.
that are commensurate with those
associated with other forms of scholar- Scholarship that is devoted to improv-
ship. ing teaching effectiveness and learning

should be accorded the same adminis- designed to provide fair and objec-
trative and collegial support that is tive information to aid faculty in the
available for other research and service improvement of their teaching.
endeavors. Faculty who wish to pursue Building consensus among faculty,
scholarly work by improving teaching or providing necessary resources, and
engaging in educational research relying on the best available re-
should be expected to conform to search on teaching, learning, and
standards of quality similar to those for measurement are critical for this
other types of scholarship (see Box 5-1 approach to evaluation.
in Chapter 5). They also should be
rewarded in ways that are comparable As discussed in Chapters 4 and 5,
to those associated with other forms of teaching portfolios, including a careful
scholarship during personnel decisions self-evaluation by the person being
on such matters as tenure, promotion, evaluated, can be an important tool for
and merit increases in salary. documenting a faculty member’s accom-
plishments in facilitating student learn-
(1.3) Valid summative assessing and academic achievement. Such
ments of teaching should not rely portfolios can be used for performing
only on student evaluations, but summative evaluation, but equally
should include peer reviews and important, for maintaining a record of
teaching portfolios used for promo- personal accomplishments and teaching
tion, tenure, and post-tenure re- issues that can serve as the basis for
view.1 Such assessments should be ongoing professional development.
Regardless of whether formalized
1
teaching portfolios are required for
Other organizations, such as the American
Association for Higher Education (AAHE), are evaluation of teaching, faculty should
currently engaged in efforts to explore issues collect a broad array of evidence of
associated with post-tenure review of faculty,
including the effectiveness of their teaching. teaching effectiveness that can be used
Therefore, the committee did not consider this for both formative and summative
issue in detail and offers no specific recommen-
dations about policies for post-tenure review of
evaluations. This evidence could
faculty. Additional information about the include, but not be limited to, the
program at AAHE and its recent publications on
following:
this issue (e.g., Licata and Morreale, 1997, 2002)
is available at <http://www.aahe.org/Bulletin/
aprilf1.htm>. Links to numerous other resources • Covering content at a level appro-
and policy statements on post-tenure review at
individual colleges and universities are available priate to course goals (particularly for a
at <http://www.google.com/search?hl=en&ie= course in a vertical sequence).
UTF-8&oe=UTF-8&q=post-tenure+review>.

• Promoting classroom continuity by multidisciplinary offerings for

minimizing absences and providing nonmajors.
high-quality substitutes when an ab- • Participating in effective mentoring
sence is necessary. and evaluation of departmental and
• Providing time for consultation other colleagues.
with students for informal and friendly • Supporting other colleagues’
advising. efforts to improve their teaching.
• Being open to critiques of one’s
own teaching. In addition, STEM departments could
• Actively fostering infrastructure consider hiring faculty who have spe-
improvements that enhance under- cific expertise in learning within their
graduate learning. disciplines. These hires would know
• Participating in departmental, the salient resources and could share
college-level, or university-wide discus- them with their departmental col-
sions of curriculum and improvement of leagues. Departmental cultures should
teaching and learning. encourage and provide venues for such
• Supervising undergraduate re- sharing of expertise in learning, thereby
search and encouraging active participa- fostering new teaching and learning
tion of undergraduates as contributing communities (Coppola and Jacobs, 2002;
coauthors of published works. Shapiro and Levine, 1999). Faculty also
• Being willing to promote participa- might be evaluated for the extent to
tion of undergraduates in professional which they help their department
meetings. integrate curriculum and attend more
• Exposing undergraduates to closely to the academic needs of a
professional settings (e.g., industry broader array of students.
internships, government laboratories, or
study abroad). (1.4) Individual faculty—begin-
• Being aware of and adopting ners as well as more experienced
innovative pedagogical approaches, teachers—and their departments
including thoughtful teaching and should be rewarded for consistent
assessment methods. improvement of learning by both
• Participating in the design of major and nonmajor students. All
valuable laboratory experiences. teaching-related activities—such as
• Helping to develop innovative grading, reporting of grades, cur-
designs for upper division and honors riculum development, training of
courses and for lower division teaching assistants, and related

committee work—should be in- At the same time, institutions

cluded in evaluation systems should recognize that disciplines
adopted for faculty rewards. approach teaching differently and
that such differences should be
Departments should encourage their reflected in evaluation procedures.
faculty to improve teaching and learning
through active participation in ongoing Activities that demonstrate a faculty
professional development programs. member’s commitment to improving
Departmental faculty need to initiate teaching skills might include participat-
and coordinate changes in departmental ing in programs at the institution’s
curricula. Chief academic and executive teaching and learning center. They
officers should extend significant might also include organizing or partici-
honors to both faculty members and pating in departmental or all-campus
departments that demonstrate excel- presentations or seminars on teaching
lence in the practice and scholarship of and learning, or engaging in formative
teaching, as defined by members of the evaluations of colleagues. These efforts
campus teaching and learning commu- both by individual faculty and academic
nity and the criteria presented in this departments to improve teaching and
report (e.g., Svinicki and Menges, learning should be publicly rewarded.
1996). Moreover, dissemination of information
on campus-wide successes in evaluating
(1.5) Faculty should accept the teaching and learning can inform
obligation to improve their teaching evaluation practices in other disciplines.
skills as part of their personal When evaluating teaching, it is critical
commitment to professional excel- to recognize the different emphases and
lence. Departments and institu- approaches among disciplines. For
tions of higher education should example, departments that stress
reinforce the importance of such laboratory-based teaching and learning
professional development for faculty as integral components of their curricu-
through the establishment and lum will have different approaches to
support of campus resources (e.g., teaching than departments in which
centers for teaching and learning) laboratory and field work are not typi-
and through personnel policies that cally part of the curriculum.
recognize and reward such efforts.

2. Recommendations for Candidates for faculty positions who

Presidents, Overseeing Boards, are expected to teach undergraduates
and Academic Officers should demonstrate knowledge of and
Scientists, mathematicians, and enthusiasm for teaching. Position
engineers who are accomplished in announcements and the interview
research often enjoy national and even process should make explicit the
international reputations, whereas those institution’s emphasis on and expecta-
who excel in teaching are rarely known tion for high-quality teaching (e.g., by
beyond the boundaries of their own expecting candidates to teach a class or
campuses. If institutions are to make a to discuss their approaches to teaching
concerted effort to enhance the impor- and improving student learning). In
tance of undergraduate teaching and addition, all instructors, including senior
student learning within and across these faculty, should be given opportunities
disciplines, they will need to find ways for ongoing professional development in
of recognizing and enhancing the status teaching and recognized and rewarded
of faculty who make distinctive contribu- for taking advantage of those opportuni-
tions to this critical endeavor. Faculty ties. Support also should be provided
cannot be fully successful if they alone for long-term, ongoing research projects
undertake the measures required to that enable effective teaching and
improve teaching and learning; faculty, learning practices on campus to be
as well as departments, need direct analyzed and applied to additional
encouragement and support from the courses and programs.
highest levels of leadership on campus.
(2.2) Campus-wide or disciplin-
(2.1) Quality teaching and effec- ary-focused centers for teaching and
tive learning should be highly learning should be tasked with
ranked institutional priorities. All providing faculty with opportunities
faculty and departmental evalua- for ongoing professional develop-
tions and accreditation reviews ment that include understanding
should include rigorous assessment how people learn, how to improve
of teaching effectiveness. University current instruction though student
leaders should clearly assert high feedback (formative evaluation),
expectations for quality teaching to and how educational research can
newly hired and current faculty. be translated into improved teach-
ing practice. Such centers should

provide equipment and facilities for the Advancement of Teaching2 ). He

required for innovative teaching. or she would report directly to the
provost or, where appropriate, to the
Centers for excellence in teaching president or chancellor.
and learning should be provided with
sufficient resources to enable them to (2.4) Faculty who have excelled
work with a broad array of departments in teaching should be publicly
and individual faculty. The centers’ recognized and rewarded. Endow-
assistance might include giving faculty ments should be established to
and administrators access to new recognize the serious contributions
information about advances in the of faculty who have made a sus-
cognitive sciences. Centers might tained contribution to quality teach-
direct faculty to ongoing research and ing.
innovative practices and offer specific
guidance for improving teaching and Such recognition might include
student learning. They also might be permanent increases in salary, promo-
charged specifically with helping faculty tions, and monetary awards in amounts
use formative evaluation to assess the comparable to those given to faculty
effectiveness of teaching and learning. being recognized for other kinds of
scholarly accomplishments. Monetary
(2.3) At least one senior univer- awards might allow recipients to pur-
sity-level administrator should be chase teaching equipment to support
assigned responsibility for encour- their teaching efforts or hire student
aging departmental faculty to adopt workers or others to assist with the
effective means (as proven by development of new laboratory or field
research) to improve instruction. exercises. Recipients might also use
such awards to attend professional
This individual would oversee and conferences or visit with colleagues on
coordinate efforts on campus to estab- other campuses to share information
lish and sustain the kinds of teaching and ideas for improving teaching and
and learning communities described in
this report and elsewhere (e.g., Shapiro
and Levine, 1999; the Campus Program
initiative established by the American
2
Association for Higher Education Additional information is available at <http://
www.aahe.org/teaching/Teaching_Initiative_
[AAHE] and the Carnegie Foundation Home.htm>.

learning. In addition, funds could be advising them about their duties to

made available to establish on campus a undergraduate students.3
series of convocations at which
awardees would be invited to speak 3. Recommendations for Deans,
about their teaching and approaches to Department Chairs, and
enhancing student learning. Excellence Peer Evaluators
in education would be determined (3.1) Departments should peri-
through a comprehensive evaluation of odically review a departmental
faculty members’ teaching based on the mission statement that includes
kinds of evidence described in Chapter appropriate emphasis on teaching
4 of this report. and student learning. These re-
views should address not only the
(2.5) Faculty should be encour- major curriculum, but also service
aged to develop curricula that offerings, such as courses designed
transcend disciplinar y boundaries, for nonmajors and prospective
through a combination of incentives teachers.
(including funding), expectations of
accountability, and development of (3.2) Individual faculty members
standards for disciplinar y and should be expected to contribute to
interdisciplinar y teaching.
(2.6) Willingness to emphasize 3

For additional information and strategies for
student learning and to make allo- implementing this recommendation, see National
cations of departmental resources Research Council (NRC), 1999a, pp.53–59. Other
major initiatives to expose graduate students to
in support of teaching should be an the challenges they will face as faculty members
essential requirement in appointing include Preparing Future Faculty, a joint effort by
the Association of American Colleges and
deans, department chairs, and Universities, the Council of Graduate Schools,
similar administrative positions. and the National Science Foundation. Additional
information about Preparing Future Faculty is
available at <http://www.preparing-faculty.org/>.
(2.7) Graduate school deans Tomorrow’s Professor Listserv, a website
maintained by the Stanford University Learning
should require that departments Laboratory, provides continuing updates and new
that employ graduate students in insights to future and recently hired faculty
members following the publication of Tomorrow’s
fulfilling their teaching mission
Professor: Preparing for Academic Careers in
should show evidence that their Science and Engineering (Reis, 1997). The list
faculties are effectively mentoring serv/website is available at <http://sll.stanford.
edu/projects/tomprof/newtomprof/
graduate teaching assistants and index.shtml>.

a balanced program of undergradu- member’s commitment to quality

ate teaching. Participation of estab- teaching. Generally, this should
lished faculty in lower division, include evaluation of a faculty
introductory, and general education member’s knowledge and enthusi-
courses should be encouraged. asm for the subject matter; familiar-
Faculty who are most familiar with ity with a range of appropriate
new developments in the discipline pedagogical methods; skills in using
can provide leadership in depart- appropriate tests and laboratory
mental curricular review and revi- experiences; quality of advising and
sion. Not all faculty must contrib- other professional interactions with
ute equally to instruction at ever y students within and beyond the
level, but it is a departmental re- classroom; and active scholarly
sponsibility to ensure that the commitment to enhancing top-
instructional needs of all students quality teaching and learning.
are met by caring, responsible
faculty. (3.5) Department heads, in
submitting personnel recommenda-
(3.3) Departments should con- tions, should provide separate
tribute to campus-wide awareness of ratings on teaching, research, and
the premium placed on improved ser vice, each with supporting evi-
teaching. They should build con- dence, as key components of their
sensus among their own faculty on overall rating and recommendation.
the suitability of the institution’s
procedures for summative evalua- (3.6) Normal departmental
tion of teaching, recognizing that the professional development activity
way practitioners of a specific disci- should include informing faculty
pline approach learning will affect about research findings that can
the ways that teaching should be improve student learning.
evaluated.
(3.7) As appropriate for achiev-
(3.4) In addition to numerical ing departmental goals, depart-
data from end-of-course student ments should provide funds to
evaluations and on participation in faculty to enhance teaching skills
specific courses, effective peer and knowledge and encourage them
reviews of teaching should provide a to undertake or rely upon educa-
subjective assessment of a faculty tional research that links teaching

strategies causally to student learn- engagement with innovative teach-

ing. Additional funds should be ing is being addressed.
made available to departments that
adopt this strategy. Such work may be particularly impor-
tant in teaching undergraduates. Thus,
Faculty should be able to apply for the institution should support faculty
these funds to participate in education who wish to change the focus of their
workshops, to present papers on their career (e.g., Huber, 2001). However,
teaching at professional meetings, or to institutions should also expect these
work with consultants or colleagues on faculty to provide evidence of new or
improving teaching and student learn- continued scholarly productivity and
ing. When a university has provided improvements in teaching (in accor-
such support, evaluations of teaching dance, for example, with the standards
should include evidence that the knowl- listed in Box 5-1 in Chapter 5). Such
edge and innovations gained from such evidence should be evaluated using
activities have been incorporated in protocols similar to those for other
some way into the faculty member’s types of scholarship.
teaching. How well departments meet
or exceed these goals might be gauged 4. Recommendations for
using the evidence and evaluation Granting and Accrediting
instruments described in Chapter 7 of Agencies, Research Sponsors,
this report. Members of departments and Professional Societies
should be free to use the additional (4.1) Funding agencies should
funds as they deem appropriate. De- support programs to enable an
partments awarded such merit funds integrated network of national and
should be publicly recognized. campus-based centers for teaching
and learning. An important goal of
(3.8) Departments should recog- such a network is to conduct and
nize that in the course of their disseminate research on ap-
careers, some faculty may shift the proaches that enhance teaching and
balance of their departmental obli- learning in STEM. The network can
gations to place a greater emphasis also provide information on the use
on instruction or educational lead- of formative and summative assess-
ership. These shifts should be ment for improving teaching and
supported, consistent with a depart- learning. To the extent possible,
mental mission, so long as active these investments should not be

made at the expense of sponsored (4.3) Accreditation agencies and

research. boards should revise policies to
emphasize quality undergraduate
These centers would focus on higher learning as a primar y criterion for
education issues in STEM, and espe- program accreditation.
cially on research on how college-level
students learn these subjects most (4.4) Professional societies
effectively. Teaching and learning should offer opportunities to dis-
centers also might be supported in their cuss undergraduate education
efforts to disseminate resources beyond issues during annual and regional
their campuses, particularly through meetings. These events might
electronic means.4 include sessions on teaching tech-
niques and suggestions for over-
(4.2) Funding agencies and coming disciplinar y and institu-
research sponsors should under- tional barriers to improved
take self-examination by convening teaching.
expert panels to examine whether
agency policies might inadvertently (4.5) Professional societies
compromise a faculty member’s should encourage publication of
commitment to quality undergradu- peer-reviewed articles in their
ate teaching. general or specialized journals on
evolving educational issues in
STEM.
4
The National Science Foundation recently
initiated a program that addresses this recom-
mendation. Its Centers for Learning and
Teaching program is designed to “…provide a
rich environment that melds research, teacher
professional development, and education
practice.” Additional information about this
initiative is available at <http://www.nsf.gov/cgi-
bin/getpub?nsf00148>.

References
Abd-El-Khalick, F., Bell, R.L., and Lederman, A menu for peer collaboration and peer review.
N.G. (1998). The nature of science and Washington, DC: Author.
instructional practice: Making the unnatural American Association for Higher Education.
natural. Science Education 82(4): 417–437. (1995). From idea to prototype: The peer review
Abrami, P.C., Leventhal, L., and Perry, R.P. of teaching—A project workbook. Washington,
(1982). Educational seduction. Review of DC: Author.
Educational Research 52, 446–464. American Association for Higher Education.
Allen, D.E., and Duch, B. (1998). Thinking (1996). Teaching, learning and technology
towards solutions: Problem-based learning (TLT) roundtable workbook. Washington, DC:
activities for general biology. Philadelphia, PA: Author.
Saunders College. American Educational Research Association,
Allen, D., Groh, S.E., and Allen, D.E. (Eds.). American Psychological Association, and
(2001). The power of problem-based learning: A National Council on Measurement in Educa-
practical “how-to” for teaching undergraduate tion. (1999). Standards for educational and
courses in any discipline. Sterling, VA: Stylus. psychological measurement. Washington, DC:
Alverno College Faculty. (1994). Student American Educational Research Association.
assessment-as-learning at Alverno College (3rd American Psychological Association. (2002).
Ed.). Milwaukee, WI: Alverno College. National guidelines and suggested learning
Ambady, N., and Rosenthal, R. (1993). Half a outcomes for the undergraduate psychology
minute: Predicting teacher evaluations from major (draft). Washington, DC: Author.
thin slices of nonverbal behavior and physical Anderson, E., (Ed.). (1993). Campus use of the
attractiveness. Journal of Personality and teaching portfolio: Twenty-five profiles.
Social Psychology 64(3), 431–441. Washington, DC: American Association for
American Association for the Advancement of Higher Education.
Science. (1989). Science for all Americans. Anderson, L.W., Krathwohl, D.R., and Bloom,
Washington, DC: Author. B.S. (Eds.). (2001). A taxonomy for learning,
American Association for the Advancement of teaching, and assessing: A revision of Bloom’s
Science. (1990). The liberal art of science. taxonomy of educational objectives, 2001.
Washington, DC: Author. Boston: Allyn and Bacon.
American Association for the Advancement of Angelo, T.A. (1995, November). Assessing (and
Science. (1993). Benchmarks for science defining) assessment. AAHE Bulletin 48(2).
literacy. New York: Oxford University Press. Angelo, T.A. (1999). The campus as a learning
American Association for Higher Education. community: Seven promising shifts and seven
(1993). Making teaching community property: powerful levers. In B.A. Pescosolido, and R.
Aminzade (Eds), The social worlds of higher
128

education: Handbook for teaching in a new Bernstein, D.J., and Quinlan, K.M. (Eds.). (1996,
century (pp. 110–116). Thousand Oaks, CA: Spring). The peer review of teaching [Special
Pine Forge Press. issue]. Innovative Higher Education 20(4).
Angelo, T.A., and Cross, K.P. (1993). Classroom Bleak, J., Neiman, H., Sternman, C., and Trower,
assessment techniques: A handbook for college C. (2000). Faculty recruitment study: Statistical
teachers (2nd ed.). San Francisco: Jossey- analysis report, August 2000. Cambridge, MA:
Bass. Harvard Graduate School of Education.
Armstrong, L. (2000). Distance learning: An Bloom, B.S. (Ed.). (1956). Taxonomy of educa-
academic leader’s perspective on a disruptive tional objectives: The classification of educa-
product. Change 32(6), 20–27. tional goals. Handbook I: Cognitive domain.
Astin, A.W., Banta, T.W., Cross, K.P., El-Khawas, New York: Longmans, Green.
E., Ewell, P.T., Hutchings, P., Marchese, T.J., Boice, R. (1992). The new faculty member. San
McClenney, M., Mentkowski, M., Miller, Francisco: Jossey-Bass.
M.A., Moran, E.T., and Wright, B.D. (1996). Borgman, C.L., Bates, M.J., Cloonan, M.V.,
Assessment forum: 9 principles of good practice Efthimiadis, E.N., Gilliland-Swetland, A.J.,
for assessing student learning. Washington, Kafai, Y.B., Leazer, G.H., and Maddox, A.B.
DC: American Association for Higher (1996). Social aspects of digital libraries. In
Education. See <http://www.aahe.org/ Proceedings of the University of California Los
principl.htm>. Angeles-National Science Foundation Social
Baker, P. (1999). Creating learning communities: Aspects of Digital Libraries Workshop, Febru-
The unfinished agenda. In B.A. Pescosolido, ary 15–17, 1996. See <http://
and R. Aminzade (Eds), The social worlds of is.geis.ucla.edu/research/dl/index.html>.
higher education: Handbook for teaching in a Boyer, E.L. (1990). Scholarship reconsidered:
new century (pp. 95–109). Thousand Oaks, Priorities of the professoriate. Princeton, NJ:
CA: Pine Forge Press. Carnegie Foundation for the Advancement of
Banta, T.W. (2000). That second look. Assessment Teaching.
Update 10(4), 3–14. Boyer Commission on Educating Undergradu-
Banta, T.W., Lund, J.P., Black, K.E., and ates in the Research University. (1998).
Oblander, F. W. (1996). Assessment in practice: Reinventing undergraduate education: A
Putting principles to work on college campuses. blueprint for America’s research universities.
San Francisco: Jossey-Bass. Menlo Park, CA: Carnegie Foundation for the
Barr, R.B., and Tagg, J. (1999). From teaching to Advancement of Teaching. See <http://
learning: A new paradigm for undergraduate notes.cc.sunysb.edu/Pres/boyer.nsf>.
education. In B.A. Pescosolido, and R. Brand, M. (2000). Changing faculty roles in
Aminzade (Eds.), The social worlds of higher research universities: Using the pathways
education: Handbook for teaching in a new strategy. Change 32(6), 42–45.
century (pp. 565–581). Thousand Oaks, CA: Braskamp, L., and Ory, J. (1994). Assessing
Pine Forge Press. faculty work: Enhancing individual and
Basinger, J. (2000, August 7). Carnegie issues institutional performance. San Francisco:
broad changes in system of classifying Jossey-Bass.
colleges. Chronicle of Higher Education. See Brinko, K.T. (1993). The practice of giving
<http://chronicle.com/daily/2000/08/ feedback to improve teaching: What is
2000080701n.htm>. effective? Journal of Higher Education 64(5),
Baugher, K. (1992). LEARN: The student quality 574–593.
team manual. Nashville, TN: LEARN. Brookfield, S.D. (1995). Becoming a critically
Bernstein, D.J. (1996, Spring). A departmental reflective teacher. San Francisco: Jossey-Bass.
system for balancing the development and Buck, G.A., Hehn, J.G., and Leslie-Pelecky, D.L.
evaluation of college teaching: A commentary (Eds.). (2000). The role of physics departments
on Cavanaugh. Innovative Higher Education in preparing k–12 teachers. College Park, MD:
20(4), 241–248. American Institute of Physics.
REFERENCES 129

Cambridge, B.L. (1996, Spring). The paradigm Kansas State University, Center for Faculty
shifts: Examining quality of teaching through Evaluation and Development.
assessment of student learning. Innovative Centra, J.A. (1993). Reflective faculty evaluation:
Higher Education 20(4), 287–298. Enhancing teaching and determining faculty
Cambridge, B.L. (Ed.). (1997). Assessing impact: effectiveness. San Francisco: Jossey-Bass.
Evidence and action. Washington, DC: Centra, J.A. (1994). The use of the teaching
American Association of Higher Education. portfolio and student evaluations for
Cambridge, B.L. (1999, December). The summative evaluation. Journal of Higher
scholarship of teaching and learning: Ques- Education 65(5), 555–570.
tions and answers from the field. AAHE Centra, J.A. (1998). The development of the student
Bulletin. See <http://www.aahe.org/Bulletin/ instructional report II, Higher Education
dec99f2.htm>. Assessment Program. Princeton, NJ: Educa-
Cambridge, B.L. (Ed.). (2001). Electronic tional Testing Service.
portfolios: Emerging practices in student, Centra, J.A. (2001). A model for assessing the
faculty, and institutional learning. Washington, scholarship of teaching. Presentation at the 9th
DC: American Association for Higher Annual American Association of Higher
Education. Education Conference on Faculty Roles and
Capelli, P. (Ed.). (1997). Change at work: Trends Rewards, February, Tampa, FL.
that are transforming the business of business. Centra, J.A., and Creech, F.R. (1976). The
Washington, DC: National Policy Association. relationship between student, teacher, and
Carnegie Foundation for the Advancement of course characteristics and student ratings of
Teaching and Pew Forum on Undergraduate teacher effectiveness. (Report No. PR-76-1).
Learning. (2002). National survey of student Princeton, NJ: Educational Testing Service.
engagement: The college student report. Menlo Centra, J.A., and Gaubatz, N.B. (2000a). Is there
Park, CA: Author. See <http:// gender bias in student evaluations of teach-
www.indiana.edu/~nsse/l>. ing? Journal of Higher Education 70(1), 17–33.
Cashin, W.E. (1990). Students do rate different Centra, J.A., and Gaubatz, N.B. (2000b). Student
academic fields differently. In M. Theall, and perceptions of learning and instructional
J. Franklin (Eds.), Student ratings of instruc- effectiveness in college courses, Higher Educa-
tion: Issues for improving practice. San tion Assessment Program. (Report No. 9).
Francisco: Jossey-Bass. Princeton, NJ: Educational Testing Service.
Cashin, W.E., and Downey, R.G. (1999). Using Chandler, S. (1991). Issues in evaluating multi-
global student rating items for summative institutional models. Paper presented at the
evaluation: Convergence with an overall American Educational Research Association
instructor evaluation criteria. Paper presented Symposium, Chicago.
at American Educational Research Associa- Chickering, A.W., and Gamson, Z.F. (1987).
tion, April, Washington, DC. Seven principles for good practice in under-
Centra, J.A. (1973). Item reliabilities, the factor graduate education. AAHE Bulletin 39, 3–7.
structure, comparison with alumni ratings. Chism, N.V.N. (1999). Peer review of teaching: A
(Student Instructional Report, No. 3). sourcebook. Boston: Anker.
Princeton, NJ: Educational Testing Service. Clark, J., and Redmond, M. (1982). Small group
Centra, J.A. (1974). The relationship between instructional diagnosis. (ERIC Ed. 217954).
student and alumni ratings of teachers. Seattle: University of Washington, Depart-
Educational and Psychological Measurement ment of Biology Education.
34(2), 321–326. Cochran, K.F. (1997). Pedagogical content
Centra, J.A. (1975). Colleagues as raters of knowledge: Teachers’ integration of subject
classroom instruction. Journal of Higher matter, pedagogy, students, and learning
Education 46, 327–337. environments. Research Matters–to the Science
Centra, J.A. (1979). Determining faculty effective- Teacher, #9702.
ness. San Francisco: Jossey-Bass. Cohen, P.A. (1981). Student ratings of instruction
Centra, J.A. (1987). Faculty evaluation: Past and student achievement: A meta-analysis of
practices, future directions. Manhattan, KS: multisection validity studies. Review of
Educational Research 51, 281–309.

Collis, B., and Moonen, J. (2001). Flexible Diamond, R.M., and Adams, B.E. (Eds.). (2000).
learning in a digital world: Experiences and The disciplines speak II: More statements on
expectations. London, UK: Kogan Page. rewarding the scholarly, professional, and
Cooper, J., and Robinson, P. (1998). Small-group creative work of faculty. Washington, DC:
instruction: An annotated bibliography of American Association for Higher Education.
science, mathematics, engineering and Diamond, R.M., and Gray, P.J. (1998). 1997
technology resources in higher education National Study of Teaching Assistants.
(Occasional Paper #6). Madison: University of Syracuse, NY: Syracuse University Center for
Wisconsin–Madison, National Institute for Instructional Development.
Science Education. Doherty, A., Riordan, T., and Roth, J. (Eds.).
Coppola, B.P., and Smith, D.H. (1996). A case for (2002). Student learning: A central focus for
ethics. Journal of Chemical Education 73, 33– institutions of higher education. (A report and
34. collection of institutional practices of the
Coppola, B.P., Ege, S.N., and Lawton, R.G. (1997). Student Learning Initiative.) Milwaukee, WI:
The University of Michigan undergraduate Alverno College Institute.
chemistry curriculum 2— Instructional Dougherty, A. (1999). Just-in-time, teaching using
strategies and assessment. Journal of Chemi- Web feedback to guide learning and teaching.
cal Education 74, 84–94. See <http://ww2.lafayette.edu/~doughera/
Coppola. B.P., and Jacobs, D. (2002). Is the talks/ets9912/>.
scholarship of teaching and learning new to Doyle, M.P. (Ed.). (2000). Academic excellence:
chemistry? In M.T. Huber, and S. Morreale The role of research in the physical sciences at
(Eds.), Disciplinary styles in the scholarship of undergraduate institutions. Tucson, AZ:
teaching and learning: Exploring common Research Corporation.
ground. Washington, DC: American Associa- Drucker, A.J., and Remmers, H.H. (1951). Do
tion for Higher Education and the Carnegie alumni and students differ in their attitudes
Foundation for the Advancement of Teaching. toward instructors? Journal of Educational
Cornell University. (1993). Teaching evaluation Psychology 42(3), 129–143.
handbook. Ithaca, NY: Cornell University, Ebert-May, D., Brewer, C.A., and Allred, S.
Office of Instructional Support. (1997). Innovation in large lectures: Teaching
Cross, K.P., and Steadman, M.H. (1996). Class- for active learning through inquiry. BioScience
room research: Implementing the scholarship of 47(9), 601–607.
teaching. San Francisco: Jossey-Bass. Edgerton, R., Hutchings, P., and Quinlan, K.
Daffinrud, S.M., and Herrera, O.L. (2000). (1991). The teaching portfolio: Capturing the
Evaluation methods and findings: The Field- scholarship in teaching. Washington, DC: The
Tested Learning Assessment Guide (FLAG) American Association for Higher Education.
final report. Madison, WI: University of Ehrmann, S.C. (2000, November/December). On
Wisconsin, LEAD Center. the necessity of grassroots evaluation of
Darling-Hammond, L. (Ed.). (1997). Doing what educational technology: Recommendations for
matters most: Investing in quality teaching. higher education. See <http://
New York: National Commission on Teaching horizon.unc.edu/TS/assessment/2000-
and America’s Future. 11.asp>.
Davis, B.G. (1988). Sourcebook for evaluating Ehrmann, S.C. (In press). Technology and
teaching. Berkeley: University of California at educational revolution: Ending the cycle of
Berkeley, Office of Educational Development. failure. Liberal Education. See draft <http://
Davis, B.G. (1993). Tools for teaching. San www.tltgroup.org/resources/
Francisco: Jossey-Bass. V_Cycle_of_Failure.html>.
Diamond, R.M., and Adams, B.E. (Eds.). (1995). Emerson, J.D., Mosteller, F., and Youtz, C.
The disciplines speak: Rewarding the scholarly, (2000). Students can help improve college
professional, and creative work of faculty. teaching: A review and an agenda for the
Washington, DC: American Association for statistics profession. In C. R. Rao, and G.
Higher Education. Szekely (Eds.), Statistics for the 21st century.
New York: Marcel Dekker.
REFERENCES 131

Emert, J. W., and Parish, C. R. (1996). Assessing Gardiner, L., Anderson, C., and Cambridge, B.
concept attainment in undergraduate core (Eds.). (1997). Learning through assessment: A
courses in mathematics. In T.W. Banta, J. P. resource guide for higher education. Washing-
Lund, K.E. Black, and F.W. Oblander (Eds.), ton, DC: American Association for Higher
Assessment in practice. San Francisco: Jossey- Education.
Bass. Gavin, R. (2000). The role of research at under-
Ewell, P.T. (1991). To capture the ineffable: New graduate institutions: Why is it necessary to
forms of assessment in higher education. defend it? In M.P. Doyle (Ed.), Academic
Review of Research in Education 17, 75–125. excellence: The role of research in the physical
Feldman, K.A. (1978). Course characteristics and sciences at undergraduate institutions (pp. 9–
college students’ ratings of their teachers and 17). Tucson, AZ: Research Corporation.
courses: What we know and what we don’t. Gilmore, G.M., Kane, M.T., and Naccarato, R.W.
Research in Higher Education 9, 199–242. (1978). The generalizability of student ratings
Feldman, K.A. (1984). Class size and college of instruction: Estimation of teacher and
students’ evaluations of teachers and courses: course components. Journal of Educational
A closer look. Research in Higher Education Measurement 15(1), 1–13.
21(1):45–116, September. Glassick, C.E., Huber, M.T., and Maeroff, G.I.
Feldman, K.A. (1989). Instructional effectiveness (1997). Scholarship assessed: Evaluation of the
of college teachers as judged by teachers professoriate. San Francisco: Jossey-Bass.
themselves, current and former students, Goheen, R.F. (1969). The human nature of a
colleagues, administrators and external university. Princeton, NJ: Princeton University
(neutral) observers. Research in Higher Press.
Education 30, 137–189. Golde, C.M., and Dore, T.M. (2001). At cross
Feldman, K.A. (1993). College students’ views of purposes: What the experiences of today’s
male and female college teachers: Part II– doctoral students reveal about doctoral
Evidence from students’ evaluations of their education. Philadelphia, PA: Pew Charitable
classroom teachers. Research in Higher Trusts.
Education 34(2), 151–211. Gray, J., Diamond, R.M., and Adam, B.E. (1996).
Fisch, L. (1998). AD REM: Spotters. The national A national study on the relative importance of
teaching and learning forum. Phoenix, AZ: research and undergraduate teaching at
Oryx Press. colleges and universities. Syracuse, NY:
Fosnot, C. (Ed.). (1996). Constructivism: Theory, Syracuse University Center for Instructional
perspectives, and practice. New York: Teachers Development.
College Press. Greene, E. (2000, June 23). Some colleges pay
Freedman, R.L.H. (1994). Open-ended question- students to go to class—to evaluate teaching.
ing. New York: Addison Wesley. The Chronicle of Higher Education, A18.
French-Lazovik, G. (1981). Documentary Greenspan, A. (2000). The economic importance
evidence in the evaluation of teaching. In J. of improving math-science education. Testi-
Millman (Ed.), Handbook of teacher evaluation mony to the U.S. House of Representatives
(pp. 73–89). Newbury Park, CA: Sage. Committee on Education and the Workforce,
Gabel, D.L. (Ed.) (1994). Handbook of research on Washington, DC. See <http://
science teaching and learning. New York: www.federalreserve.gov/boarddocs/testi-
Macmillan. mony/2000/20000921.htm>.
Gabelnick, F., MacGregor, J., Matthews, R., and Grouws, D.A. (1992). Handbook of research on
Smith, B. (1990). Learning communities: mathematics teaching and learning. New York:
Creating connections among students, faculty, Macmillan.
and disciplines. San Francisco: Jossey-Bass. Herron, J.D. (1996). The chemistry classroom:
Gaff, J.G., Pruitt-Logan, A.S., and Weibl, R.A. Formulas for successful teaching. Washington,
(2000). Building the faculty we need: Colleges DC: American Chemical Society.
and universities working together. Washington, Hestenes, D. (1987). Toward a modeling theory
DC: Association of American Colleges and of physics instruction. American Journal of
Universities. Physics 55, 440–454.

Hestenes, D., and Halloun, I. (1995). Interpreting Johnson, D.W., Johnson, R.T., and Smith, K.
the force concept inventory. Physics Teacher (1998). Active learning: Cooperation in the
33(8), 502–506. college classroom. Edina, MN: Interaction
Heterick, R., and Twigg, C. (1999). Lectures are Books.
not cheap! See <http://www.center.rpi.edu/ Joint Policy Board for Mathematics. (1994).
LForum/LM/Sept99.html>. Recognitions and rewards in the mathematical
Huber, M.T. (1999). Disciplinary styles in the sciences. Washington, DC: American Math-
scholarship of teaching: Reflections on the ematical Society.
Carnegie Academy for the Scholarship of Katz, J., and Henry, M. (1988). Turning professors
Teaching and Learning. Paper presented at into teachers: A new approach to faculty
the 7th International Improving Student development and student learning. New York:
Learning Symposium Improving Student Macmillan.
Learning Through the Disciplines, Menlo Keig, L., and Waggoner, M.D. (1994). Collabora-
Park, CA. tive peer review: The role of faculty in improv-
Huber, M.T. (2001, July/August). Balancing acts: ing college teaching. (ASHE-ERIC Higher
Designing careers around the scholarship of Education Report #2). Washington, DC:
teaching. Change, 21–29. George Washington University.
Huber, M.T., and Morreale, S. (Eds.). (2002). Kellogg Commission on the Future of State and
Disciplinary styles in the scholarship of teaching Land-Grant Universities. (1997). Returning to
and learning: Exploring common ground. our roots: The student experience. Washington,
Washington, DC: American Association for DC: National Association of State Universities
Higher Education and the Carnegie Founda- and Land-Grant Colleges.
tion for the Advancement of Teaching. Kennedy, D. (1997). Academic duty. Cambridge,
Hutchings, P. (Ed.). (1995). From idea to MA: Harvard University Press.
prototype: The peer review of teaching, a King, P.M., and Kitchener, K.S. (1994). Develop-
project workbook. Washington, DC: American ing reflective judgment: Understanding and
Association for Higher Education. promoting intellectual growth and critical
Hutchings, P. (Ed.). (1996). Making teaching thinking in adolescents and adults. San
community property: A menu for peer collabora- Francisco: Jossey-Bass.
tion and peer review. Washington, DC: Koon, J., and Murray, H.G. (1995). Using
American Association for Higher Education. multiple outcomes to validate student ratings
Hutchings, P. (Ed.). (1998). The course portfolio: of overall teacher effectiveness. Journal of
How faculty can examine their teaching to Higher Education 66(1), 61–81.
advance practice and improve student learning. Kremer, J. (1990). Constant validity of multiple
Washington, DC: American Association for measures in teaching, research, and service
Higher Education. and reliability of peer ratings. Journal of
Hutchings, P. (Ed.). (2000). Opening lines: Educational Psychology 82, 213–218.
Approaches to the scholarship of teaching and Lambert, L.M., and Tice, S.L. (Eds.). (1992).
learning. Menlo Park, CA: Carnegie Founda- Preparing graduate students to teach: A guide
tion for the Advancement of Teaching. to programs that improve undergraduate
Indiana University. (2000). The college student education and develop tomorrow’s faculty.
experiences questionnaire. See <http:// Washington, DC: American Association for
www.indiana.edu/~cseq/cseq_content.htm>. Higher Education.
International Technology Education Association. Landis, C.R., Ellis, A.B., Lisenky, G.C., Lorenz,
(2000). Standards for technological literacy: J.K., Meekder, K., and Wamser, C.C. (Eds.).
Content for the study of technology. Reston, VA: (2001). Chemistry concept tests: A pathway to
Author. See <http://www.iteawww.org/TAA/ interactive classrooms. Upper Saddle River, NJ:
STLstds.htm>. Prentice Hall.
Ireton, M.F.W., Manduco, C.A., and Mogk, D.W. Lederman, N.G., and O’Malley, M. (1990).
(1996). Shaping the future of undergraduate Students’ perceptions of tentativeness in
earth science education: Innovation and change science: Development, use, and sources of
using an earth system approach. Washington, change. Science Education 74, 225–239.
DC: American Geophysical Union.
REFERENCES 133

Lederman, N.G., Schwartz, R.S., Abd-El-Khalick, future research. International Journal of

F., and Bell, R.L. (In press). Preservice Educational Research 11, 253–388.
teachers’ understanding and teaching of the Marsh, H.W., and Roche, L.A. (1993). The use of
nature of science: an intervention study. student evaluations and an individually
Canadian Journal of Science, Mathematics, structured intervention to enhance university
and Technology Education. teaching effectiveness. American Educational
Licata, C.M., and Morreale, J.C. (1997). Post- Research Journal 30(1), 217–251.
tenure review: Policies, practices, precautions. Marsh, H.W., and Roche, L.A. (2000). Effects of
Washington, DC: American Association for grading leniency and low workload on
Higher Education. students’ evaluations of teaching: Popular
Licata, C.M, and Morreale, J.C. (2002). Post- myth, bias, validity or innocent bystander? The
tenure faculty review and renewal: Experienced Journal of Educational Psychology 92(1), 202–
voices. Washington, DC: American Association 228.
for Higher Education. Mathematical Association of America. (2000).
Liebowitz, W.R. (1999, October 22). Course Guidelines for programs and departments in
evaluations proliferate on the Web—to the undergraduate mathematical sciences.
chagrin of many professors. Chronicle of Washington, DC: Author. See <http://
Higher Education, A59. www.maa.org/guidelines/guidelines.html>.
Light, R.L. (2001). Making the most of college: Mazur, E. (1997). Peer instruction: A user’s
Students speak their minds. Cambridge, MA: manual. Upper Saddle River, NJ: Prentice
Harvard University Press. Hall.
Lloyd-Jones, R. (1977). Primary trait scoring. In McCormick, A.C. (2001). (Ed.). The Carnegie
C. Cooper, and L. Odell (Eds.), Evaluating classification of institutions of higher education,
writing: Describing, measuring, judging. 2000 edition. Menlo Park, CA: Carnegie.
Urbana, IL: National Council of Teachers of McKeachie, W.J. (1979, October). Student ratings
English. of faculty: A reprise. Academe 384–397.
Loacker, G. (Ed.). (2001). Self-assessment at McKeachie, W.J., Pintrich, P.R., Lin, Y., Smith,
Alverno College by Alverno College faculty. D.A.F., and Sharma, R. (1990). Teaching and
Milwaukee, WI: Alverno College. learning in the college classroom: A review of
Lopez, R.E., and Schultz, T. (2001). Two revolu- the research literature. Ann Arbor, MI:
tions in k–8 science education. Physics Today National Center for Research to Improve Post-
54(9), 45–49. See <http:// Secondary Teaching and Learning.
www.physicstoday.org/pt/vol-54/iss-9/ McKeachie, W.J. (1999). McKeachie’s teaching
current.html>. tips, strategies, research, and theory for college
Lovitts, B., and Nelson, C. (2000). The hidden and university teachers. (10th Ed.). New York:
crisis in graduate education: Attrition from Houghton-Mifflin.
Ph.D. programs. Academe 86(6), 44–50. McNeal, A.P., and D’Avanzo, C.D. (Eds.) (1997).
Lowman, J. (1995). Mastering the techniques of Student active science: Models of innovation in
teaching. (2nd Ed.). San Francisco: Jossey- college science teaching. Fort Worth, TX:
Bass. Harcourt Brace.
MacGregor, J., Cooper, J., Smith, K., and Menges, R., and Svinicki, M. (Eds.). (1991).
Robinson, P. (2000). Strategies for energizing College teaching: From theory to practice. San
large classes: From small groups to learning Francisco: Jossey-Bass.
communities. (No. 81). San Francisco: Jossey- Millar, S.B. (Ed.). (1998). Indicators of success in
Bass. postsecondary SMET education: Shapes of the
Marsh, H.W. (1982). Validity of students’ future. (Synthesis and Proceedings of the
evaluations of college teaching: A multitrait- Third Annual National Institute for Science
multimethod analysis. Journal of Educational Education Forum). Madison: University of
Psychology 74, 264–279. Wisconsin-Madison, Wisconsin Center for
Marsh, H.W. (1987). Student evaluations of Education Research.
university teaching: Research findings, Mintzes, J.J., Wandersee, J.H., and Novak, J.D.
methodological issues, and directions for (Eds.). (2000). Assessing science understand-

ing: A human constructivist view. San Diego, National Council of Teachers of Mathematics.
CA: Academic Press. (2000). Principles and standards for school
Mullin, R. (2001). The undergraduate revolution: mathematics. Reston, VA: Author.
Change the system or give incrementalism National Institute for Science Education. (2001a).
another 30 years? Change 33(5), 54–58. Learning through technology. Madison, WI:
Murnane, R.J., and Levy, F. (1996). Teaching the Author. See <http://www.wcer.wisc.edu/nise/
new basic skills: Principles for educating cl1/ilt/>.
children to thrive in a changing economy. New National Institute for Science Education. (2001b).
York: Free Press. Field-tested learning assessment guide.
Murray, H.G. (1983). Low inference classroom Madison, WI: Author. See <http://
teaching behaviors and student ratings of www.wcer.wisc.edu/nise/cl1/flag/>.
college teaching effectiveness. Journal of National Institute for Science Education. (2001c).
Educational Psychology 71, 856–865. Cooperative learning. See <http://
Murray, H.G., Rushton, P.J., and Paunonen, S.V. www.wcer.wisc.edu/nise/CL1/CL/>.
(1990). Teacher personality traits and student National Research Council. (1991). Moving
instructional ratings in six types of university beyond myths: Revitalizing undergraduate
courses. Journal of Educational Psychology 82, mathematics. Washington, DC: National
250–261. Academy Press. See <http://books.nap.edu/
Murray, H.G, Gillese, E., Lennon, M., Mercer, P., catalog/1782.html>.
and Robinson, M. (1996). Ethical principles in National Research Council. (1995a). Engineering
university teaching. Vancouver, BC: The education: Designing an adaptive system.
Society for Teaching and Learning in Higher Washington, DC: National Academy Press.
Education. See <http://www.umanitoba.ca/ See <http://books.nap.edu/catalog/
academic_support/uts/stlhe/Ethical.html>. 4907.html>.
Naftulin, D.H., Ware, J.E., and Donnelly, F.A. National Research Council. (1995b). Reshaping
(1973). The Doctor Fox lecture: A paradigm of the graduate education of scientists and
educational seduction. Journal of Medical engineers. Washington, DC: National Acad-
Education 48, 630–635. emy Press. See <http://books.nap.edu/
Narum, J. (1995). Structures for science: A catalog/4935.html>.
handbook on planning facilities for undergradu- National Research Council. (1996a). National
ate natural science communities (Vol. III). science education standards. Washington, DC:
Washington, DC: Project Kaleidoscope. National Academy Press. See <http://
National Academy of Sciences. (1997). Preparing books.nap.edu/catalog/4962.html>.
for the 21st century: The education imperative. National Research Council. (1996b). From
Washington, DC: National Academy Press. analysis to action. Washington, DC: National
See <http://www.nap.edu/catalog/ Academy Press. See <http://books.nap.edu/
9537.html>. catalog/9128.html>.
National Center for Education Statistics. (1999). National Research Council. (1996c). The role of
Teacher quality: A report on the preparation scientists in the professional development of
and qualification of public school teachers. science teachers. Washington, DC: National
Washington, DC: U.S. Department of Educa- Academy Press. See <http://books.nap.edu/
tion. catalog/2310.html>.
National Center for Public Policy and Higher National Research Council. (1997a). Science
Education. (2001). Measuring up 2000: The teaching reconsidered: A handbook. Washing-
state-by-state report card for higher education. ton, DC: National Academy Press. See <http:/
San Jose, CA: Author. See <http:// /books.nap.edu/catalog/5287.html>.
www.highereducation.org>. National Research Council. (1997b). Adviser,
National Council of Teachers of Mathematics. teacher, role model, friend: On being a mentor
(1989). Curriculum and evaluation standards to students in science and engineering. Wash-
for school mathematics. Reston, VA: Author. ington, DC: National Research Council. See
<http://books.nap.edu/catalog/5789.html>.
REFERENCES 135

National Research Council. (1998a). High stakes: guide for teaching and learning. Washington,
Testing for tracking, promotion, and gradua- DC: National Academy Press. See <http://
tion. Washington, DC: National Academy www.nap.edu/catalog/9596.html>.
Press. See <http://books.nap.edu/catalog/ National Research Council. (2000e). LC21: A
6336.html>. digital strategy for the Library of Congress.
National Research Council. (1998b). Developing a Washington, DC: National Academy Press.
digital national library for undergraduate See <http://books.nap.edu/catalog/
science, mathematics, engineering, and 9940.html>.
technology education: Report of a workshop. National Research Council. (2001). Knowing what
Washington, DC: National Academy Press. students know: The science and design of
See <http://books.nap.edu/catalog/ educational assessment. Washington, DC:
5952.html>. National Academy Press. See <http://
National Research Council. (1999a). Transform- www.nap.edu/catalog/10019.html>.
ing undergraduate education in science, National Research Council. (2002a). Learning
mathematics, engineering, and technology. and understanding: Improving advanced study
Washington, DC: National Academy Press. of science and mathematics in U.S. high
See <http://www.nap.edu/catalog/ schools. Washington, DC: National Academy
6453.html>. Press. See <http://www.nap.edu/catalog/
National Research Council. (1999b). Improving 10129.html.
student learning: A strategic plan for education National Research Council. (2002b). Scientific
research and its utilization. Washington, DC: research in education. Washington, DC:
National Academy Press. See <http:// National Academy Press. See <http://
books.nap.edu/catalog/6488.html>. www.nap.edu/catalog/10236.html>.
National Research Council. (1999c). Global National Science Board. (2000). Science and
perspectives for local action: Using TIMSS to engineering indicators—2000. Arlington, VA:
improve U.S. mathematics and science Author. See <http://www.nsf.gov/
education. Washington, DC: National Acad- search97cgi/vtopic>.
emy Press. See <http://books.nap.edu/ National Science Foundation. (1996). Shaping the
catalog/9723.html>. future: New expectations for undergraduate
National Research Council. (1999d). Myths and education in science, mathematics, engineering,
tradeoffs: The role of tests in undergraduate and technology. (NSF 96–139). Arlington, VA:
admissions. Washington, DC: National Author. See <http://www.nsf.gov/cgi-bin/
Academy Press. See <http://books.nap.edu/ getpub?nsf96139>.
catalog/9632.html>. National Science Foundation. (1998). Information
National Research Council. (2000a). Building a technology: Its impact on undergraduate
workforce for the information economy. education in science, mathematics, engineering,
Washington, DC: National Academy Press. and technology. (NSF 98–82). Arlington, VA:
See <http://books.nap.edu/catalog/ Author. See <http://www.nsf.gov/cgi-bin/
9830.html>. getpub?nsf9882>.
National Research Council. (2000b). Educating Neff, R.A., and Weimer, M. (Eds.). (1990).
teachers of science, mathematics, and technol- Teaching college: Collected readings for new
ogy: New practices for the new millennium. instructors. Madison, WI: Magna.
Washington, DC: National Academy Press. Novak, J. (1998). Learning, creating, and using
See <http://books.nap.edu/catalog/ knowledge: Concept maps as facilitative tools in
9832.html>. schools and corporations. Mahwah, NJ:
National Research Council. (2000c). How people Lawrence Erlbaum.
learn: Brain, mind, experience, and school: Nyquist, J.D., Abbott, R.D., Wulff, D.H., and
Expanded edition. Washington, DC: National Sprague, J. (Eds.). (1991). Preparing the
Academy Press. See <http://www.nap.edu/ professoriate of tomorrow to teach: Selected
catalog/9853.html>. readings in TA training. Dubuque, IA:
National Research Council. (2000d). Inquiry and Kendall/Hunt.
the national science education standards: A

Ory, J.C. (2000). Teaching evaluation: Past, Root, L.S. (1987). Faculty evaluation: Reliability of
present, and future. In K.E. Ryan (Ed.), peer assessments of research, teaching, and
Evaluating teaching in higher education: A service. Research in Higher Education 26, 71–
vision for the future: New directions in teaching 84.
and learning (pp. 13–18). San Francisco, CA: Rosenthal, R. (1976). Experimenter effects in
Jossey-Bass. behavioral research. New York: Appleton-
Osterlind, S.J. (1989). Constructing test items. Century-Croft.
Boston: Kluwer Academic. Rothman, F.G., and Narum, J.L. (1999). Then,
Overall, J.U., and Marsh, H.W. (1980). Students’ now, and in the next decade: A commentary on
evaluations of instruction: A longitudinal study strengthening undergraduate science, math-
of their stability. Journal of Educational ematics, engineering and technology education.
Psychology 72, 321–325. Washington, DC: Project Kaleidoscope. See
Palomba, C.A., and Banta, T.W. (1999). Assess- <http://www.pkal.org/news/
ment essentials, planning, implementing, and thennow100.html>.
improving assessment in higher education. San Rust, E. (1998). Business cares about math and
Francisco: Jossey-Bass. science achievement. In Business Coalition
Perry, R.P., and Smart, J.C. (Eds.). (1997). for Education Reform, The formula for success:
Effective teaching in higher education: A business leader’s guide to supporting math
Research and practice. New York: Agathon and science achievement (pp. 11–14). Washing-
Press. ton, DC: National Alliance for Business.
Pescosolido, B.A., and Aminzade, R. (Eds.). Sanderson, A., Phua, V.C., and Herda, D. (2000).
(1999). The social worlds of higher education: The American faculty poll. Chicago: National
Handbook for teaching in a new century. Opinion Research Center.
Thousand Oaks, CA: Pine Forge Press. Sands, R.G., Parson, L.A., and Duane, J. (1991).
Pike, G.R. (1995). The relationship between self- Faculty mentoring faculty in a public univer-
reports of college experiences and test scores. sity. Journal of Higher Education 62, 174–193.
Journal of Research in Higher Education 36(1), Schwartz, C. (1983). ABC’s of teaching with
1–21. excellence: A Berkeley compendium for teaching
Project Kaleidoscope. (1991). What works, with excellence. See <http://
building natural science communities: A plan teaching.berkeley.edu/compendium/>.
for strengthening undergraduate science and Scriven, M. (1981). Summative teacher evalua-
mathematics (Vol 1). Washington, DC: Author. tion. In J. Millman (Ed.), Handbook of teacher
Project Kaleidoscope. (1994). What works, evaluation. Beverly Hills, CA: Sage.
leadership: Challenges for the future (Vol. II). Scriven, M. (1993). Hard-won lessons in
Washington, DC: Author. program evaluation. New Directions for
Project Kaleidoscope. (1998). Shaping the future Program Evaluation 58, summer.
of undergraduate science, mathematics, Seldin, P. (1991). The teaching portfolio. Boston:
engineering and technology education: Proceed- Anker.
ings and recommendations from the PKAL day Seldin, P. (1998, March). How colleges evaluate
of dialogue. Washington, DC: Author. See teaching: 1988 vs. 1998. AAHE Bulletin 3–7.
<http://www.pkal.org/ Seymour, E. (In press). Tracking the processes
template2.cfm?2c_id=301>. of change in U.S. undergraduate education in
Reis, R.M (1997). Tomorrow’s professor: Prepar- science, mathematics, engineering, and
ing for academic careers in science and technology. Science Education.
engineering. Piscataway, NJ: IEEE Press. Seymour, E., and Hewitt, N.M. (1997). Talking
Remmers, H.H. (1934). Reliability and halo effect about leaving: Why undergraduates leave the
on high school and college students’ judg- sciences. Boulder, CO: Westview Press.
ments of their teachers. Journal of Applied Shapiro, N.S., and Levine, J.H. (1999). Creating
Psychology 18, 619–630. learning communities: A practical guide to
Rice, R.E., Sorcinelli, M.D., and Austin, A.E. winning support, organizing for change, and
(2000). Heeding new voices: Academic careers implementing programs. San Francisco, CA:
for a new generation. Washington, DC: Jossey-Bass.
American Association for Higher Education.
REFERENCES 137

Shipman, H.L. (2001). Hands-on science, 680 National Science Foundation. See <http://
hands at a time. Journal of College Science nsf.gov/cgi-bin/getpub?nsf00113>.
Teaching 30(5), 318–321. Svinicki, M., and Menges, R. (Eds.). (1996).
Shore, B.M., et al. (1986). The teaching dossier: A Honoring exemplary teaching. New directions
guide to its preparation and use. Montreal: for teaching and learning. (No. 65). San
Canadian Association of University Teachers. Francisco: Jossey-Bass.
Shulman, L. (1986). Those who understand: Uno, G.E. (1997). Handbook on teaching
Knowledge growth in teaching. Educational undergraduate science classes: A survival
Researcher 15, 4–14. manual. Norman, OK: University of Okla-
Shulman, L.S. (1993). Teaching as community homa Press.
property: Putting an end to pedagogical Walvoord, B.F., and Anderson, V.J. (1998).
solitude. Change 25(6), 6–7. Effective grading: A tool for learning and
Shulman, L. (1995). Faculty hiring: The peda- assessment. San Francisco: Jossey-Bass.
gogical colloquium—three models. AAHE Wergin, J. (1994). The collaborative department:
Bulletin 47(9), 6–9. How five campuses are inching toward cultures
Siebert, E.D., and McIntosh, W.J. (Eds.). (2001). of collective responsibility. Washington, DC:
College pathways to the science education American Association for Higher Education.
standards. Arlington, VA: National Science Wergin, J., and Swingen, J.N. (2000). Departmen-
Teachers Association. tal assessment: How some campuses are
Sorcinelli, M.D. (1999). The evaluation of effectively evaluating the collective work of
teaching: The 40-year debate about student, faculty. Washington, DC: American Associa-
colleague, and self-evaluations. In B.A. tion for Higher Education.
Pescosolido, and R. Aminzade (Eds.), The Wiggins, G. (1998). Educative assessment:
social worlds of higher education: Handbook for Designing assessments to inform and improve
teaching in a new century (pp. 195–205). student performance. San Francisco: Jossey-
Thousand Oaks, CA: Pine Forge Press. Bass.
Sorcinelli, M.D. (2000). Principles of good Wright, J.C., Millar, S.B., Kosciuk, S.A.,
practice: Supporting early career faculty. Penberthy, D.L., Williams, P.H., and Wampold,
Guidance for deans, department chairs, and B.E. (1998). A novel strategy for assessing the
other academic leaders. New Pathways II effects of curriculum reform on student
Project Forum on Faculty Roles and Rewards. competence. Journal of Chemical Education
Washington, DC: American Association for 75, 986–992.
Higher Education. See <http:// Wyckoff. S. (2001). Changing the culture of
www.aahe.org/ffrr/ undergraduate science teaching. Journal of
principles_brochure.htm>. College Science Teaching 30(5), 306–312.
Springer, L., Stanne, M.E., and Donovan, S.S. Zimpher, N. (1998). Ten changing demands on
(1998). Effects of small-group learning on college teachers in the future. Presented at
undergraduates in science, mathematics, Changing Demands on College Teachers: A
engineering, and technology: A meta-analysis Conference for Teaching Support Providers,
(Research Monograph No. 11). Madison: April 27, Columbus, Ohio. See <http://
University of Wisconsin-Madison, National www.acs.ohio-state.edu/education/ftad/
Institute for Science Education. Publications/ten-nancy.html>.
Suskie, L. (Ed.). (2000). Assessment to promote
deep learning. Washington, DC: American
Association for Higher Education. See <http:/
/www.aahe.org/catalog/
iteminfo.cfm?itemid=1&itemid=127&g=t>.
Suter, L., and Frechtling, J. (2000). Guiding
principles for mathematics and science
education research methods: Report of a
workshop. (NSF 00-113). Arlington, VA:

Appendix A
Selected Student
Evaluation Instruments
TYPES OF STUDENT influenced by their level of motivation

EVALUATION INSTRUMENTS for taking the course, attitude toward
the course or the instructor, and needs
or contextual variables (e.g., whether
Current Students: the course is required). Findings from
End-of-Course Questionnaires research on the use of student question-
Questionnaires administered at the naires suggest that when these instru-
end of the term have long been widely ments are used, the results should be
used to elicit students’ opinions about compared with data from student
individual courses or instructors questionnaires in similar courses.
(Seldin, 1998). Studies on the reliability Those who design or use data from
and validity of these types of student student questionnaires must be careful
ratings have been undertaken for more to distinguish instruments that ask
than 70 years (Centra, 1993). Students students to evaluate courses from those
are in a unique position to comment on that ask them to evaluate the instruction
their satisfaction with a course and the or the instructor. Forms are often
impact of the instruction on their own constructed to ask students to rate
learning. However, they are not subject various aspects of a course and then to
matter experts, and therefore are not in provide a rating for the professor’s
a position to make judgments about the performance. Use of such data for
currency or accuracy of course content. evaluating teaching effectiveness
In addition, research has shown that becomes problematic if most of the
ratings by students are sometimes questions asked of students focus on
139

components of the course itself, such as Current Students:

the usefulness of the textbook or Measures of Learning
amount of material covered. An extremely useful and increasingly
common approach to evaluating teach-
Current Students:
ing effectiveness is to measure students’
Interviews
knowledge or skills at the beginning of a
Interviewing students can provide course or unit of the course and again
rich, in-depth information about their after some body of material has been
responses to courses and instructors. covered in class. Instructors can then
When used appropriately, such inter- observe and quantify the amount of
views are usually either highly struc- improvement and draw inferences about
tured (following a specific set of ques- the instructor’s effectiveness in helping
tions and protocol), semistructured students learn the subject matter. For
(with a few general items), or unstruc- measures of student learning to be
tured (e.g., “Tell me about this class”). considered valid and reliable, however,
Interviews can probe details and ex- considerable effort is required to
plore aspects of a course and the develop pre- and post-learning tests that
instructor’s role in it in ways that written actually measure the kind of learning
questionnaires cannot. However, desired. In addition, changes observed
interviewing sufficient numbers of in students’ learning and performance
students to obtain an accurate picture of cannot be attributed solely to the effec-
the instructor’s teaching and interpret- tiveness of an individual instructor.
ing the results can require a great deal Many factors, including students’ ability
of time, rendering this approach some- and motivation to learn and even their
what impractical. Research also indi- health status when taking either exami-
cates that interviews are most helpful nation, can also influence the outcomes.
when they are used to provide feedback Indirect measures of student learning
for improving teaching rather than for can be obtained through questionnaires
summative evaluation. Information that ask students to assess their own
garnered from interviews also can be achievement (e.g., “How much have you
more helpful to the instructor when the learned from this course?”). Some
interviewing is done by an instructional research (e.g., Pike, 1995) has shown
improvement specialist, if available, or a that students’ answers to such questions
trusted colleague (Centra, 1993). are correlated with their performance
on end-of-course tests.

Another useful approach is for the learned well the knowledge and skills
instructor to evaluate student learning they need to move through a vertically
throughout the term. Instructors can structured departmental curriculum.
use the information obtained from these
regular assessments of student learning
to improve their teaching and make GUIDELINES FOR THE USE OF
midcourse corrections in the ap- STUDENT EVALUATIONS
proaches they are using. Faculty
members can thus conduct their own Having examined the research
classroom research, gathering mea- literature and practices in several
sures of student learning to improve different types of institutions of higher
their teaching (Brookfield, 1995; Na- education, the committee offers here
tional Institute for Science Education, guidelines for the use of student evalua-
2001b). An instructor’s use of such tions, particularly in making decisions
approaches, the range of test instru- about a faculty member’s professional
ments employed (e.g., short-answer and life. Centra (1993: especially 89–93)
essay questions, computer simulations, offers a detailed discussion of the issues
and laboratory-based problems, in involved; the suggestions offered below
addition to multiple-choice and similar are based in part on that analysis.
kinds of questions) and the ways in Make clear to faculty and students how
which the instructor responds to indica- results of student evaluations will be used.
tors of student learning can be useful Faculty members, administrators, and
measures of teaching effectiveness. students need to understand both how
Instructors also can benefit from the results will be used and who will
knowing whether students who have have access to them.
taken their courses have mastered Use student evaluation as only one
concepts and skills that will be needed piece of relevant information from several
for subsequent, higher level courses. sources. Because student evaluations
Thus, questions about specific concepts represent student views only, other
the students will have been expected to sources of information (colleagues, self-
learn can be included in pre/post- reports, evidence of student learning)
testing. Alternatively, as part of their must be considered. Student evalua-
evaluation of program effectiveness, tions are relatively easy to obtain, but
academic departments can develop that should not result in giving them
assessment instruments that can be undue weight. Note that when multiple
used to examine whether students have sources of evaluation data are used,
S E L E C T E D S T U D E N T E VA L U AT I O N I N S T R U M E N T S 141

consensus must be reached on how reliable. If the class has fewer than 10
each source will be weighted when students, it is best not to summarize the
making decisions about teaching effec- data. For sufficiently large sample
tiveness. sizes, means and standard deviations
Use several sets of evaluation results. are used most frequently to summarize
For personnel decisions, a pattern of data.
evaluation results derived from different Consider some course characteristics in
courses taught over more than one interpretations. While any single course
semester should be used. Using results variable may not have a great effect, a
from five or more classes is generally combination (e.g., small classes, course
best. Also, the results of student evalua- subject area) could affect a teacher’s
tions should be compared with a histori- mean rating.
cal record for that class or type of class, Use comparative data. Comparisons
if such data are available. among instructors within an institution
Have a sufficient number of students or, better yet, across a large number of
evaluate each course. Averaging re- similar institutions can help in interpret-
sponses from a sufficient number of ing results by minimizing the effects of
students minimizes the effects of a few any skewed distributions.
divergent opinions. Reliability estimates Do not overestimate small differences.
(see Chapter 4 for a definition of reliabil- Because student evaluations typically
ity as used in psychometrics) are excel- are quantified, there may be a tendency
lent for classes of 25 students or more. to assign them a precision they do not
In classes with fewer students, it is possess or warrant. A 10-percentile
critical to examine patterns of student difference between instructors gener-
responses across a number of classes. ally does not represent a practical
Reliability estimates for classes of 15 or distinction.
more are at an acceptable level. For For personnel decisions, emphasize
very large classes, a representative or global evaluations and estimates of
random sample of students totaling 25 learning. Overall ratings of instruction
or more can be selected to complete the or of a course tend to correlate highly
form. An effort should be made to with measured student achievement—
encourage at least 60 percent of enrolled more highly than ratings dealing with
students to participate in the evaluation, different teaching styles and presenta-
and at least 15–25 questionnaires are tion methods. Students’ estimates of
needed for results to be considered their own learning also can be useful

and reasonably accurate means of and then require students to attend a

assessing this aspect of teaching effec- session where they receive their graded
tiveness. examination and are asked to complete
Use standardized procedures for the evaluation form. This approach
administering forms in class. When allows students to review the
results may be used in personnel instructor’s comments on their final
decisions, standardized procedures are examination, making the examination a
necessary to minimize possible biasing more important component of the
effects. These procedures include overall learning experience in the
having the instructor leave the room course. Having this information and
and providing consistent information to perspective allows students to offer a
students about how the data will be more complete evaluation of the course.
used. Departments and institutions It is important to note, however, that
should also develop policies to ensure employing this technique may well
uniform procedures for distributing, result in an instructor’s receiving lower
collecting, and analyzing standardized evaluations than instructors who distrib-
forms. Normally, forms are completed ute the evaluations before administering
anonymously in class. Some schools the final examination. This difference in
also require that students either return approaches should be considered in any
their evaluation forms to an administra- summative evaluation of a faculty
tive office individually or give them to a member’s teaching.
student in the class who is assigned to Expect those being evaluated to respond
deliver them. An ideal approach is to to evaluation results. Faculty should
use special staff, such as those from the have the opportunity to discuss with
teaching and learning center, to admin- their department chair or others in-
ister and collect rating forms. Another volved in personnel decisions any
possibility is to use department secre- circumstances they believe may have
tarial staff. Use of student volunteers is affected student evaluations of their
least desirable. teaching. They also should be asked to
Student evaluations are most com- describe in writing what they were
monly completed at the end of the trying to accomplish in the course and
course and prior to final exams or how their teaching methods suited
grades. They can also be distributed at those objectives (e.g., Hutchings, 1998).
midsemester to assist in instructional Their written comments should be
improvement. Another approach is to placed in their official dossier or wher-
administer the final examination early ever the student ratings are kept. It also
S E L E C T E D S T U D E N T E VA L U AT I O N I N S T R U M E N T S 143

is important to keep in mind that tradi- point of diminishing returns. If they are
tional student rating forms often do not overused, neither students nor instruc-
reflect an instructor’s effectiveness in tors will give them the level of attention
less traditional teaching or testing required for fair evaluation of teaching
environments. or continued professional development
Limit the use of rating forms. The use by the faculty member in question.
of student rating forms may reach a

Appendix B
Samples of Questionnaires
Used to Evaluate
Undergraduate Student Learning
The student questionnaires included Educational Testing Service) are ge-

in this appendix exemplify several neric approaches to evaluating learning.
approaches to assessing student learn- Evaluation forms from several universi-
ing and, in turn, using that information ties are presented to demonstrate the
to improve teaching. The College kind of information that might be
Student Report 2000 (from the Carnegie sought from students and the variety of
Foundation for the Advancement of ways (e.g., end-of-semester,
Teaching and Pew Forum on Under- midsemester) and settings (e.g., class-
graduate Education, 2000) and the room, laboratory) in which students can
Student Instructional Report II (from the be queried.
Form Found on Page(s)
The College Student Report 2001. From the Carnegie 147–150

Foundation for the Advancement of Teaching and
Pew Forum on Undergraduate Education (2002).
Student Instructional Report II. Used nationally and 151–152

produced by the Educational Testing Service.
145

Carnegie Mellon University Eberly Center for

Teaching Excellence
Teaching Improvement Form—Lecture Courses 153–156
Teaching Improvement Form—Laboratory Courses 157–159
Teaching Improvement Form—Discussion Courses 160–162
Harvard University Derek Bok Center for

Teaching and Learning
Mid-Course Evaluation Form 163
End-of-Semester Course Evaluation Form 164–165
Kansas State University IDEA Center

Student Reactions to Instruction and Courses 166–167
Faculty Information Form for Student Evaluations 168–169
Sample Results of Student Evaluations 170–177
Hampshire College
End-of-Semester Course Evaluation Forms 178–182
Instructor Objectives Report 183–184

The College Student Report 2001

1 In your experience at your institution during the current school year, about how often have you done each of
the following? Mark your answers in the boxes. Examples: or
Very Some- Very Some-
often Often times Never often Often times Never
a. Asked questions in class or r. Discussed ideas from your

contributed to class discussions readings or classes with others
outside of class (students,
b. Made a class presentation family members, coworkers, etc.)
c. Prepared two or more drafts s. Had serious conversations with
of a paper or assignment students of a different race or
before turning it in ethnicity than your own
d. Worked on a paper or project that t. Had serious conversations
required integrating ideas or with students who differ from
information from various sources you in terms of their religious
beliefs, political opinions, or
e. Came to class without completing
personal values
readings or assignments
f. Worked with other students on

projects during class
g. Worked with classmates

outside of class to prepare
class assignments 2 During the current school year, to what extent has
h. Tutored or taught other your coursework emphasized the following mental
students (paid or voluntary) activities?
Very Quite Very
i. Participated in a community-based much a bit Some little
project as part of a regular course
j. Used an electronic medium a. Memorizing facts, ideas, or

(list-serv, chat group, Internet, methods from your courses and
etc.) to discuss or complete an readings so you can repeat them
assignment in pretty much the same form
k. Used e-mail to communicate
b. Analyzing the basic elements of
with an instructor
an idea, experience, or theory
l. Discussed grades or such as examining a particular
assignments with an instructor case or situation in depth and
considering its components
m. Talked about career plans with
a faculty member or advisor c. Synthesizing and organizing
n. Discussed ideas from your reading ideas, information, or experiences
or classes with faculty members into new, more complex
outside of class interpretations and relationships
o. Received prompt feedback from d. Making judgments about the

faculty on your academic value of information, arguments,
performance (written or oral) or methods such as examining
how others gathered and
p. Worked harder than you thought interpreted data and assessing
you could to meet an instructor's the soundness of their conclusions
standards or expectations
e. Applying theories or
q. Worked with faculty members on concepts to practical
activities other than coursework problems or in new situations
(committees, orientation, student
life activities, etc.)
SAMPLES OF QUESTIONNAIRES 147

3 During the current school More than 20 6 Which of the following have you done or do you
year, about how much Between 11 and 20 plan to do before you graduate from your institution?
reading and writing Between 5 and 10
have you done? Yes No Undecided
Between 1 and 4
None
a. Practicum, internship, field
experience, co-op experience,
a. Number of assigned textbooks, or clinical assignment
books, or book-length packs of
course readings b. Community service or
volunteer work
b. Number of books read on your own
(not assigned) for personal c. Work on a research project with a
enjoyment or academic enrichment faculty member outside of course
or program requirements
c. Number of written papers or reports
of 20 pages or more d. Foreign language coursework
d. Number of written papers or reports e. Study abroad
between 5 and 19 pages
f. Independent study or
e. Number of written papers or reports self-designed major
of fewer than 5 pages g. Culminating senior experience
(comprehensive exam, capstone
course, thesis, project, etc.)
4 Mark the box that best represents the extent to
which your examinations during the current school
year have challenged you to do your best work. 7 About how many hours do
you spend in a typical 7-day More than 30
week doing each of the 26 - 30
Very much
following? 21 - 25
16 - 20
7 # of hours 11 - 15
per week 6 - 10
6 1-5
0
5 a. Preparing for class
(studying, reading,
writing, rehearsing, and
4 other activities related to
your academic program)
3 b. Working for pay on
campus
2 c. Working for pay off
campus
d. Participating in co-
1
curricular activities
(organizations, campus
publications, student
Very little government, social
fraternity or sorority,
intercollegiate or
intramural sports, etc.)
5 Overall, how would you evaluate the quality of
academic advising you have received at your e. Relaxing and socializing
(watching TV, partying,
institution?
exercising, playing
Excellent computer and other
Good games, etc.)
Fair f. Providing care for
dependents living with
Poor
you (parents, children,
spouse, etc.)

8 To what extent has your institution experience 10 Mark the box that best represents the quality of
contributed to your knowledge, skills, and personal your relationships with people at your institution.
development in the following areas?
Relationships with:
Very Quite Very
much a bit Some little
a. b. c.
a. Acquiring a broad Administrative

general education Other Faculty Personnel and
b. Acquiring job or work-related Students Members Offices
knowledge and skills
c. Writing clearly and effectively Friendly,
Supportive, Available, Helpful,
d. Speaking clearly and effectively Sense of Helpful, Considerate,
Belonging Sympathetic Flexible
e. Thinking critically and analytically
f. Analyzing quantitative problems
g. Using computing and information
technology 7 7 7
h. Working effectively with others

6 6 6
i. Voting in local, state, or
national elections
5 5 5
j. Learning effectively on your own
k. Understanding yourself
4 4 4
l. Understanding people of other
racial and ethnic backgrounds
3 3 3
m. Developing a personal code of
values and ethics
2 2 2
n. Contributing to the welfare
of your community
1 1 1
9 To what extent does your institution emphasize Unfriendly, Unavailable, Unhelpful,

each of the following? Unsupportive, Unhelpful, Inconsiderate,
Sense of Unsympathetic Rigid
Very Quite Very Alienation
much a bit Some little
a. Spending significant amounts

of time studying and on 11 How would you evaluate your entire educational
academic work experience at this institution?
b. Providing the support you need Excellent

to help you succeed academically Good
Fair
c. Encouraging contact among
students from different Poor
economic, social, and racial
or ethnic backgrounds
12 If you could start over again, would you go to the
d. Helping you cope with your
non-academic responsibilities
same institution you are now attending?
(work, family, etc.) Definitely yes
e. Providing the support you Probably yes
need to thrive socially Probably no
Definitely no

13 Write in your year of birth: 1 9 24 Which of the following best describes where
you are living now while attending college?
14 Your sex
Dormitory or other campus housing (not fraternity/sorority
Male Female house)
Residence (house, apartment, etc.) within walking
15 Are you of Hispanic, Latino, or Spanish origin?
distance of the institution
Yes No Residence (house, apartment, etc.) within driving
distance
16 What is your racial or ethnic identification? Fraternity or sorority house
(Mark all that apply)
25 Did either of your parents graduate from
American Indian or other Native American college?
Asian American or Pacific Islander Yes, both parents No
Black or African American Yes, father only Don't know
White Yes, mother only
Other: Specify 26 Which of these fields best describes your major(s)
or your expected major(s)? Mark only one major in
17 Are you an international student or foreign each column.
national?
Primary Second Major (not minor, concentration, etc)
Yes No Major (if applicable)
Agriculture
18 What is your current classification in college? Biological/life sciences (biology,
biochemistry, botany, zoology, etc.)
Freshman/first-year Sophomore
Business (accounting, business admin.,
Junior Senior
marketing, management, etc.)
Unclassified
Communications (speech, journalism,
television/radio, etc.)
19 Since high school, which of the following
types of schools have you attended other than the Computer and information sciences
one you are attending now? (Mark all that apply) Education
Engineering
Vocational-technical school
Ethnic, cultural studies, and area studies
Community or junior college
Foreign languages and literature (French,
4-year college other than this one Spanish, etc.)
None
Health-related fields (nursing, physical
Other: Specify therapy, health technology, etc.)
Humanities (English, literature,
20 Did you begin college at your current philosophy, religion, etc.)
institution or elsewhere? Liberal/general studies
Started here Started elsewhere Mathematics
Multi/interdisciplinary studies (international
21 Thinking about this current academic term, relations, ecology, environmental studies, etc.)
how would you characterize your enrollment?
Parks, recreation, leisure studies, sports
Full-time Less than full-time management
Physical sciences (physics, chemistry,
22 Are you a member of a social fraternity or astronomy, earth sciences, etc.)
sorority?
Public administration (city management,
Yes No law enforcement, etc.)
Social sciences (anthropology, economics,
23 Do you intend to teach at some
history, political science, psychology,
pre-kindergarten through high school grade level
sociology, etc.)
within a year or two of completing your degree
Visual and performing arts (art, music,
program?
theater, etc.)
Yes No Undecided Undecided
Other: Specify
THANKS FOR SHARING YOUR VIEWS!

After completing The Report, please put it in the enclosed postage-paid envelope and deposit it in any U.S.
Postal Service mailbox. This study is supported by a grant from The Pew Charitable Trusts. Questions or
comments? Contact the National Survey of Student Engagement, Indiana University, Ashton Aley Hall, 1913
East Seventh Street, Bloomington IN 47405 or nsse@indiana.edu or www.indiana.edu/~nsse. Copyright
pending.



































Appendix C
Examples of Questions for Conducting

Peer Evaluations of Teaching
This report has emphasized the The forms included in this appendix
importance of using multiple ap- serve as examples of peer evaluation
proaches in evaluating teaching effec- surveys. French-Lazovik’s (1981) form
tiveness. As discussed in Chapters 4 is designed to assist faculty in evaluating
and 5, feedback from faculty colleagues their colleagues on the basis of written
can be a highly useful source of informa- materials that are provided in a dossier.
tion for improving teaching and learn- The forms from Syracuse University
ing. However, research has indicated and The University of Texas outline
that faculty colleagues can be far more behaviors that colleagues can observe
effective in this role if they are trained directly when they visit their colleagues’
in how to conduct peer evaluations and classrooms.
if they work from an accepted set of
criteria.
From French-Lazovik (1981): Suggested Form for

Peer Review of Undergraduate Teaching Based
on Dossier Materials 186–187
Syracuse University: Classroom Observation Worksheet 188–192
University of Texas at Austin, Center for Teaching

Effectiveness: Checklist of Teaching Skills 193–195
185


E X A M P L E S O F Q U E S T I O N S F O R C O N D U C T I N G P E E R E VA L U A T I O N S 187









Appendix D
Biographical Sketches of
Committee Members
Mar ye Anne Fox, Co-chair, is chancel- Undergraduate Science Education, she

lor of North Carolina State University. serves on the Committee on Science,
Prior to assuming the chancellorship in Engineering, and Public Policy and as
1998, Dr. Fox served as Vice President cochair of the Government–University–
for Research and the M. June and Industry Research Roundtable. Dr. Fox
J. Virgil Waggoner Regents Chair in is a former member of the Commission
chemistry at the University of Texas at on Physical Sciences, Mathematics, and
Austin. Her recent research activities Applications; the NAS Council; and the
include organic photochemistry, electro- NRC Governing Board. She also served
chemistry, and physical organic mecha- on the Committee on Criteria for Fed-
nisms. She is a former editor of the eral Support of Research and Develop-
Journal of the American Chemical ment.
Society. Previously, she was the director
for the Center for Fast Kinetics Re- Norman Hackerman, Co-chair, served
search, vice chair of the National Sci- as president of Rice University from
ence Board, and a member of the Task 1970 to 1985 and holds the positions of
Force on Alternative Futures for the president emeritus and distinguished
Department of Energy National Labora- professor emeritus of chemistry at Rice
tories (the Galvin Committee). Dr. Fox University. Prior to coming to Rice, Dr.
is a member of the National Academy of Hackerman spent 25 years at The
Sciences (NAS) and serves on several University of Texas, Austin, Texas,
NAS and National Research Council where he joined the faculty as an assis-
(NRC) committees. In addition to her tant professor of chemistry in 1945 and
role as cochair of the Committee on became president in 1967. He is now
196

professor emeritus of chemistry at The Scientific Advisory Board of the Robert

University of Texas at Austin. He taught A. Welch Foundation.
chemistry at Loyola College and Vir-
ginia Polytechnic, and worked as a Trudy Banta is vice chancellor for
research chemist for Colloid Corpora- planning and institutional improvement,
tion, Kellex Corporation, and the U.S. Indiana University–Purdue University at
Coast Guard. Dr. Hackerman was a Indianapolis. She has edited five pub-
member of the National Science Board lished volumes on assessment, contrib-
from 1968 to 1980 and chairman from uted 15 chapters to other published
1957 to 1980. He was editor of the works, and written more than 80 articles
Journal of the Electrochemical Society and reports. Making a Difference:
from 1969 to 1989. He is a member of Outcomes of a Decade of Assessment in
the NAS, the American Philosophical Higher Education was published by
Society, and the American Academy of Jossey-Bass in October 1993. Dr.
Arts and Sciences and belongs to Banta’s most recent work, Assessment in
numerous scientific organizations. He Practice: Putting Principles to Work on
is author or coauthor of 225 publica- College Campuses, was published by
tions. In addition to several previous Jossey-Bass in early 1996. She is the
awards, Dr. Hackerman received the founding editor of Assessment Update, a
American Institute of Chemists Gold bimonthly periodical published by
Medal in March 1978, the Mirabeau B. Jossey-Bass. Dr. Banta has developed,
Lamar Award of the Association of coordinated, and addressed conferences
Texas Colleges and Universities in 1981, worldwide on assessing quality in
the Distinguished Alumnus Award from higher education and matters related to
The Johns Hopkins University in 1982, outcome assessment. She has con-
the Edward Goodrich Acheson Award of sulted with faculty and administrators
the Electrochemical Society in 1984, the on campuses and at statewide confer-
Alumni Gold Medal for distinguished ences in 37 states. In 1997, she was
service to Rice University in 1984, the recognized by the American Productiv-
Charles Lathrop Parsons Award of the ity and Quality Center for leadership of
American Chemical Society in 1987, the one of the seven most effective pro-
AAAS-Philip Hauge Abelson Prize in grams in the world for using manage-
1987, the Vannevar Bush Award of the ment information in decision making.
National Science Board in 1993, and the
National Medal of Science in 1993. Dr. John Centra is a research professor
Hackerman serves as Chairman of the and professor emeritus at Syracuse
BIOGRAPHICAL SKETCHES OF COMMITTEE MEMBERS 197

University. He is a former chair of the topics related to teaching, learning, and

Higher Education Program at the evaluation; has written about these
university. Prior to coming to Syracuse topics in a number of articles, book
in 1985, he was a research psychologist chapters, and evaluation reports (includ-
at the Educational Testing Service, ing a chapter on assessment in the NRC
where he conducted studies on college report Science Teaching Reconsidered);
teaching, faculty development, student and authored or coauthored five books.
learning, the effects of colleges on
students, and other topics. He is the Denice Denton is dean of engineering
author of Determining Faculty Effective- and a professor in the Department of
ness (1979) and Reflective Faculty Evalu- Electrical Engineering at the University
ation (1993), and coauthor of Tenure, of Washington. Her current interests
Promotion, and Reappointment: Legal include plasma deposition of polymers
and Administrative Implications (1995) and the use of micromachining in solid
and more than 85 articles, monographs, state actuator design. Professor Denton
and books. He has consulted or given was codirector of the National Institute
talks at well over a hundred colleges in for Science Education 1995–1996. She
the United States and abroad. Dr. is a recipient of the National Science
Centra’s current research is on assess- Foundation Presidential Young Investi-
ing the scholarship of teaching. In 1993 gator Award (1987–1992), the American
he received a career achievement award Society of Engineering Education AT&T
from the American Educational Re- Foundation Teaching Award (1991), the
search Association’s Special Interest WM. Keck Foundation Engineering
Group on Faculty Evaluation and Devel- Teaching Excellence Award (1994), the
opment. American Society of Electrical Engi-
neers George Westinghouse Award
Barbara Gross Davis is assistant vice (1995), and the Institute of Electronic
chancellor, Student Life and Educational and Electrical Engineering Harriet B.
Development, University of California- Rigas Teaching Award (1995).
Berkeley. Dr. Davis’ primary interests
are in instructional improvement, Diane Ebert-May is director of Lyman
assessment and accreditation, faculty Briggs School, a residential, liberal arts
development and evaluation, and pro- science program within the College of
gram and curriculum evaluation in Natural Sciences at Michigan State
higher education. She has conducted University, and is a professor of botany
workshops and seminars for faculty on and plant pathology. She provides

national leadership in promoting profes- ecological research on alpine tundra

sional development opportunities for plant communities since 1971.
faculty, postdoctoral teaching fellows,
and graduate students who participate Timothy Goldsmith is professor of
actively not only in their own discipline- molecular, cellular, and developmental
based research, but also in creative biology at Yale University. He has
scholarship and research on teaching experience in the classroom and is
and learning. She chairs the Education actively involved with other educational
Committee of the National Long Term activities at Yale, the NRC, and else-
Ecological Research Network and is where. Dr. Goldsmith has served on
chairperson of the Education Section of numerous NRC boards and committees,
the Ecological Society of America. Her including the Commission on Life
current research in biology education is Sciences, the Board on Biology, and the
based on an empirically based model Board on International Comparative
she developed to test the effectiveness Studies in Education. He was a member
of active learning in a large introductory of the advisory board for the NRC’s
biology course for nonmajors and an Center for Science, Mathematics, and
ecology course for majors. From this Engineering Education. He also serves
she has developed models for using as chair of the board of directors for the
argument structure to develop assess- Biological Sciences Curriculum Study
ments for critical thinking. Dr. Ebert- and is a member of the American
May’s recent publications describe the Academy of Arts and Sciences. Among
inquiry-based instructional strategies his other teaching responsibilities, Dr.
she uses in a course with large class Goldsmith teaches an undergraduate
meetings (lectures) and multiple labora- course for nonmajors, for which he is
tory sections. Her research, funded by also writing (with W. F. Zimmerman) a
the National Science Foundation, the textbook entitled Biology, Evolution, and
National Institutes of Health, and Human Nature. His research involves
Howard Hughes Medical Institute, physiological and behavioral aspects of
focuses on alternative assessments for photopigments and photoreception in
large science courses, including student invertebrate and vertebrate animals.
self-reflection as a form of student
evaluation. Her ecological research Manuel Gomez is vice president for
continues on Niwot Ridge, Colorado, research and academic affairs at the
where she has conducted long-term University of Puerto Rico. He has

overseen the implementation of an Lewis has served in a variety of capaci-

assessment plan that is driving the ties at the University of California-
reform of undergraduate science, Berkeley, including visiting professor in
technology, engineering, and mathemat- the Graduate School of Education,
ics (STEM) education at the University director of assessment and evaluation
of Puerto Rico, a multicampus system and then project director for the
with 68,000 students, eight 2- and 4-year ModularCHEM Consortium, and
colleges, and three campuses offering currently principal investigator for the
the Ph.D. degree. Dr. Gomez is a Multi-Initiative Dissemination Project in
theoretical physicist, specializing in the Department of Chemistry. She has
solid state and condensed matter phys- also served as a National Institute of
ics. Upon his graduation, he received a Science Education Fellow at the Univer-
research fellowship from the NRC to sity of Wisconsin, Madison, and as
work on the optical properties of solids editor of FLAG (Field-Tested Learning
at the Naval Research Laboratory in Assessment Guide). She serves on the
Washington, DC. He served as profes- editorial boards of the International
sor of physics and chairperson of the Journal of Science Education and the
physics department at the University of Journal of Science Education and Tech-
Puerto Rico. He was then appointed nology.
dean of the College of Natural Sciences
and later became the director of the Jeanne L. Narum is director of the
Resource Center for Science and Engi- Independent Colleges Office and
neering. Dr. Gomez has been director Project Kaleidoscope (PKAL). PKAL is
of the Puerto Rico EPSCoR program a consortium of colleges and universi-
since its inception in 1986. He also ties across the United States that for the
served as a member of the NRC’s past 10 years has sought to discover and
Coordinating Council for Education. disseminate best practices in under-
graduate STEM education. A major
Eileen Lewis is professor of chemistry component of PKAL is the “Faculty for
at Cañada College (California). Her the 21st Century,” which has as its goal
academic training is in chemistry and identifying and providing professional
cognition, and her research interests development for up to 1,000 pretenured
include conceptual change in students’ STEM faculty members from a variety
understanding of science, curricular of types of postsecondary institutions
designs that support knowledge integra- who have been recognized as potential
tion, and systemic reform issues. Dr. leaders and educational innovators on

their campuses. As a result of her departments of mathematics, philoso-

leadership in PKAL, Ms. Narum has phy, psychology, the history and phi-
extensive background in and knowledge losophy of science, and education. He
of university science curricula, issues has held fellowships from the Educa-
related to the improvement of college tional Testing Service, the Center for
teaching, and the culture of higher Advanced Studies in the Behavioral
education. Sciences in Palo Alto, and the National
Science Foundation, among others. His
Cornelius J. Pings is immediate past more than 300 publications are mainly
president, Association of American in the areas of critical thinking, technol-
Universities (AAU), and thoroughly ogy and computer studies, and evalua-
knowledgeable about the culture of and tion. Dr. Scriven is well known for his
issues surrounding higher education. expertise in evaluation. He is credited
In addition to his recent presidency of with coining the terms “formative” and
the AAU, he held positions as provost “summative” to describe different kinds
and senior vice president for academic of personnel and program evaluations.
affairs at the University of Southern He is an ex-president of the American
California and as professor of chemical Educational Research Association and
engineering, vice provost, and dean of the American Evaluation Association,
graduate studies at the California and is the 2000 recipient of the
Institute of Technology. He also has McKeachie Award for lifetime contribu-
considerable expertise in the corporate tion to the methodology of faculty
sector, having served as director of the evaluation.
Farmers Group, Nations Funds, Maxtor
Corporation, and the Hughes Aircraft Christine Stevens is professor of
Company. Dr. Pings has been a mem- mathematics at St. Louis University.
ber and chair of the National Acad- She has extensive experience with
emies’ Committee on Science, Engineer- educational issues. She is the recipient
ing, and Public Policy. of both statewide and national awards
for distinguished college and university
Michael Scriven is professor of psy- teaching from the Mathematical Asso-
chology at Claremont Graduate Univer- ciation of America. Dr. Stevens also has
sity. His academic training is in math- served as associate program director for
ematics and the philosophy of the National Science Foundation’s
mathematical logic. He has taught in Teacher Enhancement Program. She is
the United States and Australia in involved extensively with Project NExT,

a professional development program for in conjunction with archaeologists who

new and recent Ph.D.s in the math- are seeking to locate sites of prehistoric
ematical sciences that addresses issues settlements in the New York area. As a
in the teaching and learning of under- dean of science, Dr. Weiss has been
graduate mathematics. Dr. Stevens has active in overseeing educational reform
authored articles on the implications of in departments and programs under his
the National Council of Teachers of purview. He has attended Project
Mathematics standards for undergradu- Kaleidoscope workshops on science and
ate education, an assessment of calculus mathematics at Urban and Commuter
reform efforts, and the history of Institutions and Science for All Students
mathematics. She has served on and an A.C.E. Workshop on Chairing
several committees of the Mathematical the Academic Department. He was a
Association of America and the Society convenor for a university-wide faculty
for Industrial and Applied Mathematics development workshop on mentoring
dealing with education, science policy, students and served on CCNY’s
and minority participation in mathemat- President’s Task Force on Advising and
ics. Her scholarly interests are con- Mentoring. At CCNY he was the
cerned with topological groups. principal investigator for a grant from
National Science Foundation to estab-
Dennis Weiss is dean of natural sci- lish a Faculty Development Center at
ence and mathematics at The Richard CCNY. He has taught numerous
Stockton College of New Jersey. He was courses in the earth sciences to both
previously dean of science at the City undergraduate and graduate students.
College of New York (CCNY). While at CCNY, a campus with a strong
Dr. Weiss’s research deals with bottom union, Dr. Weiss is working on his
and subbottom mapping of New York’s campus to change evaluation proce-
coastal zones, including the waterways dures for faculty.
in and around New York City and the
continental shelf lying off the coast of
New York City. This work is being done

Index
A American Association for Higher Education

(AAHE), 30, 48, 61, 65, 109, 119
Campus Program initiative, 123
AAAS. See American Association for the Forum on Faculty Roles and Rewards, 13n
Advancement of Science Making Teaching Community Property: A
AAHE. See American Association for Higher Menu for Peer Collaboration and Peer
Education Review, 86
ABET. See Accreditation Board for Engineering projects of, 86
and Technology American Association for the Advancement of
Academic advising, 113 Science (AAAS), 110
Academic freedom American Chemical Society, 46, 109
protecting students’, 31 American Educational Research Association
Accountability (AERA), 55
calls for, 12-13 American Geophysical Union
Accreditation agencies and boards Shaping the Future of Undergraduate Earth
revising policies to emphasize quality Science Education, 47
undergraduate learning as a primary American Institute for Biological Sciences, 47
criterion for program accreditation, 7, 127 American Institute of Physics, 47
Accreditation Board for Engineering and American Mathematical Society, 46
Technology (ABET), 49, 109 American Mathematics Association of Two Year
Active learning strategies Colleges, 46
promoting, 29 American Physical Society, 47
Ad hoc committees on teaching effectiveness, 93- American Psychological Association (APA), 49,
95 55
Adjustments Analysis of particular strengths and weaknesses
in expected learning outcomes, 73 of the teaching, 93
Administration of forms in class Answering students’ questions, 28
standardizing procedures for, 143 APA. See American Psychological Association
AERA. See American Educational Research Application of formative evidence about student
Association learning
Alumni to departmental programs, 2
data for evaluating teaching quality and Applications of research, 18-19, 69-127
effectiveness from, 60 evaluation methodologies, 18, 71-99
Alverno College, 52n evaluation of departmental undergraduate
programs, 18-19, 108-114
203

evaluation of individual faculty, 18-19, 100-107 Career planning, 113

into human cognition and learning, 111 Carnegie Academy for the Scholarship of
recommendations from, 4-8, 115-127 Teaching and Learning, 86
Appropriateness of assessment practices, 30, 103- Carnegie Foundation for the Advancement of
104 Teaching, 47, 77, 86, 123, 145, 147-150
assessing learning in ways consistent with the Carnegie Mellon University Eberly Center for
objectives of a course, 30 Teaching Excellence
data sources and forms of evaluation for Teaching Improvement Form—Discussion
evaluating proficiency in, 104 Courses, 160-162
determining students’ knowledge accurately Teaching Improvement Form—Laboratory
and fairly, 30 Courses, 157-159
knowing whether students are learning what Teaching Improvement Form—Lecture
is being taught, 30 Courses, 153-156
for testing student learning of specified Centers for Learning and Teaching program,
knowledge, 73 127n
Assessment Forum: Nine Principles of Good Centers for teaching and learning
Practice for Assessing Student Learning, 32- providing opportunities for ongoing
35 professional development, 5-6, 122-123
Assessment of Student Achievement in Change, 12-15
Undergraduate Education, 48 in the appearance of higher education
Assessment of student learning, 112. See also facilities, 25
Appropriateness of assessment practices calls for accountability from outside of
consistent with the objectives of a course, 30 academe, 12
more than grades, 72 calls for accountability from within academe,
nature and quality of, 25 12-13
using results to provide formative feedback to challenges of, 14-15
individual students, 73 in evaluation and documentation of teaching,
Assignments 25
appropriateness of, 93 Classroom observation
Assistance data for evaluating teaching quality and
to students with academic difficulties, 29 effectiveness from, 63-64
Colleagues. See also Formative evaluation by
faculty colleagues
data for evaluating teaching quality and
B effectiveness from, 61-63
evaluation questionnaires, 95-96
evidence about student learning from, 3
Bias College Student Report 2001, The
in undergraduate student evaluations, 58-60 Carnegie Foundation for the Advancement of
Boyer Commission on Educating Teaching, 147-150
Undergraduates in the Research Pew Forum on Undergraduate Education,
University, 40-41, 110 147-150
Brigham Young University, 64n Cooperation
interdepartmental, in improving
undergraduate STEM education, 114
Council for the Advancement and Support of
C Education, 47
Course characteristics
considering in interpretations, 142
Campus-wide centers for teaching and learning Course, Curriculum, and Laboratory
providing opportunities for ongoing Improvement program, 54n
professional development, 5-6, 122-123
204 INDEX

Course materials number of undergraduate students advised,

evaluation of, 63 mentored, or supervised by the faculty
Cultures of research and teaching, 18, 40-50 member, 53
balancing preparation for careers in research number of undergraduate students guided in
and teaching, 43-46 original or applied research by the faculty
developing and implementing improved member, 53
means for evaluating effective teaching and Departmental undergraduate programs. See
learning, 50 Evaluation of departmental undergraduate
increasing support for effective teaching by programs
professional organizations, 46-49 Departments. See also Graduate school faculties
Curriculum design contributing to campus-wide awareness of the
becoming inseparable from teaching and premium placed on improved teaching, 6-7,
learning, 25 125
the collective responsibility of faculty in all establishing panels on teaching effectiveness
departments, 2, 15, 116 and expectations, 98
evidence about student learning from, 3
periodically reviewing their mission statement
to include appropriate emphasis on
D teaching and student learning, 6, 124
practicing the scholarship of teaching, 88
providing funds to faculty to enhance teaching
Data sources for evaluation, 54-67 skills and knowledge, 7, 125-126
of course materials, 63 supporting faculty moving to greater
faculty colleagues, 61-63 emphasis on instruction or educational
graduating seniors and alumni, 60 leadership, 7, 126
institutional data and records, 66-67 Disciplinary-focused centers for teaching and
instructional contributions, 63 learning
of knowledge and enthusiasm for subject providing opportunities for ongoing
matter, 102 professional development, 5-6, 122-123
of professional involvement and contributions, Discussion
107 encouraging, 29
of professionalism with students within and Diversity
beyond the classroom, 106 seen as asset-based, 25
of proficiency in assessment, 104
self-assessment by faculty, 64-66
of skill in and experience with appropriate
pedagogies and technologies, 103 E
students in classroom observations, 63-64
teaching assistants, 60-61
undergraduate student evaluations, 54-60 Educational community
Department heads involving representatives from across, 34
providing personnel recommendations Educational Resources Information Center, 54n
containing separate ratings on teaching, Educational Testing Service, 145, 151-152
research, and service, 7, 125 Educational values
Departmental and institutional records, 53 beginning with, 33
number and levels of courses taught and Effective undergraduate teaching, 18, 25-39. See
number of students enrolled in each also Teaching effectiveness
course or section taught by the instructor challenges to, 32-39
over time, 53 characteristics of, 27-31
number of graduate students mentored in engaging students in original research, 38-39
their preparation as teaching assistants or ensuring availability for all students, 2, 15, 116
future faculty members and their improving the assessment of learning
effectiveness in teaching, 53 outcomes, 32, 35-36
INDEX 205

limitations on faculty knowledge of research Evaluation of departmental undergraduate

about, 39 programs, 18-19, 108-114
predictions about, 25 ability to enhance teaching and learning in
principles of good practice for assessing classrooms and other venues, 111-113
student learning, 33-35 efforts to improve teaching laboratories and
providing engaging laboratory and field other undergraduate research experiences,
experiences, 37-38 113-114
teaching a broad range and large numbers of interdepartmental cooperation in improving
students, 36-37 undergraduate STEM education, 114
End-of-Semester Course Evaluation Forms, 164- Evaluation of individual faculty, 18-19, 100-107
165, 178-182 involvement with and contributing to one’s
Enhancement of teaching and learning, 111-113 own profession in enhancing teaching and
applying research on human cognition and learning, 31, 106-107
learning, 111 knowledge of and enthusiasm for subject
assessing student learning, 112 matter, 27-28, 101-102
emphasizing improved teaching and learning professional interactions with students within
in introductory and lower division courses, and beyond the classroom, 30-31, 104-106
112 proficiency in assessment, 104
employing effective pedagogy, 111 skill, experience, and creativity with a range of
engaging student interest in the department’s appropriate pedagogies and technologies,
curricular offerings, 111 28-30, 101-103
incorporating advances in the discipline and understanding of and skill in using
related subject areas, 112-113 appropriate assessment practices, 30, 103-
providing academic advising and career 104
planning, 113 Evaluation of teaching in STEM disciplines, 18-
Enthusiasm for subject matter, 27-28, 101 19, 51-67, 69-127
conveying infectious, 28 the collective responsibility of faculty in all
data sources and forms of evaluation for departments, 2, 15, 116
evaluating, 102 of departmental undergraduate programs, 18-
genuine interest in what is being taught, 28 19, 108-114
Ethical standards developing and implementing improved
upholding and modeling for students the best means for, 50
in, 31 general principles and overall findings, 51-54
Evaluation methodologies, 18, 71-99. See also input from students and peers, 52-53
Implementation of evaluation methodologies, 18, 71-99
methodologies; Self-assessment recommendations, 4-8, 115-127
addressing the concerns of those critical of review of departmental and institutional
undergraduate teaching and learning, 17 records, 53
advancing and rewarding teaching review of the faculty member’s teaching
scholarship, 17 portfolio and other documentation, 54
evaluating the scholarship of teaching, 87-89 specific sources of data for, 54-67
formative evaluation by faculty colleagues, 83- Evaluation of the scholarship on teaching, 87-89
86, 93-96 adequate preparation, 87
formative evaluation by graduate teaching appropriate methods, 87
assistants, 83, 92-93 clear goals, 87
formative evaluation by undergraduate departments that practice the scholarship of
students, 76-77, 91-92 teaching, 88
improving teaching by examining student dimensions of the scholarship of teaching, 88
learning, 73-76 effective presentation, 87
using several sets of results, 142 institutions that practice the scholarship of
Evaluation of course materials teaching, 89
data for evaluating teaching quality and reflective critique, 87
effectiveness from, 63 significant results, 87
206 INDEX

Evidence of faculty member’s effectiveness making clear how results of student

adaptation of instructional techniques to evaluations will be used, 141
improve student learning, 54 meeting all classes and labs, posting and
participation in efforts to strengthen keeping regular office hours, and holding
departmental or institutional curricula, 54 exams as scheduled, 31
Evidence of student learning, 1-4 meeting responsibilities to students and to the
from combined sources of evidence, 3 public, 35
from departments and other colleagues, 3 participation in seeking external support for
from faculty members being evaluated, 3 activities that further the teaching mission,
from graduate students, 3 54
from institutional data and records, 3-4 publicly recognizing and rewarding those who
from the instructor’s willingness to seek have excelled in teaching, 6, 123-124
external support to improve teaching and rewarding for consistent improving of
learning, 3 learning by both major and nonmajor
from student portfolios, 52 students, 5, 120-121
from undergraduates and graduate teaching supporting and mentoring those working with
assistants, 3 undergraduates throughout their careers,
Expectations 2
of student learning outcomes for an individual supporting in their obligation to improve their
course of study, 73 teaching skills through departmental and
of teaching assistants, appropriateness of, 93 institutional reinforcement, 5, 121
for those being evaluated to respond to using outcomes of effective formative and
evaluation results, 143-144 summative assessments of student
learning to improve their teaching, 17
willingness to seek external support to
improve teaching and learning, 3
F Faculty Code of Conduct Manual, 31n
Faculty Information Form for Student
Evaluations, 168-169
Facilitation of learning Faculty teaching portfolios, 54
through metacognitive strategies that identify, evidence of adaptation of instructional
monitor, and regulate cognitive practices, techniques to improve student learning, 54
21 evidence of participation in efforts to
through socially supported interactions, 22 strengthen departmental or institutional
when new and existing knowledge is curricula, 54
structured around major concepts and including in valid summative assessments of
principles of the discipline, 20 teaching, 4-5, 119-120
Faculty. See also Colleagues; Evaluation of sharing of, 97-98
individual faculty; Graduate school showing participation in seeking external
faculties; Self-assessment support for activities that further the
encouraging to develop curricula that teaching mission, 54
transcend disciplinary boundaries through Feedback
a combination of incentives, 6, 124 for both instructors and students, 72
evidence about student learning from, 3 from graduating seniors and alumni, 98
expecting to contribute to a balanced program Formative evaluation by faculty colleagues, 83-86,
of undergraduate teaching, 6, 124-125 93-96
guiding information searches, 28 ad hoc committees on teaching effectiveness,
having a genuine interest in what is being 93-95
taught, 28 colleagues’ evaluation questionnaires, 95-96
involvement in a larger set of conditions that discussions between the department chair
promote change, 34-35 and individual faculty members, 97
involvement in enhancing teaching and observation, 84-85
learning, 31, 106-107
INDEX 207

from other institutions, 85-86 G

role in “formal” formative evaluation, 85-86
Formative evaluation by graduate teaching
assistants, 83, 92-93 Global evaluations and estimates of learning
analysis of particular strengths and for personnel decisions, 142-143
weaknesses of the teaching, 93 Grade distributions, 66-67
appropriateness of the instructor’s Graduate school faculties
assignments and expectations of the effectively mentoring their teaching assistants
teaching assistants, 93 and advising them about their duties to
asking teaching assistants to review undergraduate students, 6, 124
examinations and quizzes before they are Graduate teaching assistants. See also Formative
given to students, 83 evaluation by graduate teaching assistants
encouraging teaching assistants to provide asked to review examinations and quizzes
information throughout the term about before they are given to students, 83
difficulties students may be having, 83 data for evaluating teaching quality and
extent to which working with the instructor effectiveness from, 60-61
contributed to the teaching assistant’s own evidence about student learning from, 3
professional development, 93 providing information throughout the term
overall judgment of the faculty member’s about difficulties students may be having, 83
teaching effectiveness, 93 Graduating seniors
soliciting constructive suggestions from data for evaluating teaching quality and
teaching assistants, 83 effectiveness from, 60
Formative evaluation by undergraduate students, Guidelines for the use of student evaluations,
76-77, 91-92 141-144
chain notes, 80 considering some course characteristics in
direct questioning of students, 78-79 interpretations, 142
index cards, 81 emphasizing global evaluations and estimates
informal conversations, 80 of learning for personnel decisions, 142-143
minute papers and just-in-time teaching, 79 expecting those being evaluated to respond to
outside evaluators, 81 evaluation results, 143-144
repeated measurements of student learning having a sufficient number of students
and teaching effectiveness, 77-78 evaluate each course, 142
response to students’ concerns, 82 limiting the use of rating forms, 144
Small Group Instruction Diagnosis, 81-82 making clear to faculty and students how results
student study groups, 80 of student evaluations will be used, 141
student teams, 79-80 not overestimating small differences, 142
students’ course notes, 80 using comparative data, 142
Formative evaluations, 1-2 using several sets of evaluation results, 142
Formative evidence about student learning, 1-2 using standardized procedures for
applying to departmental programs, 2 administering forms in class, 143
benefits of, 16-17 using student evaluation as only one piece of
coupling with opportunities for ongoing relevant information from several sources,
professional development, 1 141-142
effectiveness for summative evaluation, 17
supporting faculty wishing to explore the
scholarship of teaching and learning, 1-2
using for summative evaluation, 17 H
Forum on Faculty Roles and Rewards, 13n
Funding agencies
supporting programs to enable an integrated Hampshire College, 178-184
network of national and campus-based End-of-Semester Course Evaluation Forms,
centers for teaching and learning, 7-8, 126- 178-182
127 Instructor Objectives Report, 183-184
208 INDEX

Harvard University Derek Bok Center for grade distributions, course retention, and
Teaching and Learning, 163-165 subsequent enrollment figures, 66-67
End-of-Semester Course Evaluation Form, quality and performance of undergraduate
164-165 research students, 67
Mid-Course Evaluation Form, 163 Instructional contributions
How People Learn: Brain, Mind, Experience, and data for evaluating teaching quality and
School, 14 effectiveness from, 63
Howard Hughes Medical Institution, 48 Instructor Objectives Report, 183-184
Integrated learning, 33
Intellectual development of individual students
contributions to ongoing, 31
I Interdepartmental cooperation
in improving undergraduate STEM education,
114
Implementation of evaluation methodologies, 96- International Technology Education Association
99 (ITEA), 110
departmental panels on teaching effectiveness ITEA. See International Technology Education
and expectations, 98 Association
feedback from graduating seniors and alumni,
98
formative discussions between the
department chair and individual faculty J
members, 97
helpful policies and procedures, 96-97
legal considerations, 98-99 Just-in-time teaching, 79
oversight committee to monitor departmental
curriculum and instruction, 98
regular meetings between new faculty
members and the department chair, 97 K
sharing faculty-generated teaching portfolios,
97-98
Independent research Kansas State University IDEA Center, 166-177
encouraging students to engage in, 113-114 Faculty Information Form for Student
Informal conversations, 80 Evaluations, 168-169
Input from students and peers, 52-53 Sample Results of Student Evaluations, 170-
evidence of learning from student portfolios, 177
52 Student Reactions to Instruction and Courses,
faculty from “user” departments for service 166-167
courses and from related disciplines for Knowing What Students Know: The Science and
interdisciplinary courses, 52-53 Design of Educational Assessment, 15
informed opinions of other members of the Knowledge of subject matter, 27-28, 101
faculty member’s department, 52 answering students’ questions and guiding
summary of professional attainments of information searches, 28
undergraduate students engaging in data sources and forms of evaluation for
research under the faculty member being evaluating, 102
evaluated, 53 helping students learn and understand the
undergraduate and graduate students, 53 general principles of their discipline, 28
undergraduate and graduate teaching providing students with an overview of the
assistants, 53 whole domain of the discipline, 28
Institutional data and records, 66-67 staying current through an active research
data for evaluating teaching quality and program or through scholarly reading, 28
effectiveness from, 66-67
evidence about student learning from, 3-4
INDEX 209

L National Research Council (NRC), 1, 11, 15, 26, 35

National Science Board, 11
National Science Education Standards, 35, 37n
Learners National Science Foundation (NSF), 11, 29n
bringing different strategies, approaches, Assessment of Student Achievement in
patterns of abilities, and learning styles, 21 Undergraduate Education, 48
motivation to learn and sense of self affecting Centers for Learning and Teaching program,
what and how much is learned and how 127n
much effort is put into learning, 21-22 Course, Curriculum, and Laboratory
using what they already know to construct Improvement program, 54n
new understandings, 20 Shaping the Future, 108
Learning process. See also Facilitation of National Survey of Student Engagement: The
learning; Principles of learning College Student Report, 77
as multidimensional, integrated, and revealed National Teaching and Learning Forum, 81n
in performance over time, 33 NCME. See National Council on Measurement in
promoting active, 29 Education
viewing as a joint venture with the students, NCTM. See National Council of Teachers of
29 Mathematics
Legal considerations, 98-99 New faculty members
Limitations regular meetings with the department chair,
on faculty knowledge of research on effective 97
teaching, 39 New scholarship on teaching, 25
on the use of rating forms, 144 NRC. See National Research Council
NSF. See National Science Foundation
M
O
Making Teaching Community Property: A Menu
for Peer Collaboration and Peer Review, 86 Observation, 84-85
Master Faculty Program, 85 Outcomes assessment, 73-76
Mathematical Association of America, 46, 109 adjusting expected learning outcomes as
Mentoring appropriate, 73
of faculty by other faculty, 85 benefits of, 75-76
Mid-Course Evaluation Form, 163 determining when in a student’s education
Minute papers, 79 specific knowledge and skills should be
Miracosta Community College, 81n developed, 73
Multidimensional learning, 33 developing expected student learning
outcomes for an individual course of study,
73
incorporating specified learning outcomes in
N statements of objectives for courses, 73
scoring, 74-75
selecting appropriate assessment strategies to
National Center for Education Statistics, 35 test student learning of specified
National Center for Public Policy and Higher knowledge, 73
Education, 12 using to provide formative feedback to
National Council of Teachers of Mathematics individual students, 73
(NCTM), 46, 110 Outside evaluators, 81
National Council on Measurement in Education Oversight committee
(NCME), 55 to monitor departmental curriculum and
National Institute for Science Education, 30, 72, 76 instruction, 98
210 INDEX

P Preparation
adequacy of, 87
of future teachers, 32
Pedagogical content knowledge, 16n Preparing for Peer Evaluation, 95
Pedagogies and technologies Primary trait analysis
ability to recognize students not achieving to in scoring outcome assessments, 74-75
their fullest potential and assisting them in Principles of good practice for assessing student
their academic difficulties, 29 learning, 33-35
contextually appropriate, 29n educational values, 33
data sources and forms of evaluation for illuminating questions people really care
evaluating skill in and experience with, 103 about, 34
enabling teaching, 25 involving a larger set of conditions that
encouraging discussion and promoting active promote change, 34-35
learning strategies, 29 involving representatives from across the
organized and clear communication to educational community, 34
students of expectations for learning and meeting responsibilities to students and to the
academic achievement, 29 public, 35
persistently monitoring students’ progress ongoing, not episodic, 34
toward achieving learning goals, 29 paying attention to outcomes and equally to
skill, experience, and creativity with a range of the experiences leading to them, 33
appropriate, 28-30, 101-103 programs with clear, explicitly stated
viewing the learning process as a joint venture purposes, 33
with the students, 29 understanding learning as multidimensional,
Peer reviews of teaching integrated, and revealed in performance
including in valid summative assessments of over time, 33
teaching, 4-5, 119-120 Principles of learning, 20-22
providing both objective and subjective effect of learners’ motivation to learn and
assessment of a faculty member’s sense of self on what and how much is
commitment to quality teaching, 7, 125 learned and how much effort is put into
Pew Charitable Trust, 48 learning, 21-22
Pew Forum on Undergraduate Education, 147- effect of the practices and activities engaged
150 in while learning on what is learned, 22
Pew Forum on Undergraduate Learning, 77, 145 enhancement of learning through socially
Portfolio Clearinghouse, The, 65 supported interactions, 22
Portfolios. See Faculty teaching portfolios facilitation of learning through metacognitive
Predictions about undergraduate teaching, 25 strategies that identify, monitor, and
changes in evaluation and documentation of regulate cognitive practices, 21
teaching, 25 facilitation of learning with understanding
changing appearance of higher education when new and existing knowledge is
facilities, 25 structured around major concepts and
curriculum and program design becoming principles of the discipline, 20
inseparable from teaching and learning, 25 learners’ different strategies, approaches,
diversity seen as asset-based, 25 patterns of abilities, and learning styles
focus of teaching shifting away from content coming from their heredity and prior
transmission, 25 experiences, 21
nature and quality of assessment, 25 learners’ use of what they already know to
a new scholarship of teaching, 25 construct new understandings, 20
pedagogies students experienced prior to Professional interactions with students, 30-31,
college changing their expectations about 104-106
good teaching, 25 advising students experiencing problems with
teaching becoming more public than ever course material, 31
before, 25 contributing to the ongoing intellectual
technology enabling teaching, 25 development of individual students, 31
INDEX 211

data sources and forms of evaluation for Syracuse University’s Classroom Observation
evaluating professionalism with students Worksheet, 188-192
within and beyond the classroom, 106 University of Texas at Austin’s Checklist of
demonstrating respect for students as Teaching Skills, 193-195
individuals and respecting their privacy, 31 Questions people really care about
encouraging the free pursuit of learning and beginning with, 34
protecting students’ academic freedom, 31
meeting all classes and labs, posting and
keeping regular office hours, and holding
exams as scheduled, 31 R
upholding and modeling for students the best
in scholarly and ethical standards, 31
Professional organizations Recommendations for evaluating teaching
encouraging publication of peer-reviewed effectiveness, 4-8, 115-127
articles on evolving educational issues in accreditation agencies and boards should
STEM, 7, 127 revise policies to emphasize quality
increasing support for effective teaching, 46- undergraduate learning as a primary
49 criterion for program accreditation, 7, 127
offering opportunities to discuss campus-wide and disciplinary-focused centers
undergraduate education issues during for teaching and learning should be tasked
annual and regional meetings, 7, 127 with providing faculty with opportunities
Program design for ongoing professional development, 5-6,
becoming inseparable from teaching and 122-123
learning, 25 for deans, department chairs, and peer
Programs That Work, 47 evaluators, 6-7, 124-126
Project Kaleidoscope, 56 department heads should provide personnel
recommendations containing separate
ratings on teaching, research, and service,
7, 125
Q departments should contribute to campus-
wide awareness of the premium placed on
improved teaching, 6-7, 125
Quality and performance departments should periodically review a
of undergraduate research students, 67 departmental mission statement that
Quality of teaching and effective learning includes appropriate emphasis on teaching
ranking more highly in institutional priorities, and student learning, 6, 124
5, 122 departments should provide funds to faculty
Questionnaires used to evaluate undergraduate to enhance teaching skills and knowledge,
student learning, 19, 145-184 7, 125-126
Carnegie Mellon University Eberly Center for departments should support faculty moving to
Teaching Excellence, 153-162 greater emphasis on instruction or
The College Student Report 2001, 147-150 educational leadership, 7, 126
Hampshire College, 178-184 effective peer reviews of teaching should
Harvard University Derek Bok Center for provide both objective and subjective
Teaching and Learning, 163-165 assessment of a faculty member’s
Kansas State University IDEA Center, 166-177 commitment to quality teaching, 7, 125
Student Instructional Report II, 151-152 faculty should be encouraged to develop
Questions for conducting peer evaluations of curricula that transcend disciplinary
teaching, 19, 185-195 boundaries through a combination of
Suggested Form for Peer Review of incentives, 6, 124
Undergraduate Teaching Based on Dossier faculty should be supported in their obligation
Materials, 186-187 to improve their teaching skills through
212 INDEX

departmental and institutional evolving educational issues in STEM, 7,

reinforcement, 5, 121 127
faculty who have excelled in teaching should professional societies should offer
be publicly recognized and rewarded, 6, opportunities to discuss undergraduate
123-124 education issues during annual and
funding agencies and research sponsors regional meetings, 7, 127
should undertake a self-examination by quality teaching and effective learning should
convening expert panels to examine be ranked highly in institutional priorities,
agency policy regarding quality 5, 122
undergraduate teaching, 7, 127 scholarly activities focusing on improving
funding agencies should support programs to teaching and learning should be
enable an integrated network of national recognized and rewarded, 4, 118-119
and campus-based centers for teaching and teaching effectiveness should be judged by
learning, 7-8, 126-127 the quality and extent of student learning,
graduate school faculties should be required 4, 118
to show evidence they are effectively Reflective critiques, 87
mentoring their teaching assistants and Reliability
advising them about their duties to of undergraduate student evaluations, 55-56
undergraduate students, 6, 124 Research. See also Applications of research;
for granting and accrediting agencies, Cultures of research and teaching
research sponsors, and professional education through, 38
societies, 7-8, 126-127 Respect
individual faculty members should be for students as individuals and for their
expected to contribute to a balanced privacy, 31
program of undergraduate teaching, 6, 124- Responsiveness
125 to students’ concerns, 82
individual faculty should be rewarded for Role of colleagues in “formal” formative
consistent improving of learning by both evaluation, 85-86
major and nonmajor students, 5, 120-121 faculty mentoring faculty, 85
normal departmental professional formative evaluation by faculty colleagues
developmental activity should include from other institutions, 85-86
informing faculty about research findings projects of the American Association for
that can improve student learning, 7, 125 Higher Education (AAHE), 86
one or more senior university-level
administrators should be assigned
responsibility for encouraging faculty to
adopt effective means to improve S
instruction, 6, 123
only deans and department chairs willing to
emphasize student learning and to make Sample Results of Student Evaluations form, 170-
allocations of departmental resources in 177
support of teaching should be appointed, 6, Scholarly activities
124 focusing on improving teaching and learning,
overall, 4-5, 118-121 4, 118-119
peer reviews and teaching portfolios should Scholarly standards
be included in valid summative upholding and modeling for students the best
assessments of teaching, in addition to in, 31
student evaluations, 4-5, 119-120 Scholarship on teaching, 9-67. See also Evaluation
for presidents, overseeing boards, and of the scholarship on teaching
academic officers, 5-6, 122-124 according the same administrative and
professional societies should encourage collegial support as for other research and
publication of peer-reviewed articles on service endeavors, 2, 15-16, 116
INDEX 213

aligning the cultures of research and teaching Student evaluations, 54-60

in higher education, 18, 40-50 bias in, 58-60
application of, 18-19, 69-127 data for evaluating teaching quality and
characterizing and mobilizing effective effectiveness from, 54-60
undergraduate teaching, 18, 25-39 only one piece of relevant information from
dimensions of, 88 several sources, 141-142
evaluating of teaching in the STEM reliability of, 55-56
disciplines, 18, 51-67 validity of, 56-58
recent perspectives on undergraduate Student Instructional Report II, 151-152
teaching and learning, 11-24 Student Reactions to Instruction and Courses
Science, technology, engineering, and form, 166-167
mathematics (STEM disciplines) Student study groups, 80
interdepartmental cooperation for improving Students. See also Evidence of student learning;
undergraduate education in, 114 Formative evaluation by undergraduate
professional societies encouraging publication students
of peer-reviewed articles on evolving for classroom observation, 63-64
educational issues in, 7, 127 direct questioning of, 78-79
Scoring of outcome assessments engaging in original research, 38-39
using primary trait analysis, 74-75 engaging their interest in departmental
Self-assessment, 54, 64-66, 86-87, 90-91 curricular offerings, 111
before-and-after, 90-91 identifying those not achieving to their fullest
data for evaluating teaching quality and potential, 29
effectiveness from, 64-66 making clear how results of student
by reports on teaching activities and teaching evaluations will be used, 141
portfolios, 64-65 using course notes of, 80
teaching portfolios, 86-87, 90 using teams of, 79-80
videotaping, 90 Suggested Form for Peer Review of
Shaping the Future, 108 Undergraduate Teaching Based on Dossier
Shaping the Future of Undergraduate Earth Materials, 186-187
Science Education, 47 Summative evaluations, 1
SID. See Small Group Instruction Diagnosis Support
Sigma Xi, 47 for faculty wishing to explore the scholarship
Skills of teaching and learning, 1-2
determining when in a student’s education Syracuse University, 95
these should be developed, 73 Classroom Observation Worksheet, 188-192
Small Group Instruction Diagnosis (SGID), 81-82
Society for Industrial and Applied Mathematics,
46
Standardized procedures T
for administering forms in class, 143
STEM disciplines. See Science, technology,
engineering, and mathematics Teaching. See also Cultures of research and
Student advising, 31 teaching; Evaluation of teaching in STEM
Student evaluation instruments, 19, 139-144 disciplines
current students, 139-141 becoming more public than ever before, 25
end-of-course questionnaires, 139-140 focus shifting away from content
guidelines for the use of student evaluations, transmission, 25
141-144 Teaching assistants. See Graduate teaching
interviews, 140 assistants; Undergraduate teaching
measures of learning, 140-141 assistants
types of, 139-141
214 INDEX

Teaching effectiveness, 27-31, 111 overview of research on effective assessment

involvement with and contributing to one’s of student learning, 23-24
own profession in enhancing teaching and seven principles of learning, 20-22
learning, 31, 106-107 statement of task and guiding principles, 15-18
judging by the quality and extent of student Undergraduate teaching assistants
learning, 4, 118 evidence about student learning from, 3
knowledge of subject matter, 27-28, 101 University of California System
professional interactions with students within Faculty Code of Conduct Manual, 31
and beyond the classroom, 30-31, 104-106 University of Texas at Austin, 95
skill, experience, and creativity with a range of Checklist of Teaching Skills, 193-195
appropriate pedagogies and technologies, Preparing for Peer Evaluation, 95
28-30, 101-103 University of Washington, 82, 92
understanding of and skill in using U.S. Department of Education
appropriate assessment practices, 30, 103- Educational Resources Information Center,
104 54n
Teaching Improvement Forms
Discussion Courses, 160-162
Laboratory Courses, 157-159
Lecture Courses, 153-156 V
Teaching laboratories, 113-114
emphasizing the role and importance of, 113
encouraging students to engage in Validity
independent research, 113-114 of undergraduate student evaluations, 56-58
Teaching portfolios. See Faculty teaching Videotaping, 90
portfolios Virginia Polytechnic Institute and State
Technologies. See Pedagogies and technologies University, 75
U W
Undergraduate teaching and learning, 11-24. See Worcester Polytechnic Institute, 64n
also Effective undergraduate teaching
impetus for and challenges to change, 12-15
INDEX 215


The National Academies Press

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The National Academies Press

Uploaded by

Copyright:

Available Formats

THE NATIONAL ACADEMIES PRESS

This PDF is available at http://nap.edu/10024 SHARE

Evaluating and Improving Undergraduate Teaching in Science,

232 pages | 8.5 x 11 | HARDBACK

National Research Council 2003. Evaluating and Improving Undergraduate Teaching

– Access to free PDF downloads of thousands of scientiﬁc reports

Copyright © National Academy of Sciences. All rights reserved.

Committee on Recognizing, Evaluating, Rewarding, and Developing

Marye Anne Fox and Norman Hackerman, Editors

Center for Education

The National Academies Press

Copyright National Academy of Sciences. All rights reserved.

Library of Congress Cataloging-in-Publication Data

Evaluating and improving undergraduate teaching in science, technology,

Printed in the United States of America

Copyright 2003 by the National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distin-

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

COMMITTEE ON RECOGNIZING, EVALUATING, REWARDING, AND

MARYE ANNE FOX (Co-chair) North Carolina State University

JAY B. LABOV (Study Director)

Copyright National Academy of Sciences. All rights reserved.

COMMITTEE ON UNDERGRADUATE SCIENCE EDUCATION

MARYE ANNE FOX (Chair), North Carolina State University

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

another, and faculty evaluations at and achievements in the balance be-

Copyright National Academy of Sciences. All rights reserved.

science, mathematics, and engineering; It is the committee’s view that a well-

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

PART I WHAT IS KNOWN: PRINCIPLES, RESEARCH FINDINGS,

1. Recent Perspectives on Undergraduate Teaching and Learning 11

2. Characterizing and Mobilizing Effective Undergraduate Teaching 25

3. Aligning the Cultures of Research and Teaching in Higher Education 40

4. Evaluating Teaching in Science, Technology, Engineering, and

PART II APPLYING WHAT IS KNOWN: STRATEGIES FOR

6. Evaluation of Individual Faculty: Criteria and Benchmarks 100

7. Evaluation of Departmental Undergraduate Programs 108

Copyright National Academy of Sciences. All rights reserved.

A Selected Student Evaluation Instruments 139

B Samples of Questionnaires Used to Evaluate Undergraduate

C Examples of Questions for Conducting Peer Evaluations of Teaching 185

D Biographical Sketches of Committee Members 196

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

This report recommends a set of determine whether fair and objective

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

courses in the discipline and related graduate education is critical to the

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.

practice. Such centers should provide mentoring graduate teaching assistants

Copyright National Academy of Sciences. All rights reserved.

their own faculty about the suitability of dent learning.

Copyright National Academy of Sciences. All rights reserved.

Copyright National Academy of Sciences. All rights reserved.