INFORMATION TO USERS

This manuscript has been reproduced from the microfilm master. UMI
films the text directly from the original or copy submitted. Thus, some
thesis and dissertation copies are in typewriter face, while others may be
from any type of computer printer.

The quality of this reproduction is dependent upon the quality of the

copy submitted. Broken or indistinct print, colored or poor quality

W
illustrations and photographs, print bleedthrough, substandard margins,
and improper alignment can adversely affect reproduction.
IE
In the unlikely event that the author did not send UMI a complete
manuscript and there are missing pages, these will be noted. Also, if
EV
unauthorized copyright material had to be removed, a note will indicate
the deletion.

Oversize materials (e g., maps, drawings, charts) are reproduced by

PR

sectioning the original, beginning at the upper left-hand comer and

continuing from left to right in equal sections with small overlaps. Each
original is also photographed in one exposure and is included in reduced
form at the back of the book.

Photographs included in the original manuscript have been reproduced

xerographically in this copy. Higher quality 6” x 9” black and white
photographic prints are available for any photographs or illustrations
appearing in this copy for an additional charge. Contact UMI directly to
order.

UMI
A Bell & Howell Information Company
300 North Zeeb Road, Ann Arbor MI 48106-1346 USA
313/761-4700 800/521-0600
PR
EV
IE
W
 ESSENTIAL ASSESSMENT CRITERIA FOR
PHYSICAL EDUCATION TEACHER EDUCATION
PROGRAMS: A DELPHI STUDY

W
DISSERTATION

Submitted to the School of Physical Education

IE
of
EV
West Virginia University

In Partial Fulfillment of the Requirements for

PR

the Degree of Doctor of Education

by

J. Michael Jacobs

Morgantown

West Virginia

1996
 UMI Number: 9639726

Copyright 1996 by
Jacobs, James Michael

W
All rights reserved.

IE
EV
PR

UMI Microform 9639726

Copyright 1996, by UMI Company. All rights reserved.

This microform edition is protected against unauthorized

copying under Title 17, United States Code.

UMI
300 North Zeeb Road
Ann Arbor, MI 48103
 W
IE
EV

(£) COPYRIGHT 1996

PR
 il
ACKNOWLEDGEMENT

The researcher would like to thank Dr. Robert Wiegand, Dissertation

Chairman for helping him to realize “what he wanted to be when he grew
up”. Dr. Wiegand's support and help in this task were incredible. I would
also like to thank the other committee members: Dr. Carl Bahneman; Dr.
Richard Walls; Dr. W. Scott Bower; and Dr. Andrew Hawkins. Their
collective encouragement has meant a great deal. I wish to thank Dr. John

W
Costello, past Dean of the School of Professional studies at Shepherd
College, for his encouragement, help in changing my professional focus, and
IE
friendship.
Special thanks to Carol Straight, secretary and friend who
EV

remembered me on my return and has pulled for me all along the way.
I dedicate this project and my degree to Dr. Dean Pease, mentor,
PR

inspiration, confidant, and friend who was not able to be here to see this
completed.
Recognition is also extended to my mother, who constantly
encouraged me to reach my academic potential, and my father, who taught
me the meaning of a diligent work ethic.
Finally, I wish to thank my wife and best friend Jeanie who has
always wanted to see me finish this passage of my life.
 iii
TABLE OF CONTENTS

List of Tables v

Chapter 1

INTRODUCTION AND STATEMENT OF THE PROBLEM

Introduction 1
The Problem 5
Statementof the Problem 5
Scope of the Study 5
Basic Assumptions 7
Limitations 8

W
Definition of Terms 8
Significance of the Study 9

Chapter 2
IE
REVIEW OF THE LITERATURE
EV
Introduction 12
Assessment in Higher Education 13
Assessment Design Considerations 21
Levels of Assessment Application 27
PR

The Delphi Survey Method 53

The Delphi in Education 59
Summary 63

Chapter 3

PROCEDURES
Introduction 65
Selection of Subjects 65
Research Design 67
Administrative Procedures 70
Round I of the Survey 70
Round II of the Survey 72
Hypotheses 74
Data Analysis 77
Summary 78
 iv
Chapter 4

RESULTS AND DISCUSSION

Introduction 79
The Delphi Survey 79
Results ofthe Delphi Survey 79
Discussion of Delphi Survey Results 82
The Addendum Survey 99
Results of the Addendum Survey 101
Discussion of the Addendum Survey 105
Summary 108

Chapter 5

W
SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS
Introduction IE 112
Summary 112
Conclusions 115
Recommendations 115
EV

Bibliography 118

APPENDICES 13 2
PR

Appendix A Summary of Assessment Elements 133

Appendix B The Pilot Study 135
Appendix C ROUND I of the Study 147
Appendix D ROUND II of the Study 157
Appendix E ROUND I and ROUND II Tally Sheets 168
Appendix F Delphi Item Analysis Data Set Report 172
Appendix G Institutional Review Board Verification 199

ABSTRACT 203

APPROVAL PAGE 211

 v

LIST OF TABLES

TABLE

1 Essential Items in the Assessment of PETE

Programs as Determined by Delphi Survey 80

2 Results of Final Round Addendum 100

W
IE
EV
PR
 Chapter 1
INTRODUCTION AND STATEMENT OF THE PROBLEM
Introduction
All of education, and especially higher education, is operating in an
ever increasing public arena (Bryson & Eladmoe-Iinquest, 1982; Gill,
1993). In an era of increased accountability, cost containment, greater
efficiency, reductions in force, and high graduate placement expectation,
educational program assessment and evaluation is now a major topic. In

W
West Virginia, a state of decreasing financial resources, ten state supported
colleges, three medical schools, and two major universities, Governor
IE
Gaston Caperton recently ordered the governing boards of the higher
EV
education system to reform and restructure the complete scholastic system
(White, 1995) . This initiative was translated into five presidential
(college) task forces to study specific areas of concern. In each area,
PR

assessment and quality were main subtopics (Riccards, 1995). With a

population just over two million, and one of the nation's lowest rates of
post secondary education, assessment data from all institutions will no
doubt be used in the process of system review in West Virginia.
Furthermore, the life expectancy of each program may well be determined
by information gleaned from those reports.
Assessment is a relatively new process on most campuses, and
appears to be entering only its second decade as a national phenomenon
(Hudgins, 1993). As one reads the breath of the literature it is apparent

1
that the issue is indeed very current. Not only are individual states
involved in the evaluation of their higher education delivery systems,
surveys of current literature also reveal issues of assessment in education
as a global concern (Kells, 1992). State legislatures in Florida and Georgia
have mandated that students will be accountable on standardized
academic skills tests after their sophomore year and prior to entry to any
teacher education program (Mingle, 1985; McTamaghan, 1985). Texas
(Tabler,1988), Maryland (Holmes & Barbour, 1987), Washington (Walkup,

W
1991), and other states (North Carolina.. 1985, Tennessee.. 1993,
Utah.. 1991) have indicated a necessity for state and regional assessment.
IE
The South Carolina Higher Education assessment Network (Johnson, 1992),
EV
has even suggested a statewide assessment clearinghouse.
Because of these ever increasing concerns, the issue of undertaking
departmental assessment is not only justifiably current, but also appears
PR

urgent. With increased state intervention being planned, it seems

apparent that all institutions, regardless of size, must prepare to meet the
challenge. If individual programs of teacher preparation in physical
education are to survive, a knowledge of assessment practices coupled
with the ability to conduct a meaningful process/product evaluation must
be acquired. To satisfy this need, assessment experts have surfaced, and
workshops in this domain are beginning to find their way into professional
education convention formats as well as onto individual campuses (Nichols,
1991; Smith, 1993).

2
 Criteria being used among present assessment plans include:
student entrance requirements, standardized test scores, general studies
grade point average, major area grade point average, education course
grade point average, student portfolios, student goal setting plans, student
teaching practicum GPA, comprehensive exams, GRE, graduate
school/employment placements, and even post graduate continued
assessment practices. The North Central Association (NCA), North Central
Association of Teacher Education (NCATE), National Association of Sport

W
and Physical Education (NASPE), and many individual state higher
education associations have addressed the issue of institutional
IE
assessment. The 25 current NCATE guidelines for certification in the basic
(baccalaureate) physical education teacher education degree program have
EV

undergone a major renovation. These guidelines establish standards that

call for specified experiences related to the preparation of future teachers.
PR

In this way program assessment is viewed as indicative of the Physical

Education Teacher Education (PETE) curriculum's ability to develop the
novice teacher. The present 25 guidelines are more rigid and the
curriculum demands are more specific. Because of the revision and more
stringent expectations, only 80 of the more than 300 PETE institutions
now hold current certification in the basic level of teacher preparation
(NCATE, 1994). Additionally, as this study was being undertaken, even
present NCATE/NASPE certifying standards were being reworked into a
more manageable though more explicit document. Whether the 80 schools

3
currently certified are involved in a more in depth program assessment is
a question of the study. At present, the certifying agencies only evaluate
areas specifically satisfied by curriculum structures.
Under such pressures and concerns it now seems apparent that
some consensus of overall process/product assessment elements and
methods might benefit all physical education teacher education programs.
There is a dire need for all PETE institutions to meet the challenge of
assessment while at the same time maintaining focus that to do so is to

W
measure their effectiveness in producing a lasting, quality novice physical
educator. Simple gatherings of quantitative data may not alone answer the
IE
question of whether a truly accountable teacher has been produced. It is
now time to attempt to arrive at consensus as to what if any elements can
EV

measure an institution's ability to train teachers to meet the challenges of

future educational issues. By use of a Delphi survey among NCATE
PR

certified institutions this study attempted to arrive at assessment

measures that would determine program effectiveness.
With the compounding lack of training and resources, a more critical
concern might also be how institutions might embark on such an
undertaking in both an effective and efficient manner. For this reason the
study attempted also to arrive at a list of assessment elements that would
not only be important areas to measure, but also a list of elements that
would be feasible and practical to undertake. If assessment is to
contribute to program maintenance it must contain elements that

4
effectively will provide indicators of novice teacher preparation and
success.
The Problem
Statement of the Problem
The purpose of this study was to arrive at a consensus of
professional opinion regarding essential elements to be used in the
assessment of physical education teacher education programs.
Scope of the Study

W
The study was executed in three stages. In stage one, a pilot study
was undertaken to determine the content validity and objectivity of the
IE
eventual survey instrument to be used. In order to accomplish this, a list
of assessment elements was compiled from the literature by the
EV

researcher. Beneath each item was a short phrase to further explain the
context of that item. Each element was then placed at either entrance,
PR

matriculation, exit, or follow up level of assessment regarding the novice

teacher. Rating of each element was then posed as to both importance and
feasibility using a five point Likert scale. This document was then drafted
in survey form and sent as a pilot study to five experts in the field of
physical education teacher education (PETE) assessment. The pilot study
instrument and instructions are found in the Appendix A. Reaction to the
individual items of the document, rating scale, as well as the level
structure were solicited in this manner. The experts endorsed the
instrument, added one additional item (Item 1A), and agreed that the

5
document had content validity and objectivity.
In stage two, the refined document was then sent, with cover letter,
to institutions identified as NCATE certified, preparing teachers in the basic
(baccalaureate) level program. These institutions were chosen as the study
group due to their demonstrated commitment of adhering to the high
national standards of PETE program certification presently in place and
their commitment to continued program evaluation. The present 25
NCATE guidelines encompass specified teacher preparation outcomes that

W
must be addressed by any PETE K-12 baccalaureate preparation institution
seeking certification. The guidelines were prepared by curriculum scholars
IE
and have been accepted by NCATE as being reflective of those skills and
knowledge structures a novice physical educator should possess.
EV

Within a Delphi study context, the survey instrument was completed

and returned, and all Likert scale ratings, by individual item, were
PR

analyzed. This constituted the end of Round I of the sampling process.

In the final part of this stage of the study, a second round of rating
the survey items was completed. In order to enhance consensus, the
Round II document also contained that rater's previous scoring as well as
the group average ratings for each item. Upon return of the Round II
document, a repeated analysis of ratings was conducted. This constituted
the end of Round II. Analysis of the data indicated that after Round I
consensus was very close. Following Round II the major difference
betweer the two rounds was only a continued shrinkage in rating variance.

6
At that point the survey portion of the study was concluded.
In analyzing the final returns, any items with MEAN ratings of 3 or
lower in either importance or feasibility were deemed non-essential. Final
consensus as to essential items was determined by those items rated as 4
or higher in both importance and feasibility based on the accepted Likert
scale. Additionally, as an added measure of consensus, any item deemed
as essential by the mean rating method also had to contain seventy-five
percent of all ratings attained in the last round at the four or higher level

W
on the Likert scale. Tally sheets of all ratings for both rounds were then
placed in the Appendix E. IE
As third stage of the study, and an added measure of
consensus, the expert panel was also asked to list the three best items of
EV

PETE assessment currently at use in their institution and to give the

rationale for selection them. This survey was titled the Addendum survey.
PR

These items were then compared to the items resulting from the Delphi
survey as a means of comparing current practice items and consensus
determined essential PETE assessment items.
Basic Assumptions
The following were basic assumptions within this study:
1) The pilot study participants, by virtue of attained assessment
coordinator position in their respective institutions were
considered experts.
2) The Delphi procedure was considered a recognized method of

7
 arriving at rater consensus.
3) The number of NCATE accredited schools were deemed sufficient
to generate results to the PETE community.
4) Consensus by definition would occur within two rounds of the
Delphi process.
Limitation of the Study
The following was the basic limitation of the study:
1) A sufficient number of raters would not remain in the study to its

W
conclusion, and thus due to experimental mortality, internal validity
might cause the study to be prematurely terminated.
IE
Definition of Terms
The following terms were selected because they are operative terms
EV

in this study:
1) Assessment - The practice of process/product evaluation
PR

carried out within an institution of higher education for the

purpose of determining program effectiveness related to
meeting stated objectives in the preparation of a professional
novice physical educator. This process includes evaluation
activity usually undertaken as a means of maintaining
continued program effectiveness.
2) Expert - A person employed in the field of physical education
teacher education at the post secondary level and who, by
virtue of either administrative position or attained publication

8
 status in the field, is capable of discerning inclusive elements
to be used in an assessment study.
4) Consensus - For the purpose of this study, consensus will be
defined as a mean score of four or above, on a five-point Likert
scale, for that assessment item as determined from all raters
scores. Further support for agreement will be that over 75% of
the experts score that item at the four or five value (keep)
or one or two (drop) on the Likert scale.

W
5) Essential - The attainment of the consensus standard in both
the important and feasibility areas of the study at the end of
IE
the final round of the study.
6) Sample Mortality - For purpose of this study it was specified,
EV

that for the study to provide significant results, 80% of those

raters that completed the initial round of the study must also
PR

complete the second round.

Significance of the Study
The study attempted to determine, through the process of Delphi
determined consensus, the essential elements of process/ product
assessment to be considered by institutions involved in the preparation of
physical educators.
Due to the increased pressures being exerted by both the state and
national legislatures and the public at large, it is imperative that any
institution involved in higher education, and teacher education in

9
particular, monitor its program. In a time when all of education is being
held accountable for both expenditures and finished product, all teacher
preparation institutions must assume a more proactive campaign in the
oversight of its delivery system. If such a posture is assumed it is
believed that the resulting assessment data collected by such an
undertaking may be disseminated to the various media as a means of
gamering much needed public support and trust. In the area of physical
education teacher education, this measure is especially vital. As the

W
country undergoes constant financial constriction, program merit must be
continually substantiated.
IE
It is the authors opinion, substantiated by 25 years of PETE
experience, that within our schools, K-12 physical education is constantly
EV

losing ground as a required area of instruction. Both because of a public

misconception of physical education as a recess/recreation period and an
PR

administrative lack of depth regarding curricular understanding, physical

educators must deliver an accountable program of study (Haberman,
1985). More importantly, the novice teacher must also have attained a
sufficient level of commitment to withstand inherent negative socialization
in the work place. Under such conditions, all those involved in the
preparation of teachers must regularly be able to measure and document
the degree of success in preparing a quality novice practitioner in physical
education. Those that are progressive are already engaged in assessment
practices. Those soon to begin will fashion their own methods of

10
process/product evaluation. It is felt that the results of this study have
outlined a set of consensually arrived at important and feasible assessment
elements.
In closing this chapter, the researcher believes that the timeliness
of this concept and the present lack of assessment item consensus
necessitated that such a study be undertaken. It was further felt that
what was gained by this study will have a significant benefit in the
consideration of future assessment stratagies.

W
IE
EV
PR

11
 Chapter 2
REVIEW OF THE LITERATURE
Introduction
The purpose of this chapter was to present the literature related to
the study. To accomplish this task the chapter was divided into the
following sections: 1) Assessment in Higher Education, 2) Assessment
Design Considerations, 3) Levels of Assessment Application, 4) The Delphi
Survey Method, and 5) The Delphi in Education.

W
With the writing of this chapter the researcher has attempted to
demonstrate that while the area of assessment is somewhat new in the
IE
field of education, no attempt has ever been undertaken to determine
EV
what specific items should be included in an assessment plan designed to
evaluate a Physical Education Teacher Education (PETE) program. In
evaluating a teacher education program on a yearly basis, it is also the
PR

belief of the researcher that any such program must be as complete as

feasibly possible given the constraints of either financial or personnel
resources.
This review of the literature has further shown the breath of
assessment design issues and the pool of possible assessment elements.
The vastness of the referenced pool of items further reinforces the need to
determine, essential assessment elements as defined in the study.
The chapter has also lent credibility to the Delphi technique as an
educationally endorsed method of gaining consensual agreement in the

12
search for the resulting essential assessment elements.
Assessment in Higher Education
As was stated in chapter one in this study, assessment is
operationally defined for the purposes of this study as the practice of
process/product evaluation carried out within an institution of higher
education for the purpose of determining program effectiveness related to
meeting stated objectives in the preparation of a professional physical
educator. The assessment process includes evaluation activity over and

W
above the expectations conveyed in any national accreditation process, and
is usually undertaken as a means of continued program improvement. In
IE
addition, Gingham & Clark ( 1994) have stated that any assessment effort
EV
should be comprehensive and part of a school wide process that includes
all students and all programs. Additionally, the entire process was stated
as something that should evolve gradually, as data are collected over time,
PR

and should involve all faculty personnel as well as the immediate

community. As was reported by Dunham (1986), the purpose of the
education program assessment process should be to provide the basis for
determining the appropriateness of the curriculum, the effectiveness of
instructional strategies, and the magnitude of student achievement as a
result of interacting with the curriculum.
In recent years the concept of assessment has become a political
issue within higher education. As state governments continued to become
more involved in the mission of higher education, student outcomes

13
assessment have played a more pronounced part in an institution’s defense
regarding program accountability (Aver, 1993). In light of high school
students graduating without basic skills and colleges having to remediate
entering freshmen, the public has adopted an uncertain viewpoint
regarding the total educational system. Coupled to the public’s perception
of ineffectiveness in the secondary school system, colleges have been
seen in a similar light and the whole evaluation movement appears to have
been created by the misconception that all levels of education share the

W
same mission. These issues have precipitated a civic call for increased
accountability, regardless of the differences between educational level, or
IE
purpose (Hudgins, 1993). As many politicians have campaigned on the
promise of improved education, the call for assessment has become the
EV

insistence that schools and colleges maintain at least minimum standards

and demonstrate that students have functionally been educated (Edgerton,
PR

1986).
The theme of 1982 NCA workshops was the strengthening and
improvement of all programs of teacher education (Jones, 1983). During
that convocation of experts a recurring theme was the need for assessment
as a means of program improvement. In a search of the roots of the
teacher education assessment movement, earliest accounts appeared to be
tied to the 1970 Ryan Act in California that ordered the assessment of all
state teacher education programs. This one legislative step also appeared
to have been the impetus behind the certification standards later imposed

14
 by NCATE (Jones and LoPresti, 1982). The present guidelines for
achieving NCATE certification in physical education teacher education are
now more stringent and demanding, and have only confirmed 80 PETE
programs as having met those present criteria (NCACS, 1994). While these
guidelines attempt to structure curriculum toward novice teacher
improvement, none of the standards reflect assessment as a means to this
end.
Hudgens (1993) has reported that the education program

W
assessment movement appears to be in its second decade, and has been
active, under state mandate, in 40 states, hi reporting on a 1991 national
IE
survey, Hudgens further concluded that 90 percent of those institutions
EV
that responded reported some level of assessment that was either under
way or being considered, and that the majority of those who returned the
survey described only minimum efforts to date. Additionally as reported
PR

by Pellow and Kuhns (1992) concerning a 1988 study, 50 percent of those

institutions involved in assessment at that time were doing so under state
mandate. These last references indicate not only higher education's
reluctance to engage in assessment, but also the trend to move toward
assessment only when forced to.
In the state of Virginia, Miller (1993) reported that assessment
across the entire campus was needed to insure the public that higher
education was accountable. In 1990, the New Mexico Commission on
Higher Education spelled out its legislative agenda for higher education

15

Reproduced with permission of copyright owner. Further reproduction prohibited without permission.