Professional Documents
Culture Documents
Jackie Carpenter, Lea R. Dopson, Young Hoon Kim & Nancy Kniatt
To cite this article: Jackie Carpenter, Lea R. Dopson, Young Hoon Kim & Nancy Kniatt
(2016) Curriculum assessment through a capstone course: a case study in hospitality
and tourism programs, Journal of Teaching in Travel & Tourism, 16:1, 40-59, DOI:
10.1080/15313220.2015.1115383
Article views: 3
Download by: [Library Services City University London] Date: 24 March 2016, At: 02:37
JOURNAL OF TEACHING IN TRAVEL & TOURISM, 2016
VOL. 16, NO. 1, 40–59
http://dx.doi.org/10.1080/15313220.2015.1115383
1. Introduction
Hospitality educators have questioned whether hospitality management programs are
adequately preparing students for the hospitality industry (Kay & Russette, 2000). In
seeking to answer this question, assessment has been emphasized as a way to assist
hospitality educators in determining the students’ ability to acquire the desired char-
acteristics and competencies identified as essential for success. Assessment is important
for several reasons: (1) to measure learning achievement against specific learning goals
and to provide feedback for individual students, (2) to assist educational institutions
and faculty members in enhancing, evolving, and improving their students’ learning
experiences, and (3) to fulfill the essential requirement for hospitality and tourism
educators to complete an assessment in order to receive accreditation (Daigle, Hayes, &
Hughes, 2007).
Setting clear, measurable, and obtainable goals for each course in addition to student
learning outcomes (SLOs) creates a significant path for hospitality educators and stu-
dents to pursue (Kay & Russette, 2000). One way of measuring the attainment of these
SLOs is through a capstone course that is designed to assess achievement of program-
matic student learning outcomes. A capstone course can be designed to be a culminat-
ing experience for students, or as an opportunity to utilize all knowledge acquired
throughout the program to analyze problems and develop creative solutions based on
that knowledge. Typically, educational institutions generate capstone courses as an
opportunity for assessment. However, it has been a challenge to develop a standardized
evaluation method. Thus, this study is designed to explain how to measure competency
and review existing curriculum, develop and refine a capstone course, and design an
assessment blueprint to be used to evaluate the program’s effectiveness through a
capstone course.
2. Research foundations
Downloaded by [Library Services City University London] at 02:37 24 March 2016
students to understand the skills they are learning and present them with a roadmap of
where learning opportunities exist within their department (University of Rhode Island,
2009). Typically, a program of study will contain five to ten general outcomes with more
clearly defined objectives designated for each individual course. Outcomes and objec-
tives should address the knowledge, skills, or abilities that are expected to be obtained,
and they should be stated in a way that can be measured or assessed (Murray et al.,
2008).
Assessment encourages education to be focused on student-based learning rather
than on teaching-based instruction. It enables educators to ensure that courses or
programs are meaningful and useful in both academic and practical perspectives.
Assessment is a continuous process and must be thought of in this way (Peach,
Downloaded by [Library Services City University London] at 02:37 24 March 2016
with a program. Students base responses on their level of confidence in identifying and
solving problems in these specific areas. Results of these indirect assessments are
immediate; however, the surveys are time-consuming and intense for students to
complete (2008).
On the other hand, direct assessment requires that students demonstrate mastery of
content by evaluating the students’ actual work (Price & Randall, 2008). Direct assess-
ment can be achieved through student papers, presentations, case studies, or pre-tests
and post-tests. The most commonly used form of direct assessment is pre-testing and
post-testing because it easily identifies the difference in student learning over the course
of the semester. The results of this form are immediate as well, but time constraints limit
the amount of information that can be tested. This form of direct assessment allows
Downloaded by [Library Services City University London] at 02:37 24 March 2016
educators to incorporate SLOs established for the curriculum (2008). By utilizing direct
assessment methods, educators can determine the achievements of students by using
course-embedded assessment. Course-embedded assessment requires that students’
work be evaluated by using the specific objectives established for the course (2008). It
allows for everyday classroom activities to provide feedback for faculty on ways the
course can be enhanced or improved. This efficient and effective method ensures
consistency among several sections of the same course (Gerretson & Golson, 2005;
Price & Randall, 2008).
(1) The main purpose of assessment is to enhance and improve the departmental
curriculum.
(2) Assessment of student learning involves faculty, staff, and students.
(3) Assessment is guided by the institution’s mission.
(4) Assessment of student learning should not be used for faculty or staff evaluations.
(5) Assessment is an ongoing and continuous process.
(6) The assessment process is evaluated periodically.
4). This endless process often challenges educational institutions due to the amount of
effort and time required. It is recommended that assessment should bridge the gap
between perception and reality by providing students with the preparation they warrant
for a successful career in the industry (Stivers & Phillips, 2009).
resourceful solutions. The capstone course should be “an in-depth opportunity for the
student to demonstrate accomplishment of the full spectrum of that learning” (Moore,
2009, p. 2). Therefore, it is essential that the capstone course include a wide and
balanced variety of expectations. The student should be given the opportunity to utilize
existing knowledge to develop a solution to new analytical problems established in
the course. Capstone courses are intended to integrate critical thinking throughout the
course, rather than only technical skills (2009).
One of the main purposes of a capstone course is to prepare students for the profes-
sional world that they are about to enter. For this reason, educational institutions usually
ensure this course is mandatory in order for students to graduate. Typically, capstone
courses are taken by students in the final semester of their educational experience; this
course acts as a transition class from undergraduate student to industry professional.
According to Henscheid (2000), an important goal for capstone courses is to provide
students with career preparation and pre-professional development. Career preparation is
especially important with the increased competition in today’s workforce and the rapid
technological advancements that occur on a constant basis. Graduates are expected to be
hospitality managers upon completion of their degree. Often, students do not feel ready
for this type of intense position immediately following graduation. Capstone courses are
intended to help to bridge the gap between school and industry (Moore, 2009).
The capstone course serves as a collaboration of all previous skills and knowledge
that students have obtained with the proficiency and critical thinking abilities required
by professionals in the industry. According to Moore (2009), students are expected to
display a mastery of learning and demonstrate the ability to apply it to new and
integrated information. The capstone course serves as a summative evaluation of the
students’ experience at an educational institution and provides an opportunity to utilize
pre-existing knowledge to develop relationships between individual courses in prepara-
tion for their career (2009).
Universities have struggled to provide an opportunity for students to connect all of
the knowledge acquired in individual courses prior to entering the workforce (Jervis &
Hartley, 2005). Today, educational institutions are attempting to tie all of students’
educational experiences together through capstone courses; they are recognizing that
universities should be accountable for clearly stating and explaining their expected
standards for student learning, as well as assessing these objectives to determine the
level at which they have been met (Moore, 2009).
JOURNAL OF TEACHING IN TRAVEL & TOURISM 45
can be accurately and adequately assessed. Capstone courses are often identified as
such, but they may or may not be designed to be culminating experiences. They may be
designed for students to use knowledge acquired through the program, but rarely are
capstone courses designed to build on this knowledge to develop solutions to complex
problems or situations using critical thinking skills. The “tying in” of individual compe-
tencies and content domains regarding their relationships to one another is the critical
component often left out of capstone course design. Whether this feature exists deter-
mines whether the course can be defined as a true culminating experience. Adequately
designed culminating capstone courses may be used to assess the entire curricula.
courses, discussion may be limited. As a result, fewer in-class activities may be assigned
and more lecture-based teaching methods may be utilized. In larger classes, instructors
may also give fewer assignments, projects, or case studies to keep the workload
manageable. Smaller classes allow for more open discussion and the ability for most
students to contribute to the conversation. The ideal number of students for a capstone
course, according to Jervis and Hartley (2005), is no less than 20 and no more than 30
students in each class because this number encourages class discussion and promotes
the manageability of the course for the instructor.
It is essential to have the entire faculty’s “buy-in” when developing a capstone course
(Dopson & Tas, 2004; Peach et al., 2007). The support and ownership of the department’s
faculty is critical in the success of the capstone course. All courses leading up to the
Downloaded by [Library Services City University London] at 02:37 24 March 2016
● Step1: Develop program competencies for the entire curriculum based on information
gathered from the literature review, industry, students, alumni, and faculty. What do
students need to know when they graduate from the program (Dopson & Tas, 2004)?
At the case study institution, hospitality management faculty reviewed competency literature to
develop a list of competencies for an undergraduate hospitality management program. These
competencies were designed to address the Accreditation Commission for Programs in
Hospitality Administration (ACPHA) curriculum standards for the Hospitality Academic
Experience and the Field Experience (ACPHA Accreditation Commission for Programs in
Hospitality Administration, 2013). The competencies were intended to complete the phrase,
“To allow graduates to succeed both short-term and long-term in the hospitality industry,
hospitality curricula should prepare graduates with the ability to. . .”
The faculty submitted this list of competencies to professionals in the hospitality industry,
hospitality management students, and hospitality management faculty. All three groups
added, deleted, or changed the competencies based on their perspectives. A previous study
of alumni satisfaction with the hospitality management program also provided insights as
48 J. CARPENTER ET AL.
to alumni perception of the existing curriculum. The result was a list of 211 competencies for
the undergraduate hospitality management curriculum.
● Step 2: Using the entire faculty, develop a competency grid (crosswalk analysis),
which evaluates and compares needed competencies for all courses in the existing
curriculum, using Bloom’s Taxonomy (Dopson & Tas, 2004).
Faculty members at the case study institution matched competencies on the list to the
content in courses they were teaching. In addition to identifying the competencies, they
determined the level of Bloom’s Taxonomy to which each competency was taught. They then
developed a competency grid that matched courses with competencies.
Downloaded by [Library Services City University London] at 02:37 24 March 2016
● Step 3: Based on the comparison achieved in Step 2, faculty should add, delete,
and/or change curriculum content (Dopson & Tas, 2004).
The faculty reviewed the competency grid and discussed each competency and its coverage
in the curriculum. Competencies were identified that were determined to have deficient
coverage in the curriculum. Also, faculty discussed possible subjects that might cover
deficient content, and content that was unnecessarily redundant was revised or removed.
At the case study institution, all hospitality management faculty members analyzed 211
competencies and assigned appropriate levels of Bloom’s Taxonomy for the entire curricu-
lum. To categorize these competencies, the faculty had identified 12 content domains:
accounting, cost controls, marketing, human resources, information technology, basic and
service management, personal attributes and leadership, going green, global issues, strategic
management, legal issues and hospitality law, and tourism. Then, the researchers took the
JOURNAL OF TEACHING IN TRAVEL & TOURISM 49
12 domains and reduced them to nine domains that were relevant to the capstone course.
For example, accounting and cost controls were combined into one category, which con-
tained all accounting concepts, called accounting, finance, and cost controls. Basic and
service management and strategic management were merged into one category and titled
strategic management. Personal attributes and leadership were narrowed to simply lea-
dership, and global issues and tourism were combined into globalization. Going green was
changed to sustainability. Legal issues and hospitality law weres reduced to legal issues,
and human resources, marketing, and information technology all stayed the same, thus
resulting in the final nine content domains.
In order to establish dependability of the domains, the researchers polled four hospitality
management full-time faculty members for their expert judgment. Many researchers have
developed their own measures for validity and reliability in their qualitative research, seeking
Downloaded by [Library Services City University London] at 02:37 24 March 2016
out verification such as dependability and trustworthiness (Golafshani, 2003; Lincoln &
Guba, 1985; Stenbacka, 2001). Faculty members agreed that the nine content domains
were accurate and representative of the information being covered in the capstone course.
All four professors approved the list and requested the study continue.
ties exist upon completion of the survey, the department’s faculty, as a whole,
should break the tie. Ultimately, it is the faculty’s responsibility to determine the
curriculum (Stevenson, 2010).
Approximately one week after completion of the focus group, a survey was e-mailed to all full-
time faculty members, as well as the entire hospitality management advisory board, including
those who had not attended the board meeting. The survey was created using the Survey
Monkey software program and asked respondents to select one of the scenarios established by
the two focus groups. Respondents were asked to select the scenario based on the weighted
percentages they felt were most accurate. Respondents were also given the option to create
their own scenario, if they felt the two scenarios were not representative of their opinions. The
link to the survey was emailed to the sample with an informed consent document explaining
Downloaded by [Library Services City University London] at 02:37 24 March 2016
that the responses would be kept confidential and that there were no foreseeable risks in
participating in the survey. The results provided 50 usable respondent surveys.
● Step 7: Use the original crosswalk of the entire program competencies, and pull out
competencies specific to a capstone experience (Stevenson, 2010).
The faculty completed a crosswalk analysis of the capstone competencies compared to the
learning objectives of the identified course using the Revised Bloom’s Taxonomy. This cross-
walk identified the levels of Bloom’s currently being taught in the class, which may or may
not match the faculty’s expectations of the Bloom’s levels that should be taught in the
course. Faculty identified the gaps between the competency levels expected by the hospital-
ity management faculty and the levels currently being taught in the class. Faculty made the
necessary changes to the course to obtain the desired competency levels based on the results
of the gap analysis. (See Table 1 for capstone crosswalk example.)
Downloaded by [Library Services City University London] at 02:37 24 March 2016
examination should be accentuated more or less than other portions. The test blueprint
represents a relationship between the topics or content domains and their importance
for a professional in the industry. The examination created from a test blueprint would
presumably determine whether an individual who took the exam would be prepared or
unprepared for a career in the industry (Spray & Huang, 2000).
The test blueprint also shows the breadth and depth of the questions that are asked.
The test blueprint depicts each competency and the associated levels of Revised Bloom’s
Taxonomy, which each competency should reflect. There are numerous designs and
templates for test blueprints, and the weighted values can be collected in multiple ways.
Interviews, focus groups, frequency scales, consequence scales, and the Rasch Rating
Scale Model are commonly used methods of determining weighted importance for
content domains (Spray & Huang, 2000).
● Step 10: Develop a test blueprint to design the assessment instrument using the
competencies identified in the course crosswalk analysis and domain weights
identified in the focus groups and the survey (Stevenson, 2010).
At the case study institution, the faculty wanted to be able to generalize the results of the
capstone assessment to the attainment of the theoretical constructs delivered throughout
the entire curriculum (SLOs). Faculty determined that the objective question multiple-choice
testing format should be used to assess achievement of these competencies in the capstone
course. A test blueprint was developed to design the assessment instrument using the
competencies and domain weights identified in the focus groups and the survey.
aspects of the industry fell within these nine domains. For example, the subject of sales
was included under marketing in one focus group, but it was not discussed in the other
focus group. How each group defined the terms may have been a contributing factor to
the differences between the two focus groups’ scenarios.
Focus-group participants agreed that all nine categories were important; however,
specific content domains were determined to have much less weight than others.
Sustainability and globalization were given the least amount of weight in both groups.
Focus-group participants felt these concepts were too new to hold a significant portion
of the weight. Participants agreed they were important, but not as important as other
content areas.
In addition, industry professionals struggled with identifying the type of position a
Downloaded by [Library Services City University London] at 02:37 24 March 2016
graduate of the program would mostly likely obtain. Several focus-group participants also
expressed frustration in determining specifically which segment of the industry graduates
would enter. Many commented that weighting was dependent on the segment of the
hospitality industry students chose. Focus-group participants were also concerned with
the level of management for which graduates were to be prepared, whether it was entry-
level management or general manager/chief executive officer. Participants commented
that at a higher level of management graduates may have specialists in some of the
domain areas within their supervision, such as human resources. Therefore, they would
require a more general understanding of the topic rather than a detailed knowledge,
which would impact the weight given to that content domain.
Leadership was a controversial subject in the focus-group discussions. Some industry
professionals questioned how leadership could be taught in a college course. Some
participants felt management should be more heavily weighted than leadership and
others felt leadership should be the most important area and, consequently, should
have the most weight. A common thread found in most of the leadership discussion was
determining how important leadership skills were in the lower levels of management.
Since some participants viewed leadership as not as easily or as clearly defined as some
of the other content domains (accounting, for example), participants formed their own
definition of leadership and argued their point of view depending on their personal
opinion. The resulting domain weighting scenarios developed by the two focus groups
are illustrated in Table 2.
One of the reasons the industry participants may have struggled with this entire
exercise is that they may have been too far removed from entry-level management
positions to easily determine the weighting of significance for each domain. The
industry professionals participating in this study all held positions such as chief execu-
tive officer, general manager, director, vice-president, or partner.
Many of the industry participants had not worked in entry-level management posi-
tions for several years. This may have played a considerable role in how the content
domains were weighted and interpreted by industry participants. Perhaps industry
participants weighted the content domains according to their current positions, or
perhaps participants estimated how much weight these domains should have according
to entry-level positions within their individual organizations. Table 3 contains a complete
Downloaded by [Library Services City University London] at 02:37 24 March 2016
content. The number of questions was then distributed among the competencies based
on this varying complexity. The other eight domains were handled in a similar manner.
The researchers utilized the competency verbs based on levels 4, 5, and 6 of the Revised
Bloom’s Taxonomy as a guide to determine the taxonomic level at which each competency
should be tested. Once an example test blueprint was developed, the instructor of the
capstone course reviewed each revised competency and the associated level by which it
would be assessed to identify any gaps or inconsistencies. Minor adjustments were made
and the instructor and chair of the department finalized the test blueprint.
The resulting test blueprint was designed to provide faculty with a consistent
measurement tool to assess each competency and content area based on a taxonomic
level. This test blueprint was designed for an exam containing 100 questions. The
Downloaded by [Library Services City University London] at 02:37 24 March 2016
department faculty then reviewed the document and came to a consensus on its ability
to effectively assess the theoretical constructs in the undergraduate hospitality manage-
ment curriculum. Finally, the researchers recommended that faculty develop three
different but equivalent versions of the test, resulting in a test bank containing 300
questions. This would enable the faculty to change the exam or to develop several forms
of the exam for each semester. (See Table 5 for the resulting test blueprint.)
Table 5. (Continued).
Analyzing Evaluating Creating Number
Weighted (Bloom’s (Bloom’s (Bloom’s of
Content domains and competencies percentage Level 4) Level 5) Level 6) questions
3. B. Develop a marketing plan for a hospitality 4
property
3. C. Examine the “branding concept” in the 4
hospitality industry
3. D. Utilize the principles of internal marketing 3
(employees) in the workplace
Domain 4: Human Resources: 10% 10
4. Examine the strategic human resources
business planning process, including:
a. Standards for labor productivity 2
Downloaded by [Library Services City University London] at 02:37 24 March 2016
Disclosure statement
No potential conflict of interest was reported by the authors.
References
ACPHA Accreditation Commission for Programs in Hospitality Administration. (2013). Document
Library: ACPHA Self-Study Guide, p. 15–16. Retrieved July 26, 2013, from http://www.acpha-cahm.
org/document-library/
Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., &
Wittrock, M. C. (2001). A taxonomy for learning, teaching, and assessing: A revision of bloom’s
taxonomy of educational objectives, abridged edition. White Plains, NY: Longman.
Apostolou, B. A. (1999). Outcomes assessment. Issues in Accounting Education, 14(1), 177–197.
Betts, S. (2008). Teaching and assessing basic concepts to advanced applications: Using Bloom’s
taxonomy to inform graduate course design. Academy of Educational Leadership Journal, 12(3),
99–103.
Bloom, B., Englehart, M., Furst, E., Hill, W., & Krathwohl, D. (1956). Taxonomy of educational objectives:
The classification of educational goals. Handbook I: Cognitive domain. New York, NY: Longmans.
Carpenter, S. (1999). Choosing appropriate consensus building techniques and strategies. In L.
Susskind, S. Mckearnan, & J. Thomas-Larmer (Eds.), The consensus building handbook: A compre-
hensive guide to reaching agreement (pp. 61–97). Thousand Oaks, CA: Sage Publications.
JOURNAL OF TEACHING IN TRAVEL & TOURISM 59
medium sized public university’s general education program. The Journal of General
Education, 54(2), 139–149.
Golafshani, N. (2003). Understanding reliability and validity in qualitative research. Qualitative
Report, 8(4), 597–607.
Harwood, E. M., & Cohen, J. R. (1999). Classroom assessment: Educational and research opportu-
nities. Issues in Accounting Education, 14(4), 691–724.
Henscheid, J. (2000). Professing the disciplines: An analysis of senior seminars and capstone courses.
Columbia, SC: University of South Carolina.
Jervis, K. J., & Hartley, C. A. (2005). Learning to design and teach an accounting capstone. Issues in
Accounting Education, 20(4), 311–339.
Kay, C., & Russette, J. (2000). Hospitality-management competencies: Identifying managers’ essen-
tial skills. Cornell Hotel and Restaurant Administration Quarterly, 41(2), 52–63.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212–218.
Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Journal of Sociology, 23, 481–483.
Moore, R. C., (2009). Capstone courses. Retrieved June 30, 2014, from http://users.etown.edu/m/
moorerc/capstone.html
Murray, M., Pérez, J., & Guimaraes, M. (2008). A model for using a capstone experience as one
method of assessment of an information systems degree program. Journal of Information
Systems Education, 19(2), 197–208.
O’Halloran, R. M., & O’Halloran, C. S. (1999). Comprehensive assessment in the hospitality and
tourism classroom. Journal of Hospitality & Tourism Education, 11(2–3), 31–33.
Okeiyi, E., Finley, D., & Postel, R. (1994). Food and beverage management competencies: Educator,
industry, and student perspectives. Hospitality and Tourism Educator, 6(4), 37–40.
Peach, B. E., Mukherjee, A., & Hornyak, M. (2007). Assessing critical thinking: A college’s journey
and lessons learned. Journal of Education for Business, 82(6), 313–320.
Price, B. A., & Randall, C. H. (2008). Assessing learning outcomes in quantitative courses: Using
embedded questions for direct assessment. Journal of Education for Business, 83(5), 288–294.
Spray, J., & Huang, C. (2000). Obtaining test blueprint weights from job analysis surveys. Journal of
Educational Measurement, 37(3), 187–201.
Stenbacka, C. (2001). Qualitative research requires quality concepts of its own. Management
Decision, 39(7), 551–556.
Stevenson, J. M. (2010). Development of a test blueprint for a hospitality management capstone
course to measure programmatic student learning outcomes (Master of Science in Hospitality
Management Thesis). University of North Texas.
Stivers, B., & Phillips, J. (2009). Assessment of student learning: A fast-track experience. Journal of
Education for Business, 84(5), 258–262.
Tas, R., LaBrecque, S., & Clayton, H. (1996). Property-management competencies for management
trainees. Cornell Hotel and Restaurant Administration Quarterly, 37(4), 90–96.
University of Rhode Island. (2009). Student learning outcome. Retrieved June 30, 2014, from http://
www.uri.edu/assessment