Professional Documents
Culture Documents
2]
Innovative Method of Needs Assessment for Faculty Development Programs in a Gulf Medical School
Education for Health, Volume 23, Issue 3, 2010
ABSTRACT
Background: Faculty development lays the foundation for the quality enhancement in medical education. However, programs are
not always based on the needs of the participants, and there is dearth of information on methods to derive faculty’s needs. The
Medical Education Unit at the University of Dammam, Saudi Arabia, carried out an innovative method to identify and prioritize
faculty needs in order to plan future activities.
Methods: A questionnaire was designed, pilot-tested and administered to all faculty members (N=200). The respondents rated the
perceived importance (high, moderate, low) and their performance (good, average, poor) on twelve competencies described in the
literature. The ratings of perceived importance - high/moderate, and self-rated performance- average/poor, were summed up to
determine priority rankings for continuing education. The respondents’ rating of various continuing education activities, their
willingness to participate and commit time, and their suggestions for strengthening faculty development were also analyzed.
Results: All the twelve competencies were perceived as 'highly important' by the subjects. They felt most confident in teaching in
large and small groups, attitudes and ethical values, and decision making skills. The competencies prioritized as "gaps" were
knowing how to develop learning resources, plan curriculum, evaluate courses and conduct research. The prioritized activities were
specialized courses, orientation workshops for the new faculty, and training in educational research skills. This implied a multi-
phased approach to faculty development. A majority (62.4%) were willing to devote 2.2 hours per week to faculty development.
Respondents suggested initiatives that should be undertaken by the Medical Education Unit and the broader institution.
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 1
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
Conclusion: We demonstrated a participatory approach to needs assessment by identifying the gaps between "perceived
importance" and “self-rated performance”, as criteria for determining priorities. Findings also demonstrated the need for adopting a
comprehensive approach to faculty development in which both departmental and organizational initiatives are required. Our
findings are applicable to the Gulf Region context and our methodology can be applied anywhere.
Keywords: Need assessment, faculty development, medical education, continuing medical education
Introduction
Faculty development (FD) has assumed a great significance in meeting the diverse roles and responsibilities of a medical educator
as a clinician, researcher, administrator, and educational leader1. The main driving forces for FD are public accountability, the
changing nature of health care delivery and the need to sustain personal academic vitality2.
A volume of literature on faculty development has accumulated over the past two decades replete with useful lessons. It has been
shown that FD programs should be tailored to the needs of institutions, departments and individuals, take a systematic approach in
their planning, implementation and evaluation, utilize self-directed learning and participatory education approaches, and contribute
to both professional and personal development of the faculty3-4. Of these, the initial step of needs assessment is perhaps most
crucial because it helps all teachers realize their fullest potential and enables program organizers to optimize and prioritize their
activities.
Medical schools in the Gulf Region have witnessed a rapid increase in the enrolment of students, leading to a shortage of faculty.
Many schools have established Medical Education Units (MEU) to organize faculty development5-6. The Medical Education Unit
of the College of Medicine, University of Dammam was established in 2005 with the objective of enhancing faculty skills in
teaching, assessment and educational research. The college follows a conventional curriculum and is operated by Saudi faculty and
expatriates of diverse backgrounds. A logical step for those of us directing the Medical Education Unit was therefore to conduct a
needs assessment to inform us in planning our activities for the future.
Our principal objectives were to use a participatory process to prioritize the competencies on which to focus and the functional
activities to be undertaken by the unit. In our development efforts we also wanted to estimate the willingness of faculty to
participate and the time they would commit, so we could target a realistic set of activities. Finally, we sought to elicit suggestions
from the faculty for ways to strengthen our FD initiatives.
Previously published faculty needs assessment efforts of schools have employed a variety of tools and techniques, including
questionnaires7-12, Likert scales13-14, focus groups15, interviews16, and Delphi techniques17, generally to assess continuing medical
education (CME) needs in various settings. A few studies have captured the differences between perceptions of ‘what an ideal
CME should be’ and ‘what is actually practiced’18. There were two studies particularly relevant to our study. One attempted to
assess the difference between faculty participants’ ‘current ability’ and ‘ideal ability’ in their various roles19 and the other study
dealt with the difference between ‘perceived importance’ and ‘self-rated level of competence’ as the basis for prioritizing faculty
development needs20. Another important development in the field is the effort of Hesketh et al.21 to identify twelve competencies
expected of a medical educator.
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 2
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
Our study considers these twelve competencies as the basis for a FD needs assessment in our school and takes the view that,
ideally, the focus of FD programs should be prioritized on the basis of the difference (gap) between expected competencies and
actual performance of faculty. The greater the gap between expectations and performance, the greater the need for these areas to be
stressed in FD efforts.
Methods
We drafted a semi-structured, partly open-ended and partly closed-ended questionnaire. Central in the questionnaire was a three-
point Likert scale in which respondents were asked to rate the perceived importance (high, moderate, low) and their perceived
current performance (good, average, poor) on each of twelve competencies derived from the work of Hesketh et al.21 (see
Appendix).
Other important issues addressed in the questionnaire were the perceived functional activities and services provided by the Medical
Education Units. The subjects were asked to rate the activities in terms of relevance and usefulness to bolstering their performance.
These items were derived from the literature (Davis et al.3) and the experience of the researchers. Subjects were also asked to
indicate if they were willing to participate in FD programs and the amount of time that they could devote to this purpose. An open-
ended question was used to elicit comments and suggestions from the respondents for strengthening FD. A draft questionnaire was
created and reviewed by the research team for its content validity, then pilot-tested with ten faculty member volunteers to assess
the instrument in terms of time required for completion, language and user-friendliness. Based on feedback, the questionnaire was
modified and finalized. It was administered through the chairpersons of 25 departments to the entire faculty of College of Medicine
(N=200). Completion was voluntary and respondents were reassured that responses would be confidential.
Method of analysis: The analysis of numerical data was carried out using Microsoft Excel. The ratings made by the participants
with respect of the ‘perceived importance’ versus ‘self-rated performance’ for each of the twelve competencies were tabulated in
the form of a 3 x 3 contingency table, as illustrated in Table 1.
Table 1: Illustrative contingency table showing how ratings made by the participants on “Perceived Importance” and
“Self-rated Performance” for a given competency (teaching in large/small groups in this example) are combined to
calculate a priority score.
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 3
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
We counted the number of responses for perceived importance as well as self-rated performance for each of the twelve
competencies. For identifying the priority scores, we summed the four cells pertaining to Importance (High/Average) and
Performance (Poor/Moderate). This sum represents the gap between importance and performance, or the ‘training deficit’. Based
on the deficit scores, the priorities of competencies were ranked.
For prioritizing the functional activities expected of the Medical Education Unit, we relied on response counts and ranked them
accordingly. Similarly, we counted the number of subjects who were willing to participate as teachers or resource persons and
calculated the mean time and range to quantify this information.
The qualitative analysis of the open-ended comments and suggestions for strengthening FD was initially carried out by one
researcher who listed all comments, identified general themes and sub-themes and then grouped all comments within them. This
work was then checked independently by a second researcher as a verification of the themes and the proper grouping of all the
comments under the appropriate theme. Theme assignment differences were resolved through discussion.
Results
Participants’ Profile
A total of 109 questionnaires were returned, yielding a response rate of 54.5%. Table 2 describes the response rate by gender,
faculty rank and by pre-clinical versus clinical department appointment. All the 25 departments of the undergraduate course
(MBBS) were represented among respondents except Microbiology. Sixty-seven (61.5%) had more than 15 years of experience on
the faculty. Expatriate faculty outnumbered Saudi nationals 55% to 45%. Response rates were not different among faculty ranks
and between clinical and pre-clinical departments. However, the response rate was higher in males versus females.
Table 2: Survey participation and response rate by gender, faculty rank, and pre-clinical versus clinical department
appointment
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 4
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
Prioritization of Competencies
Table 3 shows the respondents’ rating of perceived importance, self-rated performance, the priority scores and the ranking of
priorities arrived for each of the twelve competencies. All twelve competencies were perceived as important by the faculty, as
revealed by their ratings which ranged between 72 and 86. However, their self-rated performance received highest ranking on
teaching in large/small groups (81), attitudes and ethical values (80), and decision making skills (74). The faculty felt least
confident in developing learning resources (42), planning curriculum (46), and evaluating courses and conducting research (47).
The priority scores derived (as per the procedure outlined in Table 1) from the sum of ratings of four cells – perceived importance
high/average and self-rated performance poor/average are shown in the Table 3. Developing learning resources (54) received
highest priority followed by planning curriculum (52) and evaluating courses and conducting research (49). Decision-making skills
(23), attitudes and ethical values (16), and teaching in large/small groups (16) received lowest priority.
Table 3: Respondents' rating of perceived importance, self-rated performance, priority scores and ranking with respect to
twelve faculty competencies
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 5
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
The activities prioritized by the respondents from the given list (as provided in the questionnaire) are shown in Table 4. The table
includes additional activities suggested by the respondents. Organization of specialized workshops, for example: 'How to frame
Multiple Choice Questions?' (91.7%), orientation program for newly recruited faculty (91.7%), and training in educational research
(88.9%) top the priority list of activities and services. The lowest priority was assigned to consultation services to the faculty,
e.g., how to lead a small group discussion (70.6%), assisting faculty in pursuing educational projects (68.8%), and sponsoring a
few interested faculty for higher degree programs in education (67.9%).
Faculty Development Need Assessment Survey (2009) College of Medicine, University of Dammam
A majority of the participants (62.4%) expressed willingness to participate in the activities of MEU and ability to spare an average
of 2.2 hours per week (range 1 – 8) for this purpose. Those willing to join as facilitators or resource persons (37.6%) indicated that
they can spare 2.4 hours per week. Nineteen percent indicated that they have no time for FD activities.
The comments and suggestions offered by the respondents fell within two categories: initiatives to be addressed by the school and
those to be undertaken by the MEU (Table 5). The initiatives expected at the school level were to make a policy decision for
participation in FD to be mandatory for faculty, strengthen faculty development infrastructure and facilities, and provide incentives
and recognition. The initiatives expected of the MEU were to assume more power and responsibilities to monitor course delivery,
prepare an annual calendar of activities, distribute learning resources, and emphasize practical aspects of faculty development.
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 6
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
Table 5: Comments and suggestions given by the respondents for strengthening faculty development and functioning of
school’s Medical Education Unit
Discussion
Our study demonstrates a participatory approach to a medical school’s needs assessment in which the entire faculty is involved in
identifying FD needs and priorities. We have borrowed a framework from earlier studies and applied a method of arriving at the
priority areas based on the gap between the skills that faculty perceive as most important for their roles and how they rate their own
performance in these skills.
The needs identified and prioritized by our respondents can be explained on the basis of their background experience and local
context. The respondents in our study are mostly senior faculty with more than 15 years of experience. It is natural for them to
think that their teaching skills are satisfactory, but they need support in producing learning resources, planning curriculum and
conducting research, especially in view of the challenges faced in handling new technologies like e-learning and sustaining
leadership towards the end of their career. Decision-making seems to have received low priority by a high proportion of our
predominantly expatriate faculty, who do not hold many administrative responsibilities in our context. Attitudes and ethical values
are often thought to be "self-learned behaviors", rather than "molded" by faculty development. Contrary to our expectation, the
competence in learner assessment was given only average priority. This might be explained by the training the faculty already
received through a series of workshops in this area conducted by the MEU during the last four years.
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 7
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
The activities prioritized for faculty development through identifying gaps between perceived importance and current performance
should be considered along with the faculty’s suggestions for strengthening FD initiatives. Our respondents expect the MEU to
tailor its activities for faculty at different levels of experience. Accordingly, activities should be phased in such a manner that all
new faculty are sensitized through an orientation program. Most would attend basic instructional workshops, a few would opt for
specialized workshops on different aspects, pursue research and assume educational leadership. This has been previously described
as multi-phased22 or tiered approach to faculty development23. Low priorities assigned to providing consultation to the faculty on
specific issues (e.g., how to conduct a small group discussion), assisting the faculty in pursuing educational projects, and
sponsoring interested faculty for higher degree programs abroad (e.g., Masters in Health Professional Education) indicate that most
faculty are not yet ready for long-term programs leading to educational leadership.
A major development in the field of FD is the premise that the success of FD initiatives does not rest on the efforts of the few
individuals who organize them, but depends upon organizational variables including the institution’s infrastructure and facilities,
overall leadership, faculty incentives and recognition for contributions to FD24-28. This is reflected in our respondents’ suggestions
for strengthening FD and the operations of the MEU, which have been categorized as initiatives needed by the MEU and those
needed by the institutional leadership. Preparing a calendar of activities and distributing learning resources are within the purview
of MEU. On the other hand, the steps to be taken for making the faculty participation a mandatory requirement, providing
incentives, standardizing Multiple Choice Questions, monitoring the quality of courses, and strengthening college-wide
infrastructure rest on policy choices made by the top school administration.
A valuable addition to our study is the data obtained on the willingness of the faculty to participate in FD and the time they are
willing to commit as either a participant or a resource person. This has not been highlighted in the earlier studies. We plan to create
a database of those willing to join as participants and resource persons, to be used in scheduling future programs. Those who are
not willing can be advised to pursue alternative career pathways as core scientists, clinical specialists or researchers.
Our study has some limitations. The data generated through the questionnaire, especially self-ratings of performance, is likely
subject to desirability bias and over-represents the opinions of male faculty. The faculty may not accurately recognize the
limitations of their skills as teachers. Respondents likely over-reported their willingness to commit time to FD, and their actual
cooperation with future activities stemming from this survey may well be less than they promise. Moreover, what we have gathered
is preliminary information. Further assessments are needed to validate our findings by incorporating other tools such as Delphi
technique, focus groups, interviews and external consultation. Though the findings of our study are immediately applicable in our
setting, the methodology should be replicable in any setting worldwide.
Conclusion
Through our needs assessment survey, we demonstrated a participatory approach which is simple, feasible and useful in the Gulf
Region context and should also be in other regions. We focused on identifying the gap between faculty’s perceived importance and
self-rated performance in twelve skill areas, as criteria for prioritizing FD content. Our study also led to a database for identifying
participants who are motivated and willing to give time to FD as facilitators and learners. This study also highlighted the need for
adopting a multi-phased approach in delivering FD tailored to faculty at each career stage, and emphasized the roles of both
medical education units and schools in strengthening faculty development.
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 8
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
References
1. Bligh J, Brice J. Further insights in to the role of medical educator. Academic Medicine. 2009; 84:1161-1165.
2. Gruppen LD, Simpson D, Sealer NS, Robins L, Irby DM, Mullan PB. Educational fellowship programs: common themes and overarching
issues. Academic Medicine. 2006; 81:990-994.
3. Davis M, Karunathilake I, Harden RM. AMEE Education Guide no. 28: The development and role of departments of medical education.
Medical Teacher. 2005; 27:665-675.
4. Mclean M, Cilliers F, Van Wyk JM. AMEE Guide No. 36. Faculty Development: Yesterday, today and tomorrow. Medical Teacher. 2008;
30:555-584.
5. AL-Wardy NM. Medical Education Units: History, functions and Organization. Sultan Qaboos University Medical Journal. 2008; 8:149-156.
6. Hamdy H, Telmesani AW, Al Wardy N, Abdel-Khalek N, Carruthers G, Hassan F, Kassab S, Abu-Hijleh M, Al-Roomi K, O’malley K, El Din
Ahmed MG, Raj GA, Rao GM, Sheikh K. Medical Teacher. 2010; 32:219-224.
7. Mann KV. Not another survey! Using questionnaires effectively in needs assessment. The Journal of Continuing Education in the Health
Professions. 1998; 18:142-149.
8. Laidlaw TS, Mc Leod H, Kaufman DM, Langille DB, Sargent J.. Implementing a communication skills program in medical school: needs
assessment and programme change. Medical Education. 2002; 36:115-124.
9. Maltais P, Goulet F, Borduas F. Educational skills and knowledge needed and problems encountered by Continuing Medical Education
providers. The Journal of Continuing Education in the Health Professions. 2005; 20:91-96.
10. Wood TJ, Marks M, Jabbour M. The development of a participant questionnaire to assess continuing medical education presentations.
Medical Education. 2005; 39:568-572.
11. Leite P. Faculty Development Needs Assessment Survey. 2007. Kansas State University’s College of Technology and Aviation accessed from
http://www.sal.k-state.edu/facultystaff/mission.htm
12. Farley H, Caseletto J, Ankel F, Young KD, Hockberger R. An assessment of the Faculty Development needs of junior clinical faculty in
Emergency Medicine. Academic Emergency Medicine. 2008; 15:664-668.
13. Robinson BE, Barry PP, Resnick N, Bergen MR, Stratos GA. Physician confidence and interest in learning more about Geriatric topics: A
need assessment. Journal of the American Geriatric Society. 2001; 49:963-967.
14. Bauer TA, Sanders J. Needs assessment of Wisconsin primary care residents’ and faculty interest in global health rating. BMC Medical
Education. 2009; 9:36.
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 9
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
15. Drickamer MA, Levy B, Irwin K, Rohrbaugh RM. Perceived needs for Geriatric Education by Medical Students, Internal Medicine Residents
and Faculty. Journal of General Internal Medicine. 2006; 21:1230-1234.
16. Crandall SJ. Using interviews as a need assessment tool. Journal of Continuing Education in the Health Profession. 2005; 18: 155-162.
17. McLeod PJ, Steinert Y, Meagher T, McLeod A. The ABCs of pedagogy for clinical teachers. Medical Education. 2003; 37:638-644.
18. Moore DE. Needs assessment in the new health care environment: Combining discrepancy analysis and outcomes to create more effective
CME. Journal of Continuing Education in the Health Professions. 2005; 18:133-141.
19. Amin Z, Eng KH, Gwee M, Hoon TC, Rhoon KD. Addressing the needs and priorities of medical teachers through a collaborative intensive
faculty development programme. Medical Teacher. 2006; 28:85-88.
20. Wallin DL, Smith CL. Professional development needs of full-time faculty in technical colleges. Community College Journal of Research
and Practice. 2005; 29: 87-108.
21. Hesketh EA, Bagnall G, Buckley EG, Friedman M, Goodall E, Harden RM, Laidlaw JM, Leighton-Beck L, McKinlay P, Newton R, Oughton
R. A framework for developing excellence as a clinical educator. Medical Education. 2001; 35:555-564.
22. Benor DE. Faculty development, teacher training and teacher accreditation in medical education: twenty years from now. Medical Teacher.
2000; 5:503-511.
23. Amin Z, Burdick WP, Supe A, Singh T. Relevance of Flexner Report to the contemporary Medical Education in the South Asia. Academic
Medicine. 2010; 85:333-339.
24. Rubeck RF, Witzke DB. Faculty development: A field of dreams. Academic Medicine. 1998; 73:S 32-37.
25. Steinert Y. Faculty development in the new millennium: key challenges and future directions. Medical Teacher. 2000; 22:44-50.
26. Steinert Y, Mann K, Centeno A, Dolmans D, Spencer J, Gelula M. A systematic review of faculty development initiatives designed to
improve teaching effectiveness in medical education: BEME Guide No. 8. Medical Teacher. 2006; 8:497-526.
27. Bland CJ, Seaquist E, Pacala JT, Center B, Fistad D. One school’s strategy to assess and improve the vitality of its faculty. Academic
Medicine. 2002; 77:368-376.
28. Adkoli BV, Sood R. Faculty Development and Medical Education Units in India: A survey. The National Medical Journal of India. 2009;
22:28-32.
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 10
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
Appendix
Needs Assessment Survey of Faculty Development Programs at College of Medicine, University of Dammam, Saudi Arabia – 2009
Questionnaire
Instructions for completing: Most of the questions require you to put tick (√) mark on the right response; However, a few questions are of open-
ended type. You may provide additional information if considered necessary.
1. General Information
2. Please tell us about your education and experience in medical education. Check all that apply.
I have not attended any workshop in medical I have attended national/international workshops
education
I am associated with a committee on medical I have presented a poster
education
I have acted as faculty/resource person for I have made an oral presentation
workshops
I have received fellowship in medical education I have pursued a research project in medical education
I have formal degree/diploma in medical education I have published paper(s) in medical education
Kindly attach a list of your publications and achievements in medical education. This will help us making an archive of
resources to be accessible to all faculty.
3. Please rate the competencies listed below, in terms of their importance and your performance in that area on a three point scale given
below.
Importance My performance
High Mode Low List of Competencies for a teacher Good Average Poor
rate
Teaching large and small groups
Teaching in clinical setting
Facilitating and managing learning
Planning curriculum
Developing learning resources
Assessing learners
Evaluating courses and conducting research
Understanding of educational principles
Acquiring appropriate attitudes and ethical values
Acquiring decision making skills
Developing managerial and communication skills
Achieving personal development
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 11
[Downloaded free from http://www.educationforhealth.net on Monday, October 12, 2015, IP: 14.139.245.2]
4. Please suggest activities and services which should be extended by the MEU in the future: Please check all that apply.
Check
Orientation workshops for the newly recruited faculty
Microteaching sessions for enhancing teaching skills
Basic instructional course for all teachers
Specialized courses (e.g., on assessment)
Consultation services to the needy faculty on request
Educational projects
Training in research and publication
Sponsor interested faculty for pursuing higher degrees abroad
Any other: Please list
Any other: Please list
5. Please indicate the role you can play in the future and the time you can spare for this purpose (hours per week). Fill one of the three
choices.
The role I can play for Medical Education Unit Time I can spare Hours/per wk
I am keen on updating my skills as a teacher
I can contribute as a resource person or mentor
I have other pressing commitments, thank you No time!
6. Any other comments and suggestions for strengthening faculty development and functioning of Medical Education Unit. (Use
additional sheets if required).
© BV Adkoli, KU Al-Umran, MH Al-Sheikh, KK Deepak, 2010. A licence to publish this material has been given to Education for Health:
http://www.educationforhealth.net/ 12