Professional Documents
Culture Documents
net/publication/269423772
CITATIONS READS
58 19,954
2 authors, including:
Savithiri Ratnapalan
University of Toronto
138 PUBLICATIONS 1,816 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Savithiri Ratnapalan on 14 January 2016.
Abstract: Although the concept of needs assessment in continuing medical education is well ac-
cepted, there is limited information on needs assessment in postgraduate medical education. We
discuss the learning needs of postgraduate trainees and review the various methods of needs as-
sessment such as: questionnaire surveys, interviews, focus groups, chart audits, chart-stimulated
recall, standardized patients, and environmental scans in the context of post graduate medical edu-
cation.
Keywords: residents, medical education, needs assessment, questionnaire surveys, interviews, fo-
cus groups, chart-audit, chart-stimulated recall, standardized patients, environmental scans.
Learning points
Medical residents learn by several methods such cators don’t use needs assessments, they may not
as; self-directed learning, informal bedside teaching know what the learners want and what will motivate
and structured teaching activities.1 Some of the struc- them to learn.
tured learning activities are resident lecture series,
morning reports, grand rounds, conferences, skill We have attempted in this paper to discuss resi-
workshops, and special seminars or lectures. The dents’ learning needs; who determines their learning
educational activities planned for residents aim to needs; and methods of assessing the learning needs
help them gain the necessary knowledge and skills for various educational activities.
equal to those of their peers and, facilitate improved
patient care. Learning Needs
There are formative evaluations during the resi- Residents are adult learners engaged in post-
dency program and summative evaluations at the end graduate professional education. Their needs can be
of residency to become licensed to practice inde- classified by two main criteria: who is determining
pendently. The qualifying examinations are based on the needs (learners, educators, or others) and what
well-defined educational objectives determined by standards are used as the ideal.
the professional body conducting the examination.
These examinations attempt to ensure that newly Normative needs are defined as the measured
qualified physicians have gained the necessary gap between the set standards and the individual’s or
knowledge and skills equal to those of their peers.2 group’s current knowledge.4 In medical education
normative needs encompasses knowledge, skills,
In the current climate of consumer-driven ser- attitudes and performance of the residents and are
vices, educators face increasing pressure to plan determined by the certifying bodies such as the Royal
educational activities based on the identified needs of Colleges (United Kingdom, Canada) or the specialty
the target audience.3 We feel that assessing learning boards such as the American Board of Pediatrics in
needs is a fundamental step to ensure the relevance of the USA. These standards are set based on value
the educational activity to a target audience. If edu- judgement, expert opinion, or research information.
1
Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical Med Educ Online [serial online] 2002;7:7. Available from URL
education: A review http://www.med-ed-online.org
The opinion of those being assessed is not taken into viduals rather than against normative standards. 4
account. Prescribed needs are those areas that educa- For example residents from university X are less
tors or program planners determine as inadequate and skilled in counseling compared to residents from
that need educational intervention.4 Prescribed university Y.
needs usually take normative needs into account.
Unperceived needs are discrepancies not per-
Felt or perceived needs are what the individuals ceived by learners as learning needs, characterized by
or the group have identified as what they want to the sentence, “I do not know what I don’t know.”
learn.4 Their knowledge, experience, and the envi- Teachers, professional bodies, clients or patients,
ronment they work in influence most of these needs. allied health professionals, and national and interna-
These are the learner’s needs as perceived by the tional organizations are often the bodies that identify
learner. These needs are characterized by the sen- these unperceived needs; for example, through epi-
tence, “I know what I don’t know.” Expressed needs demiological reviews of health care problems within
are what an individual or group express as their the institution or community.6
needs.4 It would be naïve to assume that all perceived
needs are expressed because there may be several real All the above described needs are useful in iden-
or perceived barriers to expression. For example, tifying educational objectives and planning educa-
learners may not want to be identified as lacking tional activities in postgraduate medical teaching
knowledge, or they may lack the opportunity to ex- (Table 1).
press their needs. Learners may also lack the motiva-
tion, communication skills, or assertiveness to ex- Needs assessment
press their needs. This lack of expression (or de-
mand) can create an unhealthy situation, because it A needs assessment is a systematic process to
can be confused with the absence of a need for a collect and analyze information on what a target
particular educational intervention.4 group needs to learn. Although the value of conduct-
ing a needs assessment is not well established in
Comparative needs are those learning needs postgraduate medical education, needs assessment is
identified by comparing two similar groups or indi- being used by some medical educators to develop
Table 1
Types of Learning Needs
2
Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical Med Educ Online [serial online] 2002;7:7. Available from URL
education: A review http://www.med-ed-online.org
curriculum for the training programs. We feel that sponses may be difficult to generalize since they
needs assessment should be conducted before any often reflect the needs, views, and wants of only the
adult educational activity so that the available re - responders. Surveys can, however, be used in con-
sources can be used to the maximum benefit of the junction with other needs assessment methods to
learners. provide reliable information.
3
Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical Med Educ Online [serial online] 2002;7:7. Available from URL
education: A review http://www.med-ed-online.org
training films to educate new trainees about the latest ment tool in a variety of situations such as in continu-
military hardware and to boost morale. Now focus ing professional education to determine learning
groups are responsible for everything from what we needs prior to education, to assess change after edu-
watch on television, how movies end, how grocery cation, and to assess variations in professional prac-
stores display their items, and how politicians speak tice. 13 They are also used to evaluate competence
to the public.12 and performance.14
Focus groups are usually, seven to ten randomly In 1984, McAuley et al used two peer assessors
selected participants who meet criteria to be consid- to evaluate 25 randomly selected charts and showed
ered consumers or learners for a particular service (or that chart audits had a 93% sensitivity and 88%
educational activity). A skilled facilitator, who en- specificity in identifying unsatisfactory patient care.15
courages a sense of synergy but explores differences Sengstaken and King used chart audits to identify the
in opinion, conducts the interview. Planning, devel- need for education in pain management of geriatric
oping core questions, facilitating or moderating the patients. 16
session, and analysis of data are the essential steps in
conducting focus group interviews.12 Residents are the frontline medical staff in most
tertiary care facilities. Random or targeted chart
Focus group interviews differ from individual audits in these institutions will identify areas of
interviews, because members of the group draw weakness in patient care, and educational interven-
strength from one another to express some opinions tions to correct these issues can be arranged.
that they may otherwise view as unpopular. This is
particularly important in eliciting negative views or Chart-Stimulated Recall
constructive criticism from learners.
Chart-stimulated recall was developed by
The use of focus groups for needs assessments in Maatsch in 1981 to assess clinical competence in
postgraduate medical education is increasing. In emergency medicine.17 In chart stimulated recall, a
Canada, the Royal College of Physicians and Sur- patient’s chart is used to stimulate the professional’s
geons conducts focus group interviews of all the recall of the particular case and its management.
residents to assess residency programs for accredita- Chart-stimulated recall is a case-based interview that
tion. Focus groups provide a broad range of qualita- provides the rationale for management decisions and
tive data in a timely, cost-effective manner and help other options that were considered and ruled out.
to clarify and elaborate the quantitative data. They Interviewers must be trained health professionals.
are very useful in areas of health care where physi- Pilot studies assure inter-rater reliability and consis-
cian attitudes or behaviour modifications are targeted tency. Interviews are usually 15 to 20 minutes in
as behaviour modification can only occur when the length for each chart-stimulated recall. Three to six
individuals or groups concerns are addressed. chart-stimulated recalls are enough to provide reli-
able and valid assessment.14 Chart-stimulated recall
Data from focus groups may not represent the is more expensive than chart audits, and is likely not
needs of the target audience unless a sufficient num- to be feasible if large numbers of physicians have to
ber of groups have been interviewed. There are no be assessed.
clear guidelines as to what constitutes a sufficient
sample size for an educational needs assessment. Chart-stimulated recall gives residents a chance
Focus group interviews of all the residents in a pro- to explain the rationale behind a particular manage-
gram (such as those conducted by the Royal college ment decision. For example, a chart audit may ap-
of Physicians and Surgeons of Canada for residency pear to indicate that patient A received inadequate
program evaluations), will be the ideal although not analgesia on day X suggesting that the resident had
always feasible. As with questionnaires, most often poor skills at assessing pain, or that the resident did
focus groups are used in conjunction with other not know the proper dosages. Chart stimulated recall
methods of needs assessment for planning educa- provides the resident with an opportunity to explain
tional activities. his or her actions that may reveal aspects of situation
that justified his or her actions but were not apparent
Chart Audits from the chart; or may further clarify the deficiencies
in knowledge or skills. Thus the educational objec-
Chart audits are reviews and assessments of tives identified could be issues such as time man-
patients’ medical records that use present criteria and agement and how to utilize back-up help, rather than
standards. Chart audits are used as a needs assess- pain management alone.
4
Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical Med Educ Online [serial online] 2002;7:7. Available from URL
education: A review http://www.med-ed-online.org
5
Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical Med Educ Online [serial online] 2002;7:7. Available from URL
education: A review http://www.med-ed-online.org
Table 2
Types of Needs Assessments
Types of Needs Type of data Good for Advantages for Can be used to
Assessments collected determining postgraduate
medical training
Questionnaires quantitative perceived needs Can sample large Identify seminar
expressed needs groups topics
A study from Dublin describes how they used Successful Educational Activities
questionnaires and semi-structured interviews to
conduct a national medical educational needs’ as- Time, money, and administrative support may be
sessment of interns to develop an intern education limited in any educational activity. It is always diffi-
and training program. 23 Another Canadian study cult to decide how much of the limited resources
describes a successful core curriculum initiative for should be spent on needs assessment. A careful needs
postgraduate medical education using surveys and assessment, however, is an important first step in
environmental scan (objectives from governing bod- planning education for any health professional. Resi-
ies ; AAMC Core Curriculum Project from the dents are time- constrained due to the numerous ‘on-
American Association of Medical Colleges and call duties’ and the heavy service load in most pro-
CanMEDS 2000 Project from the Royal College of grams; their time investment should be given serious
Physicians and Surgeons of Canada). 24 Needs as- thought. 26
sessment by residents and faculty are also used to
evaluate and modify training programs. 25 An educational activity that does not warrant the
time investment of the learners would have diminis h-
In summary, we think that the type, purpose, and ing attendance and cease to serve any purpose. The
timing of the educational activity will guide the needs teaching has to be relevant, applicable, and bring
assessment methods chosen by educators. We have about a positive change in patient care and physician
attempted to describe the types of learning needs, the practices. This has to be achieved in the least possible
various methods of needs assessment and, some time to make it not only an effective, but also an
examples of the situations when these could be used efficient process. Needs assessment of postgraduate
in postgraduate medical education (Table2 ). medical trainees (i.e., the target population) before
planning an educational activity would help achieve
these goal.
6
Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical Med Educ Online [serial online] 2002;7:7. Available from URL
education: A review http://www.med-ed-online.org
7
Ratnapalan S, Hilliard RI. Needs assessment in postgraduate medical Med Educ Online [serial online] 2002;7:7. Available from URL
education: A review http://www.med-ed-online.org
22. Ury AW, Reznich CB, Weber CM . Needs communication skills programme in medical
Assessment for a Palliative Care Curriculum school: needs assessment and programme
.Journal of Pain and Symptom Manage- change. Medical Education. 2002; 36:115-124.
ment.2000;20 (6):408-416.
26. Leach DC. Residents' work hours: the Achilles
23. Hannon F B. Aa national medical education heel of the profession? Academic Medicine.
needs’ assessment of interns and the develop- 2000; 75(12):1156-7.
ment of an intern education and training pro-
gramme. Medical Education 2000; 34:275- Correspondence
284. Savithiri Ratnapalan MBBS, MRCP, FRCPC.
Dept. of Pediatric Medicine
24. Taylor K L, Chudley A. Meeting the needs of The Hospital for Sick Children
future physicians: a core curriculum initiative 555 University Avenue
for postgraduate medical education at a Cana- Toronto ON, Ontario M5G 1X8;
dian university. Medical Education. 2001; Telephone: 416 813–7938;
35:973-982. Fax: 416 813– 7562;
E-mail: savithiri.ratnapalan@ utoronto.ca
25. Laidlaw T S, MacLeeod H, Kaaufman D M,
Laangille D B, Sargeant J. Implementing a