An Observation-Based Method for Assessing History & Physical Exam Skills in an Emergency Department

CAN IT BE DONE?
Mia Mallory M.D., Mike Fitzgerald Ph.D., Matthew Mittiga M.D., Elena Duma M.D., Charles Schubert M.D., Hamilton P. Schwartz M.D., Constance McAneney M.D., Javier Gonzalez del Rey M.D., M.Ed.
Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati Childrens Hospital Medical Center, Divison of Emergency Medicine.

Background

Results

Respondents Suggested Improvements

Respondents Perceived Benefits

Direct observation methods are important

33/34 categorical interns were observed using the

tool (97% compliance rate).

Faculty Perceptions

Faculty responses

for resident evaluation .

ACGME has mandated direct observation
methods as part of resident evaluation.

32 interns & 10 faculty completed the follow-up

survey email/ interviews respectively.
Benefits, barriers & improvements were identified.
Average time to completion was 15 minutes.

The Pediatric Emergency Department (PED)

Complete off-shift
-Take care of the patient during off shift time & finish
observation & complete chart documentation

Facilitated feedback (3/10)

Stick with simple cases (e.g., non urgent patients)

Provides focus for guided teaching (2/10)

Make forms easier to access with central repository

Improved intern decision-making skill (1/10)

Both intern and faculty should remind each other & find

Shows concern for their education (1/10)

opportunity to complete

setting provides a unique opportunity to

Identified knowledge gaps (3/10)*

Builds empathy (1/10)

observe & assess residents in a fast-paced,

Respondents Perceived Barriers

Keep it faculty driven

demanding environment.

Faculty responses

Intern Perceptions

Intern responses

ED volume (10/10)*

Make feedback concrete and specific

Direct and immediate feedback /education(25/32)

Time to get together (2/10)

More frequent encounters from a broader range of

Identified knowledge gaps (7/32)

Although observation tools are available for

assessing residents clinical skills and the

Tendency to wait until last minute (1/10)

strengths of direct observation have been

Complete in a less random fashion, don't do last minute

demonstrated,

Intern responses

-staffing shortages and the need to

maximize patient flow in the PED make

it

difficult to ensure that each resident is

directly observed and is provided direct

faculty

and

Have all faculty complete observations

Complete off shift when faculty member isn't so busy

ED volume (5/32)

Schedule when ED volume is lighter

Intimidating with faculty present (5/32)

Change session timing to scheduled procedure shift

Random timing of observation (4/32)

Consider videotaped encounters

observation-based method for assessing history

(Hx) and physical exam (PE) skills in a busy
PED.
To indentify the factors that contributed to its
utility and feasibility.

* Indicates number of respondents who mentioned the indicated theme

Conclusions
Methods
Development of the observation checklists :

To evaluate the implementation of an

Small demand on time (3/32)

Limited to a few faculty ( 3/32)

immediate feedback.

Objective

Identified knowledge starting point (6/32)

Qualitative feasibility data from an initial implementation trial

Specific Hx and PE skills checklists developed based on ACGME core competency requirements

The system of direct observation of interns in

a PED that provides adequate assessment of
Hx and PE skills revealed significant benefits
especially in the area of identification of
knowledge gaps and immediate feedback.
We were able to develop a user friendly
observation tool.

One general & four condition-specific checklists developed for gastroenteritis, asthma and fever

Modifications of the tool resulted in following process in July 2008:

Limitations

Ten volunteer (PED) faculty assigned to observe and provide feedback to pediatric interns

The number of observations per intern was

limited to one.

Interns performed a single Hx and PE in the second half of their PED rotation
Faculty members were responsible for scheduling the observation

Generalizability may be limited by using only

ten volunteer PED faculty.