THE DELPHI TECHNIQUE

Randall B. Dunham
November 5, 1996

Purpose

The purpose of the Delphi technique is to elicit information and judgments from
participants to facilitate problem-solving, planning, and decision-making. It does so
without physically assembling the contributors. Instead, information is exchanged
via mail, FAX, or email. This technique is designed to take advantage of participants’
creativity as well as the facilitating effects of group involvement and interaction. It is
structured to capitalize on the merits of group problem-solving and minimize the
liabilities of group problem-solving.

Requirements

The Delphi technique requires a Coordinator to organize requests for information,

information received, and to be responsible for communication with the participants.
The Delphi technique requires an efficient communication channel to link the
Coordinator with each of the participants. It is common to use the U.S. mail for this
purpose, but FAXes and email can decrease the time required for completing a Delphi
technique. Elapsed time from beginning to end of the process averages 44 days using
the U.S. mail and as little as five days using email. Nevertheless, the Coordinator’s
job can take substantial time. Recent experience suggests that coordination of the
Delphi technique using email with 20 participants and the processing of three
questionnaires could utilize 30-40 hours of the Coordinator’s time.

Process

1. Identify the issue and solicit ideas. For example:

What action could be taken to provide faster response to patient

inquiries between visits?

Prepare and send the first questionnaire, which asks each participant to engage in
individual brainstorming so as to generate as many ideas as possible for dealing
with the issue.

2. Response to first questionnaire. Each participant lists his/her ideas

(Questionnaire #1) in a brief, concise manner and returns the list anonymously to
the Coordinator. These ideas need not be fully developed. In fact, it is preferable
to have each idea expressed in one brief sentence or phrase. No attempt should be
made to evaluate or justify these ideas at this point in time.

3. Create and send Questionnaire #2. The Coordinator prepares and sends a
second questionnaire to participants that contains all of the ideas sent in response
to the first questionnaire and provides space for participants to refine each idea, to
comment on each idea’s strengths and weaknesses for addressing the issue, and to
identify new ideas.
 2

4. Response to second questionnaire. Participants anonymously record their

responses to Questionnaire #2 and return them to the Coordinator.

5. Create and send Questionnaire #3. The Coordinator creates and sends a third
questionnaire that summarizes the input from the previous step and asks for
additional clarifications, strengths, weaknesses, and new ideas.

6. Continuation of the process. If desired, the Coordinator performs iterations of

the preceding process until it becomes clear that no new ideas are emerging and
that all strengths, weakness, and opinions have been identified.

7. Resolution. Resolution may occur in one of two ways.

• If dominant, highly evaluated ideas emerge via consensus, the exercise is

declared finished. The end product is a list of ideas with their concomitant
strengths and weaknesses.

• The Coordinator conducts a formal assessment of the group’s opinions of the

merits of the ideas. There are a number of ways to conduct a formal
evaluation. In one method, the Coordinator prepares a questionnaire that lists
all the ideas and asks participants to rate each one on a scale. For example, a
7-point scale could be used that ranges from 0 (no potential for dealing with the
issue) through 7 (very high potential for dealing with the issue). If this
approach is used, participants send the rating forms to the Coordinator, who
compiles the results and rank-orders the ideas based on the evaluations.

A second approach for evaluating the ideas is that which is used in the
Nominal Group Technique for “voting.” With this approach, the Coordinator
asks each member to identify the top five ideas and assign five points to the
most promising idea, 4 points to the next most promising, and 3, 2, and 1 points
to the third, fourth, and fifth-best ideas. These votes are returned to the
Coordinator, who tallies the results and prepares a report. The report notes
the rank order of the ideas based on the total number of points received and
indicates the number of people who voted for each idea.
 QUESTIONNAIRE #1

The purpose of this questionnaire is to elicit your ideas regarding the following issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please engage in individual brainstorming so as to generate as many ideas as possible

for dealing with this issue. Please list each idea in a brief, concise manner and email
your response to me. Your ideas need not be fully developed. In fact, it is preferable
to have each idea expressed in one brief sentence or phrase. No attempt should be
made to evaluate or justify these ideas at this point in time. Your ideas will be
anonymously included in the next questionnaire.

Idea #1:

Idea #2:

Idea #3:

Idea #4:

Idea #5:

Idea #6:

Idea #7:

Idea #8:

Idea #9:

Idea #10:
 QUESTIONNAIRE #2

The purpose of this questionnaire is to report all of the ideas sent in response to the
first questionnaire and to solicit new ideas for dealing with the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please refine ideas already received by clarifying them where desired and by listing
the strengths and weaknesses you associate with each. Please list any new ideas at
the bottom of the questionnaire and comment on each new idea’s strengths and
weaknesses for addressing the issue. Your ideas will be anonymously included in the
next questionnaire.

Idea #1. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #2. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #3. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #4. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #5. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #6. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #7. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #8. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:
 2

Idea #9. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #10. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #11. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #12. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #13. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #14. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #15. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

My new ideas:

#1. .................................
• Strengths:
• Weaknesses:

#2. .................................
• Strengths:
• Weaknesses:

#3. .................................
• Strengths:
• Weaknesses:
 QUESTIONNAIRE #3 (alternative A)

Attached is a listing of all strengths, weaknesses, and clarifications of ideas sent in

response to the second questionnaire dealing with the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please further refine these ideas by providing additional clarification where desired
and by listing additional strengths and weaknesses you associate with each. Please
list any new ideas at the bottom of the questionnaire and comment on each new idea’s
strengths and weaknesses for addressing the issue.

Please note that this will be the last questionnaire. After receiving all participants’
responses to this questionnaire, I will provide you with a sheet on which to assign
votes for five ideas you feel will best deal with the issue. All votes will be tallied and
results sent to participants. Your ideas will be anonymously included in the next
report.

Idea #1. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #2. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #3. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #4. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #5. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #6. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:
 2

Idea #7. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #8. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #9. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #10. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #11. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #12. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #13. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #14. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #15. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #16. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:
 3

Idea #17. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #18. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #19. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #20. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #21. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

My new ideas:

#1. .................................
• Strengths:
• Weaknesses:

#2. .................................
• Strengths:
• Weaknesses:

#3. .................................
• Strengths:
• Weaknesses:
 4

ATTACHMENT: RESPONSES TO SECOND QUESTIONNAIRE

Idea #1. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #2. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #3. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #4. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #5. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #6. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #7. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #8. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #9. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #10. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #11. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................
.
.
.
.
 BALLOT

The purpose of this ballot is to solicit votes for the five ideas that best deal with the
issue:

What action could be taken to provide faster response to patient

inquiries between visits?

The following are all of the ideas that have been submitted by participants. Please
identify the top five ideas and assign five points to the most promising idea, 4 points
to the next most promising, and 3, 2, and 1 points to the third, fourth, and fifth-best
ideas. Vote for only five ideas. Your ideas will be anonymously included in the next
questionnaire.

Idea #1: .................

I would give this idea (0, 1, 2, 3, 4, 5) points.

Idea #2: .................

I would give this idea (0, 1, 2, 3, 4, 5) points.

Idea #3: .................

I would give this idea (0, 1, 2, 3, 4, 5) points.

Idea #4: .................

I would give this idea (0, 1, 2, 3, 4, 5) points.

Idea #5: .................

I would give this idea (0, 1, 2, 3, 4, 5) points.

Idea #6: .................

I would give this idea (0, 1, 2, 3, 4, 5) points.

Idea #7: .................

I would give this idea (0, 1, 2, 3, 4, 5) points.

Idea #8: .................

I would give this idea (0, 1, 2, 3, 4, 5) points.

.
.
.
.
 VOTING RESULTS

The following reports the results of votes cast for ideas submitted on the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please note the rank order of the ideas based on the total number of points received.
Also indicated are the number of people who voted for each idea.

Idea #6 received 22 points. 10 people voted for this idea.

Idea #13 received 20 points. 14 people voted for this idea.

Idea #2 received 19 points. 8 people voted for this idea.

Idea 18 received 15 points. 10 people voted for this idea.

.
.
.
.
.
 QUESTIONNAIRE #3 (alternative B)

Attached is a listing of all strengths, weaknesses, and clarifications of ideas sent in

response to the second questionnaire dealing with the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please further refine these ideas by providing additional clarification where desired
and by listing additional strengths and weaknesses you associate with each. Please
list any new ideas at the bottom of the questionnaire and comment on each new idea’s
strengths and weaknesses for addressing the issue.

Please note that this will be the last questionnaire. After receiving all participants’
responses to this questionnaire, I will send each of you a rating form on which you will
be asked to assess the merits of all ideas submitted using a 7-point scale. After those
rating forms have been returned, I will compile the results and rank-order the ideas
based on participants’ evaluations. Your ideas will be anonymously included in the
next report.

Idea #1. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #2. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #3. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #4. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #5. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #6. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:
 2

Idea #7. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #8. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #9. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #10. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #11. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #12. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #13. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #14. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #15. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #16. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:
 3

Idea #17. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #18. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #19. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #20. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

Idea #21. .................................

• Your clarification (if any):
• Strengths:
• Weaknesses:

My new ideas:

#1. .................................
• Strengths:
• Weaknesses:

#2. .................................
• Strengths:
• Weaknesses:

#3. .................................
• Strengths:
• Weaknesses:
 4

ATTACHMENT: RESPONSES TO SECOND QUESTIONNAIRE

Idea #1. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #2. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #3. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #4. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #5. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #6. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #7. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #8. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #9. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #10. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................

Idea #11. .................................

• Strengths listed:....................................................
• Weaknesses listed:....................................................
.
.
.
.
 RATING SHEET

The purpose of this questionnaire is to report strengths, weaknesses, and clarification of all
ideas submitted in response to the third questionnaire for dealing with the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please assess the merits of all ideas listed below using a scale that ranges from 0 (no
potential for dealing with the issue) through 7 (very high potential for dealing with the issue).
Please email your evaluation to me. After all rating forms have been returned, I will
compile the information, rank-order the ideas based on the evaluations, and send you the
results. Your ideas will be anonymously included in the next report.

Idea #1: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

Idea #2: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

Idea #3: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

Idea #4: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

Idea #5: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

Idea #6: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

Idea #7: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

Idea #8: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.

Idea #9: .....................................

This idea has (0,1,2,3,4,5,6,7) potential for dealing with the issue.
.
.
.
 RESULTS

The following reports the results of points that participants assigned to the ideas submitted
on the issue:

What action could be taken to provide faster response to patient

inquiries between visits?

Please note the rank order of the ideas based on the total number of points received.

Idea #6 received 66 points, for an average rating of 3.3

Idea #13 received 56 points, for an average rating of 2.8

Idea #2 received 25 points, for an average rating of 1.25

Idea 18 received 20 points, for an average rating of 1.0

.
.
.
.
.