See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/264228121

Brainstorming, Brainstorming Rules and Decision Making

Article in The Journal of Creative Behavior · March 2009

DOI: 10.1002/j.2162-6057.2009.tb01304.x

CITATIONS READS
255 15,681

2 authors, including:

Paul Paulus
University of Texas at Arlington
175 PUBLICATIONS 10,523 CITATIONS

SEE PROFILE

All content following this page was uploaded by Paul Paulus on 12 September 2018.

The user has requested enhancement of the downloaded file.

 Journal of Creative Behavior

VICKY L. PUTMAN
PAUL B. PAULUS

Brainstorming, Brainstorming
Rules and Decision Making

ABSTRACT
This study investigated the effect of brainstorming experience on the ability of
groups to subsequently select the best ideas for implementation. Participants
generated ideas either alone or in interactive groups and with either the regular
brainstorming rules or with additional rules designed to further increase the num-
ber of ideas generated. All participants subsequently were asked to select their
top five ideas in a group evaluation phase.
Groups of individuals generating ideas in isolation (nominal groups) gener-
ated more ideas and more original ideas and were more likely to select original
ideas during the group decision phase than interactive group brainstormers. Addi-
tional rules increased idea generation but not idea originality or idea selection.
Key words: brainstorming, decision making, facilitators, group creativity.

INTRODUCTION
A popular method for generating ideas is the technique of group brainstorm-
ing (Sutton & Hargadon, 1996) in which groups follow Osborn’s (1957) rules.
These rules instruct the group members to build upon previous ideas, to not criti-
cize any ideas, to generate a large number of ideas, and to feel free to contribute
wild ideas. Although Osborn’s rules have been shown to improve group brain-
storming performance compared to groups not using the rules (Parnes & Meadow,
1959), there is a great deal of evidence that interactive brainstorming in face-to-
face groups leads to fewer ideas and fewer high quality ideas than nominal groups,
which are composed of a comparable numbers of individual brainstormers (Diehl
& Stroebe, 1987; Mullen, Johnson, & Salas, 1991). These effects may be due to a
variety of social and task factors such as evaluation apprehension, social loafing,
production blocking, or downward comparisons (Diehl & Stroebe, 1991; Paulus,
Dugosh, Dzindolet, Coskun, & Putman, 2002). For example, group members may
feel inhibited about expressing their ideas in groups because of social anxiety

1 Volume __ Number ___ _______ Quarter 2008

Putman - Brainstorning.P65 1 7/31/2008, 9:27 AM

Black
 Brainstorming and Decision Making

(Camacho & Paulus, 1995) or they may have difficulty in effectively expressing
their ideas when they have to wait their turn while others are expressing theirs
(Nijstad, Stroebe & Lodewijkx, 2002). Social loafing (Karau & Williams, 1993)
may occur in groups when individuals do not feel as accountable or identifiable
to external evaluators for their performance in groups as they would if they per-
formed as individuals. Several studies have found that performance in groups
may move in the direction of the low performing members (Camacho & Paulus,
1995; Paulus & Dzindolet, 1993). There is evidence that each of these factors play
a role in the relatively poor performance of brainstorming groups, although some
suggest that production blocking or cognitive interference is the most important
factor (Nijstad, Stroebe & Diehl, 2003; Diehl & Stroebe, 1991)
However, such negative effects of group brainstorming are not inevitable (Paulus
& Brown, 2003; Paulus, et al., 2002; Paulus & Nakui, 2005). For example, studies
in which groups exchange ideas on computers or by means of writing have found
that such groups can perform as well or better than nominal groups (Dennis &
Williams, 2003; Dugosh, Paulus, Roland, & Yang, 2000; Paulus & Yang, 2000).
Positive effects of group brainstorming may occur when other group members
are a source of motivation (Paulus, Larey, Putman, Leggett, & Roland, 1996) or a
source of cognitive stimulation (Dugosh et al., 2000; Dugosh & Paulus, 2005).
Many manipulations appear to increase the number of ideas generated by both
nominal and interactive groups. Providing interactive groups or nominal groups
with high standards of reference for their performance has similar positive ben-
efits for both types of groups (Paulus & Dzindolet, 1993). Trained facilitators who
help manage the group interaction also often appear to increase the brainstorm-
ing performance of groups (Kramer, Kuo & Dailey, 1997; Kramer, Fleming, &
Mannis, 2001; Offner, Kramer, & Winter, 1996; Oxley, Dzindolet, & Paulus, 1996).
Facilitators have not been used with nominal groups, but we have found that
the addition of “facilitator rules” to the brainstorming rules has similar positive
effects on number of ideas generated for both interactive and nominal groups
(Paulus, Nakui, Putman, & Brown, 2006). These additional rules were developed
from a search of the facilitator literature (Oxley et al., 1996). The additional rules
are as follows: stay focused on the task; do not tell stories; do not explain ideas;
keep people talking, possibly by bringing up previous ideas; encourage others to
contribute. These additional rules may lead groups to be more efficient in their
idea sharing (less blocking) and may increase the motivation of the all the
brainstormers in the group to persist in sharing ideas. One purpose of this study
was to replicate the utility of these additional rules for brainstorming. A second
purpose was to determine the impact of prior solitary or group brainstorming and
the use of regular or additional rules on a subsequent ideal evaluation session.
In most real-world settings groups that generate ideas also spend some time
sifting through these ideas to select the best ones for further consideration
(Mintzberg, Raisinghani, & Theoret, 1976). Essentially, groups often must decide
which of the ideas that have been generated are the best ones. In the group brain-
storming literature quality has been defined along several dimensions. One of the

Putman - Brainstorning.P65 2 7/31/2008, 9:27 AM

Black
 Journal of Creative Behavior

chief aims of brainstorming is to come up with original or novel ideas. Novelty

can be defined objectively as low frequency ideas from the total pool of ideas
(Dennis, Valacich, Carte, Garfield, Haley, & Aronson, 1997). Typically, originality
is determined by the subjective ratings of judges (Amabile, 1996; Barki &
Pinsonneault, 2001; Diehl & Stroebe, 1987). Another important dimension
for idea quality is feasibility. Feasible ideas are those that have some reasonable
potential for application given worldly constraints. Again, feasibility is typically
based on subjective ratings of judges. The best ideas may be those that are both
original and feasible (Amabile, 1996; Simonton, 1988).
There is a literature on the idea evaluation process itself (e.g., Runco, 2003).
For example, Mumford and his colleagues have examined the factors that influ-
ence this process and the outcomes of different procedures (Daley & Mumford,
2006; Mumford, Lonergan, & Scott, 2002). They also point out that this evalua-
tion process is multifaceted involving appraisal, forecasting, and refinement
(Mumford, Blair, & Marcy, 2006). Although the idea evaluation process is of inter-
est as a separate area of study, our concern was with the evaluation of ideas
generated by the brainstormers themselves.
Evaluation of individually generated ideas in a subsequent group discussion
session was part of the Nominal Group Technique (Hegedus & Rasmussen, 1986;
Van de Ven & Delbecq, 1971), but there are only a few studies that have examined
the evaluation/decision making process after face-to-face brainstorming. In a study
of the effects on brainstorming on an induction task, it was found that prior brain-
storming did lead to improvement in subsequent induction performance (McGlynn,
McGurk & Effland, 2004). Kramer, et al. (1997) examined decision-making in a
self-report format. Groups and nominals were given Osborn’s typical brainstorm-
ing instructions and a short practice session, whereas groups in another condi-
tion were given no instructions or training. The participants brainstormed and
then reached a decision about the issue. For all three conditions, self-rated deci-
sion quality was similar, but the two trained conditions were more positive both in
their evaluation of the group decision as well as in their evaluation of the decision
process. Larey and Paulus (1999) instructed nominal and interactive group par-
ticipants to brainstorm for one session and then to select the best ideas from a
separate pool of ideas that they did not generate. He found that nominals per-
formed better on the divergent task of brainstorming, whereas the groups per-
formed better on the convergent task of deciding upon the top ideas. Faure (2004)
had individual and group brainstormers select the best three ideas as a group
after brainstorming. These ideas were either ones they had generated or ones
generated by another group. Even though the individual brainstormers gener-
ated more ideas than group brainstormers, there were no differences in the
quality of the subsequent ideas selected as a group. However, groups selecting
from their own pool of ideas selected ideas higher in effectiveness than those who
selected from ideas of another group. Most recently, Rietzschel, Nijstad, & Stroebe
(2006) examined idea selection after both group and individual brainstorming.
Groups generated and selected ideas in groups and individuals generated and

Putman - Brainstorning.P65 3 7/31/2008, 9:27 AM

Black
 Brainstorming and Decision Making

selected ideas alone. Although individuals generated ideas that were on the aver-
age more original than ideas generated by groups, the individuals did not select
more original ideas than groups.
The past research does not provide a clear consensus about the effectiveness
of group evaluation of ideas after brainstorming. This is not surprising since there
are a number of bases for predicting alternative outcomes. Group members can
help eliminate bad alternatives, such as ideas that are not feasible or not likely to
be accepted (Larey & Paulus, 1999; Laughlin, VanderStoep & Hollingshead, 1991).
The diverse perspectives of the group members may be useful in evaluating the
utility of the various ideas (Jackson, 1992; Schulz-Hardt, Jochims, & Frey, 2002).
However, there is some support for predicting that groups will not be particularly
good at the evaluation process. The well-established bias that groups often
have for paying attention to shared or common information may lead groups to
focus on ideas that reflect similar perspectives instead of considering original or
infrequent ideas (Stasser, Vaughan, & Stewart, 2000; Wittenbaum, Hubbell, &
Zuckerman, 1999)
There are also some methodological reasons why previous studies may not
have found differences in idea evaluation after individual and group brainstorm-
ing. In the Rietzschel et al. (2006) study the brainstorming and decision making
was either as individuals or groups, so there was no assessment of the relation-
ship of the prior brainstorming experience on a common decision making pro-
cess. In the Faure (2004) study the brainstorming procedure was designed to
minimize differences in performance between individuals and groups by having
groups generate ideas in a round robin fashion. As a result she obtained only a
26% difference in ideas between those conditions in comparison with the typical
100% difference. In contrast the present study investigated group decision mak-
ing after typical oral individual and group brainstorming, unconstrained by a round
robin format. The nominal groups should be able to take advantage of their greater
pool of ideas in the group decision phase and select more good ideas. However, it
is also possible that the nominal groups will have difficulty shifting to an interac-
tive format in the decision-making session. Since there has not yet been any shar-
ing of ideas for the nominals, this process will have to be included in the decision
phase as well. If the nominal group members are susceptible to the shared infor-
mation bias, they may not fully share their more original ideas with the rest of the
group. This may be particularly true because nominal groups will have generated
more total ideas and therefore will have a potentially more difficult task than
interactive groups when selecting the top ideas from the total idea pool.
The additional rules manipulation enabled us to examine the independent con-
tribution of number of ideas and type of brainstorming (nominal or interactive)
both for the generation of ideas and for the decision-making process. If the num-
ber of ideas generated is responsible for an advantage in the decision making
phase, similar effects should be observed for the additional rules manipulation
and the nominal/group interaction manipulation. On the other hand, if the oppor-
tunity for interaction during brainstorming is critical for group decision-making,

Putman - Brainstorning.P65 4 7/31/2008, 9:27 AM

Black
 Journal of Creative Behavior

then the effects of group-type should not match the effects of the additional rules
manipulation on decision-making. If the familiarity and cohesiveness from prior
interaction is helpful in subsequent group decision making, then group
brainstormers should be more successful than individual brainstormers in select-
ing the best ideas (e.g., original ones). However, familiarity and cohesiveness could
also inhibit a full sharing of unique or original ideas in interactive groups (Stasser
& Birchmeier, 2003) and lead the groups to focus on more frequent or highly
feasible ideas. Moreover, individual brainstormers may be more attentive to the
shared ideas in a subsequent group decision session since they are sharing these
ideas for the first time. This increased attention may lead to a better decision for
individual brainstormers than for group brainstormers.
This study was not designed to examine why nominal brainstorming and addi-
tional rules improve brainstorming performance. However, this study did provide
an opportunity to evaluate one potential factor. Part of the problem in interactive
brainstorming is that the shared idea generation process makes it difficult to
express ideas when they come to mind (Diehl & Stroebe, 1991). Furthermore, we
have noted that brainstormers often get “off-task” by needlessly elaborating on
ideas or by telling stories. This makes it even more difficult for group brainstormers
to generate a high number of ideas (c.f., Dugosh, et al., 2000). Some of the addi-
tional rules are designed to minimize this tendency. We will examine the extent to
which the rules make groups and/or individuals more efficient in the idea genera-
tion process by analyzing the number of words used to express the ideas.

METHOD
PARTICIPANTS
One-hundred and twenty participants were recruited from the introduction to
psychology classes. Their participation served to fulfill a research requirement.

DESIGN
The experimental design was a 2 (nominal vs. group) by 2 (Osborn vs. addi-
tional rules) factorial design. The experiment involved a brainstorming session
that was followed by a decision-making session. Half of the participants gener-
ated ideas as individuals and the other half generated ideas in groups of three.
Half of the participants were given only Osborn’s rules for brainstorming and the
other half of the participants were given the additional rules (see Appendix). All
participants made decisions in groups of three.

PROCEDURES
All participants were brought into a central room for informed consent proce-
dures and instructions. Participants were told we were interested in the general
issue of brainstorming and their ability to apply the rules provided in brainstorm-
ing. The experimenter read the instructions and also told the participants to write
down each idea on the notepad provided, for later reference during the decision-

Putman - Brainstorning.P65 5 7/31/2008, 9:27 AM

Black
 Brainstorming and Decision Making

making session. After the instructions, half of the participants were taken to small
rooms to brainstorm alone and half were taken to a large room to brainstorm in
an interactive group of three people. In session 1, all participants orally
brainstormed for 15 minutes into individual microphones about ways to improve
the university. The experimenter left the room when the session began.
In session 2, the participants who had previously brainstormed alone were now
placed into a group of three in the large lab room for the decision making phase.
In contrast, for the interactive group participants the decision making phase
occurred in the same lab room in which they brainstormed. These participants
simply moved over to a different part of the lab room where the chairs were
arranged for the decision-making session. All participants were given instructions
for decision making, which indicated that they should discuss their ideas from
the previous session and come to a decision about the top 5 ideas. The groups
were told to rank-order what they decided were the best 5 ideas. Research by
Hollingshead (1996) showed that groups make better decisions when they are
told to rank-order their answers instead of simply trying to come up with the over-
all best answer. Forcing the groups to compare the ideas should cause the group
members to more carefully consider a greater amount of relevant information.
The decision process was videotaped.
The groups were told that they had 15 minutes to make this decision, but that
they would be given more time if needed. The experimenter left the room once
the session began. The group was finished with their decision once they filled out
the top 5 list. A flip chart was provided for the groups to use to list their ideas for
discussion. When one page was filled with ideas, it was attached to the wall. The
experimenter recorded the length of time to make the decision. After they had
decided on the top 5 ideas, the individual participants were given a questionnaire
about their perceptions of the two sessions on 9-point scales. They were asked
to rate the number and quality of ideas generated, rate their enjoyment of the
brainstorming tasks, rate the extent to which the rules helped brainstorming, their
feelings (nervous, self-conscious, ate ease, calm, self-aware, and worried) on
five point scales.
CODING AND RELIABILITY
All audiotapes from the brainstorming session were transcribed verbatim and
then coded for number of ideas. The lists of ideas written by each participant
during the brainstorming session also were checked for any ideas that were writ-
ten but not spoken. These were counted since they could be shared during the
decision phase. For the interactive groups the number of ideas for each of the
three members was combined into a group total, deleting any repeated ideas. For
the nominal brainstormers the three transcripts were compared and any repeated
ideas were deleted, which resulted in an idea list that included ideas from all 3
nominals. A second person coded 20% of the transcripts to check for reliability.
The overall Cronbach’s alpha was 0.99. The number of words used for each idea
was counted and used to develop a score of brainstorming efficiency for each
person—the number of words used divided by the number of ideas generated.

Putman - Brainstorning.P65 6 7/31/2008, 9:27 AM

Black
 Journal of Creative Behavior

All ideas from the brainstorming session were combined into a total list of 1514
ideas. This list was narrowed down to 604 non-repeated ideas for rating purposes,
since many participants generated similar ideas. For example, the idea of “more
parking lots” and the idea of “more parking spaces” were both combined into the
idea of “more parking” for the 604 idea list. Therefore, the ideas that were sepa-
rate ideas for brainstorming purposes were not necessarily separate ideas for
the rating list. For example, one group generated two different but similar ideas
about textbooks (e.g., “don’t overcharge for books,” “cheaper books”). The group
received credit for two separate ideas in the brainstorming phase but for the deci-
sion-making phase both ideas were given the same label and both ideas received
the same rating on both dimensions. One rater created the shortened list of ideas
and two other raters evaluated the combination decisions. Any disagreements
were discussed until there was complete agreement on the final list.
Feasibility and originality were each measured on a 1 to 5 Likert scale with 5
being the highest on each dimension. An idea was rated as high in feasibility if the
idea was considered to be practical and likely to be implemented at the univer-
sity. An idea was rated as high in originality if it was thought to be creative and not
typically mentioned by most brainstormers. The quality ratings of the ideas in
terms of feasibility and originality were checked for reliability. As in previous stud-
ies concerning idea quality (Dennis et al., 1997; Diehl & Stroebe, 1987, 1991),
one highly trained rater coded all of the ideas and then another highly trained
rater coded 25% of the ideas. The agreement among the raters was assessed by
counting the number of times that the two raters were within one rating point of
each other. This number was then expressed as the percentage of the total num-
ber of ideas that were rated by both raters. The overall feasibility reliability was
93.17% and the overall reliability for originality was 85.71%. New ideas that were
brought up in the decision-making session (often combinations of brainstorming
ideas) were rated with a reliability of 98.63% for feasibility and 93.15% for original-
ity. Condition information was not available during the rating process so that
this could not influence the ratings. The correlations between the measures of
feasibility and originality was .29.
Raters coded the videotapes of the decision phase for time to decision, degree
of sharing of ideas, cohesion, conflict, and interruptions. The interrater reliabilities
for these measures ranged from .64 (cohesion) to .98 (time to decision).

MEASURES OF IDEA QUALITY AND DECISION-MAKING PERFORMANCE

Four measures of idea quality were used in this study. Within each group of
three members, the average feasibility of the ideas was calculated as well as the
average originality. The frequency of the idea in the entire pool of brainstorming
ideas was used as a third measure of idea quality. The average frequency of the
ideas generated by each group was calculated. Because the best types of ideas
might be ones high in feasibility as well as high in originality, the fourth measure
of idea quality in this study was a combined additive measure of feasibility and
originality (Dennis et al., 1997; Diehl & Stroebe, 1987, 1991).

Putman - Brainstorning.P65 7 7/31/2008, 9:27 AM

Black
 Brainstorming and Decision Making

To measure the quality of the decision made by the groups, the average feasi-
bility of the group’s top 5 ideas was calculated as well as the average originality
and the average feasibility/originality combined score. Calculating the average
frequency of the top 5 ideas was complicated by the fact that there were often
groups that selected an idea as one of their top 5 but did not say the idea in their
brainstorming session or combined several brainstorming ideas into one top 5
idea. Since the new ideas were not part of the brainstorming session, there were
no frequency scores for these new ideas. Therefore, in calculating our measure of
frequency we gave a frequency of 1 both to ideas that were new to the decision-
making session (not said by any of the 40 groups while brainstorming) and to top
5 ideas that were combinations of ideas from the brainstorming session. The ideas
that were common ideas (said by many groups but not that particular group)
were given the frequency number of the idea in the entire pool of the experiment.
Only 7 of the 40 groups (5 interactive groups and 2 nominal groups) had no new
ideas or no combined ideas in their top 5 list.
RESULTS
BRAINSTORMING PERFORMANCE
The number of ideas generated in session 1 was analyzed by means of a 2 x 2
between-groups ANOVA. One factor was the group type (nominal or interactive
group) and the other factor was the type of rules given (Osborn’s rules or addi-
tional rules). Both main effects were significant. Participants who brainstormed
as nominal groups generated more ideas (M = 45.75) than those who brainstormed
in an interactive group (M = 29.95), F (1, 36) = 11.94, p < 0.002. eta2 = .25. Partici-
pants given the additional rules generated more ideas (M = 44.00) than those
who were given only the Osborn rules (M = 31.70), F (1, 36) = 7.23, p < 0.02,
eta2 = .17. The interaction was not significant (see Table 1). Tukey post hoc com-
parisons of the four conditions indicated that only the Osborn group condition
and the Additional Rules Nominal condition were significantly different.
TABLE 1. Mean Number of Nonrepeated Ideas Generated and Number of Words
Per Idea as a Function of Type of Rules and Group Type.
Type of rules
Osborn’s Additional
Group No. No. No. No.
type ideas words/idea ideas words/idea
Interactive
M 23.40 123.47 36.50 62.75
SD 6.57 60.06 9.50 29.65
Nominals
M 40.00 62.30 51.50 41.63
SD 17.81 17.65 19.65 19.19

Putman - Brainstorning.P65 8 7/31/2008, 9:27 AM

Black
 Journal of Creative Behavior

The ratio of number of words per idea also was analyzed by a 2 x 2 ANOVA.
Participants who were given the Osborn rules used more words to express
their ideas
(M = 92.88) than did participants who were given the additional rules
(M = 52.19),
F (1, 36) = 12.82, p < 0.002. eta2 = .26. Participants in groups used more words
per idea (M = 93.11) than did nominals (M = 51.96), F (1, 36) = 13.11, p < 0.002,
eta2 =.27. (see Table 1). A regression analysis indicated that the word usage
mediated the effect of group condition on ideas. The regressions of number of
ideas on group condition and of number of words on group condition were both
significant, B = .47, t(38) =3.24, p < .01 and B = .43, t (38) = 2.91, p < .01. The
relationship between groups and number of ideas was no longer significant when
both number of ideas and number of words were entered into the regression model,
B = .20, t(37) = 1.63, ns. The effect of number of words was still significant,
B = .61, t(37) = 4.85, p < .01. Sobel’s test indicated that this effect is significant
(Z = 2.49, p < . 05).

BRAINSTORMING QUALITY
For originality, the ANOVA revealed a main effect for group type. The ideas
generated by the nominals had a higher mean level of originality (M = 2.83) than
did the ideas generated by the interactive groups (M = 2.63), F (1, 36) = 6.63, p <
0.02, eta2 = .16. The ANOVA for frequency also revealed a group type main effect.
The ideas generated by the groups were more frequent in the idea pool (M =
9.06) than were the ideas generated by the nominals (M = 7.08), F (1,36) = 10.21,
p < 0.005, eta2 = .22 (see Table 2). There were no significant effects of rule type
on the measures of idea quality.

TABLE 2. Mean Rating of Idea Originality and Mean Idea Frequency as a

Function of Type of Rules and Group Type.

Type of rules
Osborn’s Additional
Group type Originality Frequency Originality Frequency
Interactive
M 2.58 9.74 2.69 8.39
SD 0.25 2.42 0.23 2.09
Nominals
M 2.77 7.11 2.88 7.06
SD 0.18 1.67 0.28 1.53

Putman - Brainstorning.P65 9 7/31/2008, 9:27 AM

Black
 Brainstorming and Decision Making

DECISION PHASE; ANALYSES FOR TOP 5 IDEAS

For the top 5 ideas selected by the groups, the ANOVAs revealed no significant
differences among conditions for the feasibility measure. For the average origi-
nality of the top 5 ideas, there was a group type main effect. Participants who
brainstormed as nominals selected top 5 ideas that were more original (M = 2.88)
than did those participants who brainstormed as interactive groups (M = 2.55), F
(1, 36) = 9.05, p < 0.01, eta2 = .20. For the frequency of the top 5 ideas, there was
a group type main effect. Interactive brainstorming groups selected ideas that
were more frequent in the idea pool (M = 10.19) than those who brainstormed as
nominal groups (M = 7.46), F (1, 36) = 4.42, p < 0.05, eta2 = .11. To determine to
what extent the mean quality of the selected ideas in the decision phase differed
from the average of the ideas of the brainstorming phase for nominals or groups,
separate analyses were done within each of these conditions. The only significant
effect obtained was for frequency of ideas in groups. The ideas selected in the
decision phase were more frequent (M = 11.51, SD = 4.56) than those in the brain-
storming phase (M = 9.06, SD = 2.31), F (1, 19) = 5.15, p < .05, eta2 = .21.
The amount of time taken to make a decision was not correlated with the num-
ber of ideas generated but was significantly correlated both with the originality of
the brainstorming ideas, r = 0.38, p < 0.05, and with the originality of the top 5
ideas, r = 0.33, p < 0.05. No significant differences were obtained for data ob-
tained from the videotapes of the decision session.

QUESTIONNAIRE ANALYSES
For questions about feelings during the brainstorming session, the only signifi-
cant result was a main effect for rules. The participants who were given the Osborn
rules reported being significantly calmer (M = 4.27) than did the participants
who were given the additional rules (M = 3.75), F (1, 116) = 7.92, p < 0.01. For
satisfaction with the group’s final decision, there were also main effects for rules.
Participants who were given the Osborn rules were more satisfied with the final
decision (M = 8.68) than were those participants who were given the additional
rules (M = 8.20), F (1, 116) = 7.99, p < 0.006. Participants who were given the
Osborn rules felt the process was more thorough (M = 7.78) than did those
participants who were given the additional rules (M = 7.20), F (1, 116) = 5.66,
p < 0.02. There were no differences among the conditions in ratings of behavior
during the decision phase.
DISCUSSION
This study examined both the idea generation and idea selection processes for
brainstorming groups as a function of the type of brainstorming conditions and
the rules employed. The pattern of number of ideas generated was consistent
with past research (e.g., Diehl & Stroebe, 1987; Paulus et al., 2006), with both
nominal status and addition rules leading to increased generation of ideas. The
ideas generated in the nominal conditions also tended to be more original
and less frequent than those generated in the group conditions. One of the main

10

Putman - Brainstorning.P65 10 7/31/2008, 9:27 AM

Black
 Journal of Creative Behavior

questions in this study was the extent to which participants in the various condi-
tions would be able to sort through their idea pool and then select the best ideas.
The idea selection reflected the higher quality of ideas generated in the nominal
condition, with those who had generated ideas as nominals subsequently select-
ing more original and less frequent ideas than those who had generated ideas in
groups. So this study demonstrated that the advantage of nominal groups car-
ries over to a subsequent idea selection phase. This outcome may seem obvious,
but it should be noted that this did not occur in two prior studies of idea selection
after brainstorming (Faure, 2004; Rietzschel, et al., 2006). In the Rietzschel et al.
(2006) study individuals who brainstormed individually also generated more ideas
and ideas that were more original and less feasible than those who brainstormed
in groups. The individuals subsequently selected ideas as individuals and the
groups as a group. There was no difference between the conditions in the quality
of ideas selected. It is possible, that the contrast between individual idea genera-
tion and group decision-making in this study was critical in producing a benefit of
nominal brainstorming in the decision making phase. Individual brainstormers
may be more attentive to the ideas being shared in the group brainstorming ses-
sion since they are hearing them for the first time. This may increase their sensi-
tivity to the quality (originality and frequency) of those ideas. It is also possible
that performing as an individual prior to group decision-making makes one less
prone to some of the factors that inhibit effective decision making such as the
commonality bias (Stasser & Birchmeier, 2003). As mentioned earlier, the failure
of the Faure (2004) study to find a difference in group idea selection after indi-
vidual or group brainstorming may related to the smaller than usual difference in
performance between the individual and group brainstormers.
Our findings indicate that having a high number of ideas to sort through does
not necessarily distract or overwhelm participants in their attempts to select the
best ideas. The interaction time and prior sharing of ideas that the groups had
together during brainstorming did not seem to put them at an advantage over the
nominals who had not interacted with each other prior to the decision-making
session. However, spending more time in the decision-making phase was related
to more original ideas being selected in the top 5. Thus taking more time can lead
to a better decision making outcome (c.f., Hirokawa, 1990). Surprisingly, the mean
originality of the ideas selected in the decision phase was not higher than mean
of the ideas in the brainstorming phase. This finding is consistent with a similar
finding by Rietzschel et al. (2006) that individuals or groups are not particularly
good at picking out high quality ideas. Essentially the groups in our study per-
formed at a chance level in the selection of the best ideas along the dimension of
originality. There was a bias away from chance in the selection of more frequent
ideas in the decision phase. This suggests that ideas that are more likely to have
been have been generated by more than one individual in the group or are more
likely to “resonate” with group members (Tindale, Meisenhelder, Dykema-
Engblade, Hogg, 2001) and have a higher chance of being selected in group

11

Putman - Brainstorning.P65 11 7/31/2008, 9:27 AM

Black
 Brainstorming and Decision Making

decision making. This of course consistent with the bias toward shared informa-
tion found in the problem solving literature (Stasser & Birchmeier, 2003).
The differential pattern of results for the group type and rule type manipula-
tions for brainstorming and decision-making are theoretically instructive. Although
the additional rules condition enhanced the production of number of ideas, it did
not affect the quality of the ideas generated or selected in the decision phase.
This result and that of past studies (Faure, 2004; Rietzschel et al., 2006) suggest
that simply generating a high number of ideas does not insure that an increased
number of good ideas will be generated and therefore selected in the decision
phase. Why is increased idea generation related to increased originality for the
nominal groups but not for the additional rules conditions? Possibly, with the lack
of social distraction in the nominal conditions there is more careful processing of
the ideas being generated (reflection about the ideas and memory for the ideas;
Brown, Tumeo, Larey, & Paulus, 1998). As a result there may be a higher number
of good ideas generated and a higher number available for the decision-making
phase. This analysis is consistent with the functional perspective of Hirokawa
(1990), which emphasizes the importance of thorough processing, especially when
there is a high quantity of information.
The use of additional rules enhanced the number of ideas generated by nominals
as well as for groups. One possible basis for this effect, as well as the superior
performance of the nominals, is the number of words used to express ideas. High
levels of brainstorming performance for nominal groups and additional rules
groups were associated with a low number of words per idea. These findings and
the related mediation analysis suggest that brainstorming efficacy is related to
efficiency of idea generation. Interactive groups generated about half as many
ideas as nominals possibly because they used almost twice as many words per
idea as the nominal groups. Group discussions may be more off-task than nomi-
nal idea generation because interactive groups may be prone to discuss issues
not directly related to the brainstorming task. Alternatively, group members gen-
erate their ideas in a more conversational style than nominals and this conversa-
tional style may require more words. Interactive group members must make
affirming statements to keep the conversation going. However, those statements
can take time away from generating ideas or in other ways distract the idea gen-
eration process (Dugosh et al., 2000). In a similar fashion, participants given the
additional rules are more efficient in how they express their ideas, compared to
participants who are only given Osborn’s rules. The additional rules may lead
group members to be more task focused. One compelling feature of the results
is that the additional rules interactive groups and the nominal Osborn groups
were similar both in number of ideas generated and in numbers of words used to
express those ideas. This finding suggests that if the wordiness of interactive groups
and nominal groups is similar, then their performance will be similar. This effect
occurred in spite of the fact that groups have experienced some degree of pro-
duction blocking and suggests that the associative benefits of group interaction
may counter the negative effects of production blocking (Brown & Paulus, 2002).

12

Putman - Brainstorning.P65 12 7/31/2008, 9:27 AM

Black
 Journal of Creative Behavior

The findings for the quality of the brainstorming ideas also replicate previous
results (Barki & Pinsonneault, 2001; Casey, Gettys, Pliske, & Mehle, 1984; Diehl &
Stroebe, 1987, 1991). Groups not only generated fewer total ideas than did
nominals, but they also generated fewer original ideas. Nominal groups gener-
ated more original and low frequency ideas than interactive groups. Unlike most
of the previous idea quality studies, the current study also found differences in
mean level of originality between nominals and groups.
The questionnaire results revealed that the additional rules participants had
some negative reactions to the decision-making phase. Possibly, the pressure
from the implied standards set by the additional rules may be one reason that
these participants were less satisfied with their decision, felt that their decision
was less thorough, and felt less calm than did the participants who were only
given the Osborn rules. The negative feelings of the additional rules participants
are of course inconsistent with the fact that they performed better in the brain-
storming phase and the same in the decision phase in comparison to the Osborn
participants. The finding is reminiscent of previous findings that groups typically
do not have a very good idea of how well they are performing (e.g., Paulus,
Dzindolet, Poletes, & Camacho, 1993).
Although this study has increased our knowledge about the idea evaluation
process after brainstorming, we were not able to measure the processes critical
to the various effects obtained. For example, we do not know exactly what it is
about the additional rules that leads to increased performance. Our results sug-
gest that increased efficiency in expressing ideas in the additional rules condition
(fewer words per idea) may be an important factor. However, we did not obtain
measures of how effectively participants followed the rules. Videotape analyses
of the group decision process revealed no differences among the conditions in
the extent of sharing of ideas, how well the ideas were discussed, degree of cohe-
sion or degree of conflict. Future studies will need to determine more exactly the
basis for the effects of additional rules and the effects of prior brainstorming on
idea evaluation. Our task situation was limited in time for brainstorming and deci-
sion-making. More extensive time for both brainstorming and decision-making
might affect the type of results obtained and might yield a more extensive and
richer the type of evaluation process (Mumford et al., 2006).
In conclusion, it was found that individuals who generated ideas alone per-
formed better in both the brainstorming and group decision phase in comparison
to those who brainstormed in groups. Additional brainstorming rules facilitate
generation of number of ideas but not the number of high quality ideas and the
subsequent decision making performance. The high number of ideas generated
by the nominal groups and the additional rules groups appear to be related to the
fact that these groups use fewer words to express their ideas. Future studies will
need to determine why the high idea generation of the nominal groups is associ-
ated with increased originality but high idea generation produced by another
manipulation such as additional rules or individually generating ideas and select-
ing ideas (Rietzschel et al., 2006) does not have a similar benefit.

13

Putman - Brainstorning.P65 13 7/31/2008, 9:27 AM

Black
 Brainstorming and Decision Making

REFERENCES
AMABILE, T. M. (1996). Creativity in context. Boulder, CO: Westview Press.
BARKI, H, & PINSONNEAULT, A. (2001). Small group brainstorming and idea quality: Is electronic
brainstorming the most effective approach? Small Group Research, 32, 158-205.
BROWN, V. R, & PAULUS, P. B. (2002). Making group brainstorming more effective: Recommendations
from an associative memory perspective. Current Directions in Psychological Science, 11, 208-
212.
BROWN, V., TUMEO, M., LAREY, T. S., & PAULUS, P. B. (1998). Modeling cognitive interactions during
group brainstorming. Small Group Research, 29, 495-526.
CAMACHO, L. M., & PAULUS, P. B. (1995). The role of social anxiousness in group brainstorming. Journal
of Personality and Social Psychology, 68, 1071-1080.
CASEY, J. T., GETTYS, C. F., PLISKE, R. M., & MEHLE, T. (1984). A partition of small group predecision
performance into informational and social components. Organizational Behavior and Human
Performance, 34, 112-139.
DAILEY, L., & MUMFORD, M. D. (2006). Evaluative aspects of creative thought: Errors in appraising the
implications of new ideas. Creativity Research Journal, 18, 367-384.
DENNIS, A. R., VALACICH, J. S., CARTE, T. A., GARFIELD, M. J., HALEY, B. J., & ARONSON, J. E.
(1997). Research report: The effectiveness of multiple dialogues in electronic brainstorming.
Information Systems Research, 8, 203-211.
DENNIS, A. R., & WILLIAMS, M. L. (2003). Electronic brainstorming: Theory, research, and future
directions. In P.B. Paulus & B. A. Nijstad (Eds.), Group creativity. (pp. 160-178) New York: Oxford.
DIEHL, M., & STROEBE, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a
riddle. Journal of Personality and Social Psychology, 53, 497-509.
DIEHL, M., & STROEBE, W. (1991). Productivity loss in idea-generating groups: Tracking down the blocking
effect. Journal of Personality and Social Psychology, 61, 392-403.
DUGOSH, K. L., & PAULUS, P. B. (2005). Cognitive and social comparison processes in brainstorming.
Journal of Experimental Social Psychology, 41, 313-320.
DUGOSH, K. L., PAULUS, P. B., ROLAND, E. J., & YANG, H. C. (2000). Cognitive stimulation in
brainstorming. Journal of Personality and Social Psychology, 79, 722-735.
FAURE, C. (2004). Beyond brainstorming: Effects of different group procedures on selection of ideas
and satisfaction with the process. Journal of Creative Behavior, 38, 13-34.
HEGEDUS, D. M., & RASMUSSEN, R. V. (1986). Task effectiveness and interaction process of modified
nominal group technique in solving an evaluation problem. Journal of Management, 12, 545-560.
HIROKAWA, R. Y. (1990). The role of communication and group decision-making efficacy: A task-
contingency perspective. Small Group Research, 21, 190-204.
HOLLINGSHEAD, A. B. (1996). Information suppression and status persistence in group decision-making:
The effects of communication media. Human Communication Research, 23, 193-219.
JACKSON, S. E. (1992). Team composition in organizational settings: Issues in managing an increasingly
diverse work force. In S. Worchel, W. Wood, & J. A. Simpson (Eds.) Group process and productivity
(pp. 138-173). Newbury Park: Sage.
KARAU, S. J. AND WILLIAMS, K.D. (1993). Social loafing: A meta-analytic review and theoretical
integration. Journal of Personality and Social Psychology, 65, 681-706.
KRAMER, M. W., KUO, C. L., & DAILEY, J. C. (1997). The impact of brainstorming techniques on
subsequent group processes: Beyond generating ideas. Small Group Research, 28, 218-242.
KRAMER, T. J., FLEMING, G. P., & MANNIS, S. M. (2001). Improving face-to-face brainstorming through
modeling and facilitation. Small Group Research, 32, 533-557.
LAREY, T. S., & PAULUS, P. B. (1999). Group preference and convergent tendencies in small groups: A
content analysis of group brainstorming performance. Creativity Research Journal, 12, 175-184.

14

Putman - Brainstorning.P65 14 7/31/2008, 9:27 AM

Black
 Journal of Creative Behavior

LAUGHLIN, P. R., VANDERSTOEP, S. W., & HOLLINGSHEAD, A. B. (1991). Collective versus individual
induction: Recognition of truth, rejection of error, and collective information processing. Journal of
Personality & Social Psychology, 61, 50-67.
MCGLYNN, R. P., MCGURK, D. & EFFLAND, V. S. (2004). Brainstorming and task performance in groups
constrained by evidence. Organizational Behavior and Human Decision Processes, 93, 75-87.
MINTZBERG, H., RAISINGHANI, & THEORET, A. (1976). The structure of unstructured decision processes.
Administrative Science Quarterly, 21, 246-275.
MULLEN, B., JOHNSON, C., & SALAS, E. (1991). Productivity loss in brainstorming groups: A meta-
analytic integration. Basic and Applied Social Psychology, 12, 3-23.
MUMFORD, M. D., BLAIR, C. S., MARCY, R. T. (2006). Alternative knowledge structures in creative thought:
Schema, associations, and cases. In J. C. Kaufman and J. Baer (Eds.), Creativity and reason in
cognitive development. (pp. 117-136). New York: Cambridge University Press.
MUMFORD, M. D., LONERGAN, D. C., SCOTT, G. (2002). Evaluating creative ideas: Processes, standards,
and context. Inquiry: Critical thinking across the disciplines, 22, 21-30.
NIJSTAD, B., & STROEBE, W. (2006). How the group affects the mind: A cognitive model of idea
generation in groups. Personality and Social Psychology Review, 10, 186-213.
NIJSTAD, B. A., STROEBE, W., & DIEHL, M. (2003). Cognitive stimulation and interference in idea-
generating groups. In P. B. Paulus and B. A. Nijstad (Eds.), Group creativity: Innovation through
collaboration (137-159). New York: Oxford University Press.
NIJSTAD, B. A., STROEBE, W., & LODEWIJKX, H. F. M. (2002). Cognitive stimulation and interference
in groups: Exposure effects in an idea generation task. Journal of Experimental Social Psychology,
38, 535-544.
OFFNER, A. K., KRAMER, T. J., & WINTER, J. P. (1996). The effects of facilitation, recording, and pauses
on group brainstorming. Small Group Research, 27, 283-298.
OSBORN, A. F. (1957). Applied imagination (1st ed.). New York: Scribner.
OXLEY, N. L., DZINDOLET, M. T., & PAULUS, P. B. (1996). The effects of facilitators on the performance
of brainstorming groups. Journal of Social Behavior and Personality, 11, 633-646.
PARNES, S. J., & MEADOW, A. (1959). Effect of “brainstorming” instructions on creative problem-solving
by trained and untrained subjects. Journal of Educational Psychology, 50, 171-176.
PAULUS, P. B., & BROWN, V. (2003). Enhancing ideational creativity in groups: Lessons learned from
research on brainstorming. In P.B. Paulus & B. A. Nijstad (Eds.), Group creativity. (pp. 110-136)
New York: Oxford.
PAULUS, P. B., DUGOSH, K. L., DZINDOLET, M. T., COSKUN, H., & PUTMAN, V. L. (2002). Social and
cognitive influences in group brainstorming: Predicting production gains and losses. European Social
Psychology Review, 12, 299-325.
PAULUS, P. B., & DZINDOLET, M. T. (1993). Social influence processes in group brainstorming. Journal
of Personality and Social Psychology, 64, 575-586.
PAULUS, P. B., DZINDOLET, M. T., POLETES, G., & CAMACHO, L. M. (1993). Perception of performance
in group brainstorming: The illusion of group productivity. Personality and Social Psychology
Bulletin, 19, 78-89.
PAULUS, P. B., LAREY, T. S., PUTMAN, V. L., LEGGETT, K. L., & ROLAND, E. J. (1996). Social influence
processes in computer brainstorming. Basic and Applied Social Psychology, 18, 3-14.
PAULUS, P. B., & NAKUI, T. (2005). Facilitation of group brainstorming. In S. Schuman (Ed.), The IAF
handbook of group facilitation (pp. 103-114). San Francisco: Jossey-Bass.
PAULUS, P. B., NAKUI, T., PUTMAN, V. L., & BROWN, V. R. (2006). Effects of task instructions and brief
breaks on brainstorming. Group dynamics: Theory, research, and practice, 10, 206-219.
PAULUS, P.B., & YANG, H.C. (2000). Idea generation in groups: A basis for creativity in organizations.
Organizational Behavior and Human Decision Processes, 82, 76-87.
RIETZSCHEL, E., NIJSTAD, B. A., & STROEBE, W. (2006). Productivity is not enough: A comparison of
interactive and nominal brainstorming groups on idea generation and selection. Journal of
Experimental Social Psychology, 42, 244-251.

15

Putman - Brainstorning.P65 15 7/31/2008, 9:27 AM

Black
 Brainstorming and Decision Making

RUNCO, M. A. (2003). Idea evaluation, divergent thinking, and creativity: Perspectives on creativity
research. In M. A. Runco (Ed.) Critical creative processes.(pp. 69-94) Cresskill, NJ: Hampton Press.
SCHULZ-HARDT, S., JOCHIMS, M., & FREY, D. (2002). Productive conflict in group decision making:
Genuine and contrived dissent as strategies to counteract biased information seeking. Organizational
Behavior and Human Decision Processes, 88, 563-586.
SIMONTON, D. K. (1988). Scientific genius: A psychology of science. New York: Cambridge University
Press.
STASSER, G., & BIRCHMEIER, Z. (2003). Group creativity and collective choice. In P.B. Paulus & B. A.
Nijstad (Eds.), Group creativity. (pp. 85-109) New York: Oxford.
STASSER, G., VAUGHAN, S. I., & STEWART, D. D. (2000). Pooling unshared information: The benefits of
knowing how access to information is distributed among group members. Organizational and
Human Decision Processes, 82, 102-116.
SUTTON, R. I., & HARGADON, A. (1996). Brainstorming groups in context: Effectiveness in a product
design firm. Administrative Science Quarterly, 41, 685-718.
TINDALE, R. S., MEISENHELDER, H. M., DYKEMA-ENGBLADE, A. A., HOGG, M. A. (2001). Shared
cognition in small groups. In M. A. Hogg and S. Tindale (Eds.), Blackwell Handbook of Social
Psychology: Group Processes. Malden, MA: Blackwell Publishers.
VAN DE VEN, A. & DELBECQ, A. L. (1971). Nominal versus interacting group processes for committee
decision-making effectiveness. Academy of Management Journal, 14, 203-212.
WITTENBAUM, G. M., HUBBELL, A. P., & ZUCKERMAN, C. (1999). Mutual enhancement: Toward an
understanding of the collective preference for shared information. Journal of Personality & Social
Psychology, 77, 967-978.

This paper is based in part on a dissertation by the first author under the supervision of the second
author. Thanks are due to Toshihiko Nakui for his assistance with the data analysis. For additional
information about this research, contact the authors at paulus@uta.edu or the College of Science, Box
19047, University of Texas at Arlington, Arlington, TX 76019.

16

Putman - Brainstorning.P65 16 7/31/2008, 9:27 AM

Black
 Journal of Creative Behavior

APPENDIX

OSBORN’S RULES AND ADDITIONAL RULES FOR INTERACTIVE GROUPS

Brainstorming is a form of group interaction which is used to facilitate the flow
of ideas. It is widely used in a large number of U.S. corporations, and is generally
used when new, unique, original, and creative ideas are desired. It is not used to
solve everyday problems. The procedure is relatively straightforward and easy to
comprehend. The following rules are for brainstorming in groups. We want you to
apply these rules as best as you can while working as a group.

1.) Criticism is ruled out. Adverse judgment of ideas must be withheld. No one
should criticize anyone else’s ideas. Say everything you think of.

2.) Freewheeling is welcome. The wilder the idea the better. It is easier to tame
down than to think up. Don’t be afraid to say anything that comes to mind.
The farther out the idea the better. This will stimulate more and better ideas.

3.) Quantity is wanted. The greater the number of ideas the more likelihood of
winners. Come up with as many as you can.

4.) Combination and improvement are sought. You should try to suggest how
ideas of others can be joined or changed into still better ideas. Don’t be afraid
to combine and improve on them.

5.) Stay focused on the task. Concentrate on the problem at hand and avoid
engaging in irrelevant thought processes or discussions. When it is neces-
sary to interrupt a group member, say something like, “Remember that we
need to stay focused on our task.”
a.) Do not tell stories. We are only interested in your ideas. Do not allow
your group members to tell stories about their experiences.
b.) Do not explain ideas. Do not allow your group members to expand on
why they think something is good or bad. Let them say an idea and then
interrupt them.

6.) Keep the brainstorming going. During a lapse of time when no one is talk-
ing, someone in the group should say something like, “Let’s see what other
ideas we can come up with for (restate the problem).”

7.) Return to previous categories. When the group members are not talking
very much, go back to categories of ideas that have already been mentioned
and try to build on these previous ideas. For example, say “Does anyone
have any more ideas related to (restate an idea already suggested)?”

17

Putman - Brainstorning.P65 17 7/31/2008, 9:27 AM

Black

View publication stats