You are on page 1of 10

Concept Exploration Through

Gregory Smith
Design Engineer
Michelin Americas Research Corporation
Morphological Charts:
An Experimental Study
Jenkins Richardson
Product Engineer This paper presents two design experiments to evaluate the impact of the size and shape
E-Z-Go (A Textron Company) of a morphological chart used to identify promising concepts within those charts. In each
e-mail: experiment, students designed either a burrito folder or a pet feeder. The first experiment
used two rectangular morphological charts; one vertically oriented with more functions
Joshua D. Summers and one horizontally oriented with more means. The second experiment used four mor-
Associate Professor phological charts, two charts similar to the first experiment and two nearly square with a
Clemson University similar number of functions and means. Participants from similar populations were en-
e-mail: rolled and similar protocols were followed to align the results for comparison. A compar-
ison of the significant aspects of each experiment emphasized the problem, the
Gregory M. Mocko participants, the morphological charts employed, and the experimental procedures. Fur-
Assistant Professor
thermore, a critique of the two experiments is provided and joint conclusions are drawn.
Clemson University Experimental results suggest that reducing information in the morphological charts, such
as the number of functions, enhances the likelihood for developing higher quality inte-
grated concept solutions. While accurate for most configurations, this benefit is limited in
that the largest sized chart failed to show worse results compared with smaller counter-
parts. [DOI: 10.1115/1.4006261]

1 Motivation functions than means. Adding functions to a morphological chart

failed to improve the results.
The engineering design process consists of conceptual, embodi-
A morphological chart is a table of functions and solution
ment, and detail design phases [1–3]. In conceptual design, the prin-
means for each function. Normal convention entails listing func-
ciple solution is created; in embodiment design, the physical layout
tions in a column in the left side of the table and solution means
of the solution is determined. During detail design, the materials
to the right of each function. A generic morphological chart for
are specified, the production systems are finalized, and the lower
four functions of a burrito-folding device is shown (Table 1) per
level dimensional issues are resolved [4]. This work, involving
this convention. Various terms exist for these solutions; Dym and
morphological charts, focuses on the conceptual design phase.
Little use the term means [5], Pahl and Beitz use working princi-
The conceptual design phase ideally follows a linear process but
ple [4], and Suh [6] use the term design parameters. Here, the
typically is realized through a highly iterative one. The design prob-
authors use means to refer to solutions for specific functions. In
lem is elucidated and function decomposition follows. Solutions for
this morphological chart, a verb–noun pair (e.g., “receive tortilla”)
each subfunction are identified and combined to create a potential
represents each function. A solution phrase (e.g., “table” or
integrated conceptual design. Multiple designs are evaluated, and a
“locating plate”) represents each mean.
principle solution, or solution set, is selected, which is further
explored in the embodiment and detail design phases. This research
is limited to the combinatorial activities in conceptual design that 1.1 Uses. Morphological charts, also known as concept com-
are supported by these morphological charts [1,5,6]. bination tables [9] or function–means tables [5], are design tools
Idea generation tools may be classified as either intuitive or logi- for generating integrated conceptual design solutions for design
cal [7]. Intuitive idea generation tools attempt to promote creativity, problems. Tables are constructed by decomposing design prob-
while logical idea generation tools follow a prescribed set of quasi- lems by listing all the critical functions in a column. The means
automated steps [7]. Morphological charts are defined as intuitive for each function are listed in the rows to the side (Table 1). Com-
idea generation tools [7]. Specific guidelines regarding morphologi- bining one means for each function produces an integrated con-
cal chart construction and use to create high quality design concepts ceptual design solution. Repeating this process with every
are currently unavailable in the literature. Here, the authors detail possible combination in the morphological chart generates an ex-
the development of the guidelines for purposes of using morpholog- haustive list of conceptual design solutions. Thus, morphological
ical charts more effectively as logical idea generation tools. Though charts provide a sense of the size of the design space [5].
previous work suggests a plethora of possibilities that cause evalua- The creation of a morphological chart involves compiling lists
tion and selection difficulties, no specific guidance or concrete stud- of desired functions, all at the same level of detail [5]. For each
ies support this assertion [8]. As such, the authors examine the function, a list of possible means is generated, each typically at
quality of integrated conceptual design solutions generated by the same level of abstraction, though recent work suggests meth-
undergraduate mechanical engineering students in two design ods to accommodate varying levels of abstraction [10]. Morpho-
experiments. Results show a statistically significant difference in logical chart size is calculated by multiplying the number of
the quality of concepts generated from different sizes and shapes of means for each function. For a morphological chart with functions
the morphological charts. Specifically, a chart with more means F1, F2, F3 … Fn, where each function has a number of means (M1
than functions produces better concepts than the one with more for F1, M2 for F2 … Mn for Fn), the number of integrated concep-
tual design solutions is determined by

Contributed by the Tribology Division of ASME for publication in the JOURNAL Y

OF MECHANICAL DESIGN. Manuscript received May 27, 2010; final manuscript Number of solutions ¼ Mi (1)
received March 12, 2011; published online April 24, 2012. Assoc. Editor: Yan Jin. i¼1

Journal of Mechanical Design Copyright V

C 2012 by ASME MAY 2012, Vol. 134 / 051004-1

Downloaded From: on 01/28/2016 Terms of Use:

Table 1 Sample morphological chart Table 4 Number of integrated conceptual design solutions in
three different sized morphological charts
Function Means
Number of means Number of
Receive tortilla Table Locating plate Work on top of a for each function possible solutions
stack of tortillas
Receive filling Hopper Packets Tube Chart shown in Table 1 3, 3, 2, and 2 36
Combine Dispense Wrap tortilla Adding one new function 3, 3, 2, 2, and 2 72
materials filling onto tortilla around filling with two means
Fold tortilla Spatula Hinged work surface Adding two means to 3, 5, 2, and 2 60
an existing function

Table 2 Compatibility matrix for two functions

As one example of potential design space size, recall the burrito
Receive filling folder morphological chart in Table 1. Here, there are three means
for the function “receive tortilla,” three for the function “receive
Hopper Packets Tube filling,” two for the function “combine filling,” and two for the
function “fold tortilla.” Therefore, the total possible combination
Combine materials Dispense filling onto tortilla OK OK OK
is the product of 3, 3, 2, and 2, which equals 36 (Table 3).
Wrap tortilla around filling OK ? NO
As shown, the list of possible combinations quickly grows quite
large [5]. Adding another function to a morphological chart multi-
plies the number of design solutions by the number of means
If there are three solutions for function “a,” two for function
added with that function. For example, adding another function
“b,” and five for function “c,” then the total number of concepts is
with two means to the chart shown in Table 1 doubles the Qnumber
30 (3  2  5 ¼ 30). For a filled rectangular morphological chart,
of integrated conceptual design solutions from 36 to 72 ( (means
with n functions where the number of means for each function is
per function) ¼ 3  3  2  2  2 ¼ 72). Alternatively, adding
the same (M1 ¼ M2 ¼ M3 ¼ … ¼ Mn ¼ M), an upper bound of the
means to existing functions in a chart increases the number of sol-
size of the design space is MN.
utions. For example, adding two new means to the “receive
To use a morphological chart, individual solution concepts are
Q function increases the number of solutions to 60
created by combining one solution means from each function. The
( (means per function) ¼ 3  5  2  2 ¼ 60), as illustrated in
solution list size, which quickly grows quite large when additional
Table 4. Thus, it is clear that adding identical numbers of informa-
means and functions are added to the morphological chart, can be
tion units (cells within the morphological chart) varies the total of
reduced by eliminating impractical concept means, to prune the
information represented in the entire table. Understanding this
initial chart. Impractical combinations of allowable means are
variance is the crux of this research.
also eliminated from combinatorial consideration, thereby reduc-
While these advantages and disadvantages are illustrated, few
ing the number of design solutions [1,5,9,11]. One such pruning
studies have systematically explored the justification of these
approach, compatibility matrices, is proposed in Ref. [4]. The
compatibility matrix in Table 2 evaluates the suitability of com-
bining concept means for two selected functions. The total num-
ber of compatibility matrices required is 2n where n is the number 1.3 Computational Automation. There have been several
of functions identified. Here, a tube combined to receive the filling attempts to automate the morphological chart exploration, recog-
interferes with wrapping the tortilla around the filling and should nizing the combinatorial challenge of exploring the design space.
not be considered in the integration step. For example, the interactive approach of Bryant et al. [12] is a
Morphological charts are used in an open design space and hybrid of previously developed tools: an automated morphologi-
when the designer wishes to use a methodical process to generate cal search [13] and a computational concept generator [14]. The
concepts to explore this space. While different idea generation automated morphological search is a web-based tool through
techniques can be used to populate the means within the charts, which users populate morphological charts using information in
there is no systematic approach for exploring this extensive design the design repository [15]. The computational concept generator
space. Defining guidelines to explore and refine this space is the employs a user-defined functional description of a product. This
objective of this research. function block diagram is converted to a matrix describing inter-
functional relationships. Using a function-component matrix
1.2 Advantages and Disadvantages. As with all design (FCM) and a design-structure matrix (DSM) from design reposi-
tools, morphological charts have several advantages and disadvan- tory (DR) information, a list of solutions is created, filtered, and
tages that influence their use. Advantages include the elucidation presented to the user. By combining the characteristics of these
of unexpected pairings of features [9], the creation of new con- two tools, an interactive morphological search is created, with
cepts otherwise not considered by designers [5], and capability for both the solution accessibility of the web-based morphological
encompassing the entire design space. Morphological charts used chart search and the connectivity information in the computational
as design tools are limited in that (i) number of integrated concep- concept generator. The major limitation of this method is the
tual design solutions can grow quite extensive [5], (ii) not all com- amount of design knowledge currently entered in the DR.
binations generated are good solutions to design problems [5], and Tiwari et al. [16] used a Genetic Algorithm to obtain solutions
(iii) absence of a method to choose promising solutions for further (combinations of means) from a morphological chart. Here, the
evaluation. means combination process is automated via representation as a

Table 3 List of possible combinations

Number Concepts Receive tortilla Receive filling Combine materials Fold tortilla

1 1–1–1–1 Table Hopper Dispense filling onto tortilla Spatula

2 2–1–1–1 Locating plate Hopper Dispense filling onto tortilla Spatula
… … … … … …
36 3–3–2–2 Work on top of stack Tube Wrap tortilla around filling Hinged work surface

051004-2 / Vol. 134, MAY 2012 Copyright V

C 2012 by ASME Transactions of the ASME

Downloaded From: on 01/28/2016 Terms of Use:

Table 5 Comparison of experiments

Experiment #1 Experiment #2 Justification for change

Design problem Device to prepare a burrito Device to feed a cat or dog Possibility of overlap of
participants between two experiments
(overlap did not occur)
Evaluation criteria Low cost (9) Easy to clean (9) Low cost (7) Reduce impact of weightings
(weight) Easy to clean (3) Low number of parts (1)
Low number of parts (1)
Number of evaluation 3 4 Greater granularity
Suggested evaluation values 9 (good) 3 (fair) 1 (poor) 9 (very good) 6 (good) 3 (ok) 1 (poor) Reduce impact of weightings
Evaluation method Decision matrix Not applicable

combinatorial multi-objective optimization problem. Multiple cri- Research was limited to the morphological chart size and shape
teria used to judge each combination determine that which is opti- as related to the number of functions and means per function in
mal for each criterion. Positive outcomes include a minimal the charts that designers control in construction. The number of
computational effort, use of a multitude of information from the functions and the number of means per function influence the
designer, quick feedback to the designer from many candidate sol- number of integrated concepts produced by the chart. Additional
utions, and consistent result events if uncertainty is in the inputs impact areas, such as participants creating charts or participants
(i.e., a range of values rather than a single value). The limitations with varying degrees of experience, were not investigated to avoid
to this approach include the amount of parameters requiring input, introducing experimental variation, and complicate statistical
the lack of representation of nonbehavioral characteristics (i.e., analysis without increasing the participant numbers. A data com-
aesthetics), a consistency of inputs (multiple designers introduce parison of each experiment is summarized in Table 5.
input variations), and the obtained solutions may not be globally
Weas et al. [17] discuss using the analysis of interconnected de- 2.1 Design Problem. The design problems for the two
cision areas (AIDA) to combine solution principles (means). experiments were developed to create an innovative solution.
AIDA is a systematic approach for managing complex urban plan- Each problem entailed creating a burrito folder and an automatic
ning problems. This work describes an experiment with under- pet feeder. The design problem was taken from an assignment in
graduate students who applied the AIDA principles to a design the course from a previous semester to ensure that it was suitable
problem. Here, AIDA utilizes an interaction matrix that helps for the participants’ level of knowledge and expertise. By review-
designers to identify compatible sets of solution principles. Posi- ing project reports from the previous assignment, the functional
tive outcomes entailed the automated exploration of the design decomposition and the morphological chart were created using
space, allowed designers to quickly view the effects of design means extracted from the students’ work. The automatic pet
decisions, and the creation and evaluation of unique design con- feeder problem was designed based upon positive results of the
cepts. Limitations include only allowing one performance metric first experiment. In each problem, existing solution implementa-
per iteration (multiple performance criteria with user-defined tions were not obvious to any participant. The problems were
weights being more efficient) and only one binary compatibility developed such that they required multiple functional operations
value (a scale being more appropriate). and in domains requiring no expertise or familiarity beyond that
Others have tried to develop automation tools to help explore of a sophomore mechanical engineering student. Postexperiment
design space [18–20], yet these are not directly related to mor- interviews with participants found no unease about the lack of
phological charts. While each approach provides tools for requisite expertise. This design problem has been used in two sub-
exploring the design spaces within morphological charts, they sequent research experiments [25–28].
cannot exhaustively explore the solution concepts. The objective
here is to understand how to reduce the design space to improve
navigation without sacrificing the solution quality. Thus, guide- 2.2 Morphological Charts. Various sized morphological
lines are needed to help designers learn to construct morphologi- charts were created. As seen in Tables 6 and 7, experiment one
cal charts that effectively explore the design space represented examined two sized “rectangular” morphological charts with sig-
therein. nificant disparity between functions and means. This was the first
attempt to determine experimentally the effect of amount of com-
binatorial information on the quality of solutions generated.
2 Experimental Method Though the number of information units (total number of means)
The hypothesis addressed in this research is: Designers can cre- within each chart was held constant at 15, they were distributed to
ate higher quality integrated conceptual design solutions using either three or five functions. The authors note that the means for
morphological charts with fewer integrated conceptual design sol- populating both charts were quite similar as the chart size and the
utions. More specifically, designers in a certain time can explore design space orientation were of interest, not individual means
more easily a morphological chart with less combinatorial infor- within the charts. Means in both charts are shaded, indicating
mation than one with combinatorial possibilities. Previous studies overlap. For common functions (fill tortilla, fold burrito, and dis-
suggest that designers are challenged by the design space and that pense burrito), three means were common and distributed
reducing it would enhance exploration [21–23]. Using the strategy throughout the charts to prevent bias associated with mean selec-
of Shah et al. [24], the authors found that while many aspects tion position. Means selected in each chart was done randomly to
regarding the use of morphological charts are worthy of study, prevent inclusion of high quality potential means in the easier
examining the impact of morphological chart size regarding the navigated smaller chart. Because the purpose is to compare chart
quality of integrated concepts generated can yield the best results. size, each has different means.
Other items affecting the quality of integrated concepts include Because results suggested a possible effect, in the second
the amount of time to consider the morphological chart, designer experiment additional relationships between morphological chart
experience, and the presence of known good or bad solutions in sizes were studied. Four chart sizes (Tables 8 through 11) tested
the charts, all of which are beyond the scope of this research. the combinations of two different function numbers and two

Journal of Mechanical Design MAY 2012, Vol. 134 / 051004-3

Downloaded From: on 01/28/2016 Terms of Use:

Table 6 Experiment #1 morphological chart #1 (3 3 5)

Function Means

Store and Extrude through Pour from hopper Transfer with spoon Unwrap individual serving Sprinkle in powder form
dispense filling tube from sack from bucket size and drop then rehydrate
Fold burrito Plate under tortilla is Spatula lifts the edges Flexible work Work-surface edges lift up and Tortilla punched through
hinged and folds of the tortilla surface rolls up slide toward center hole in work surface
Dispense burrito Slide on conveyor belt Slide down chute Push off to side Drop from elevated area Grab with mechanical hand

Table 7 Experiment #1 morphological chart #2 (5 3 3)

Function Means

Store filling Multiserving package Bulk filled hopper Single serving package
Position tortilla Physical stop Visual marker Work on top of a stack of tortillas
Fill tortilla Extrude filling through tube Pour filling onto tortilla Spoon filling onto tortilla
Fold burrito Spatula lifts edges Roll into tube Punch through opening in table
Dispense burrito Gravity Conveyor belt Mechanical hand

Table 8 Experiment #2 morphological chart #1 (4 3 7)


Function 1 2 3 4 5
Fill bowl Screw Rotary pocket Conveyor belt Solenoid Piston
Regulate food Weight Windup–spring Fixed volume container Fluid displacement Trip laser
Signal (to fill bowl/ inform owner) Weight Lever in bowl Camera/image processing Proximity sensor Radar
Power source Battery Gravity Air pressure—compressed air Windup–spring Engine

Table 9 Experiment #2 morphological chart #2 (4 3 5)


Function 1 2 3 4 5 6 7
Fill bowl Screw Rotary pocket Conveyor belt Solenoid Piston Blower Robotic arm
Regulate food Weight Windup—spring Cam Fixed volume Fluid displacement Plunger Trip laser
Signal (to fill Switch Weight Lever in bowl Camera/image Proximity sensor Radar Pressure sensor
bowl/inform owner) processing
Power source Battery Gravity Air pressure—compressed air Plug–AC Windup—spring Flywheel Engine

Table 10 Experiment #2 morphological chart #3 (8 3 5)


Function 1 2 3 4 5
Fill bowl Screw Rotary pocket Conveyor belt Solenoid Piston
Quantity of food Weight Butterfly valve Fixed volume container Wheel—fixed volume Trip laser
Regulate time Cam Pendulum Windup—spring Reversible chemical reaction Fluid displacement
Record keeping Analog–dial Graduation etched on Dial-on dispenser Measure and display weight Rotary counter
transparent (moves if dispenser
container moves!)
Signal owner to Buzzer (electronic) Light SMS/email Emit odor Play recorded
fill the reservoir message
Signal itself to fill bowl Weight Lever in bowl Camera/image processing Proximity sensor Radar
Error check Capacitance Laser LVDT Electric contact—bowl Weight
and dispenser
Power source Battery Gravity Air pressure—compressed air Windup—spring Engine

different means numbers. Here, the means of the largest chart to over 5 million possible combinations). Table 12 shows morpho-
were populated and pruned, without considering the quality of logical chart comparison of experiments one and two.
means to fit the smaller charts. Essential functions were retained
upon pruning. 2.3 Participants. Participants in both experiments were en-
The second experiment used morphological charts with several rolled in the same second year mechanical engineering course
concepts several magnitudes larger than the first experiment (625 (Table 13) with none participating in both experiments, while the

051004-4 / Vol. 134, MAY 2012 Transactions of the ASME

Downloaded From: on 01/28/2016 Terms of Use:

Table 11 Experiment #2 morphological chart #4 (8 3 7)


Function 1 2 3 4 5 6 7
Fill bowl Screw Rotary Conveyor Solenoid Piston Blower Robotic
pocket belt arm
Quantity Weight Butterfly Cam Fixed Wheel— Plunger Trip laser
of food valve volume fixed volume
Regulate Timer— Cam Pendulum Bimetal— Windup— Reversible Fluid
time microprocessor heater spring Chemical displacement
Record Analog— Graduation Barometer Gauge stick Dial on Measure and Rotary
keeping dial etched dispenser (moves if display counter
on transparent dispenser moves!) weight
Signal owner Buzzer Light Vibrate SMS/email Emit odor Ring bell Play recorded
to fill the (electronic) (mechanical) message
Signal itself to Switch Weight Lever Camera/image Proximity Radar Pressure sensor
fill bowl in bowl processing sensor
Error check Hall effect Capacitance Laser LVDT Electric contact— Weight Photo detector
bowl and dispenser
Power source Battery Gravity Air pressure– Plug—AC Windup—spring Flywheel Engine
compressed air

Table 12 Morphological chart comparison

Experiment #1 Experiment #2
(burrito folder) (pet feeder)

Number (and sizes) of morphological charts 2 (3  5, 5  3) 4 (4  5, 4  7, 8  5, 8  7)

Method used to change the number of functions Change level of decomposition Add or remove specific functions
Method used to change the number of means Add or remove specific means
Number of possible concepts from smallest morphological chart 125 625
Number of possible concepts from largest morphological chart 243 5,764,801

Table 13 Participants

Experiment #1 Experiment #2
(burrito folder) (pet feeder)

Participants’ class Sophomore level mechanical engineering course in design and kinematics
Number of class sections 2 1
Number of participants 25 29
Minimum experience with morphological charts Morphological charts lecture to class before the experiment
Morphological charts review at start of experiment
Additional experience with morphological charts Unknown
Number of repeat participants Unlikely due to the time separation (1 year) between experiments

relevant characteristics remained identical. Although the first For the first experiment, the evaluation criteria were low cost
experiment encompassed two sections of the same class with dif- (9), easy to clean (3), and number of parts (1). For each solution,
ferent professors, both performed equivalently and results were the participants were instructed to score their integrated solution
considered as a single group (see Sec. 4.1 for proof of statistical against these criteria as either good (9), fair (3), or poor (1). It was
equivalence). As the second experiment encompassed a single determined that low evaluation concepts (poor, 1) for the highest
class section, pooled population analysis was not required. The weighted criteria (low cost, 9) never achieved similar scores to
minimum experience with morphological charts was normalized high evaluation concepts (good, 9) regardless of the evaluations
in which students were required to build charts interactively. for the other criteria (number of parts and easy to clean). An inter-
mediate evaluation level (6) was established and low cost-7 and
2.4 Procedure. Both experiments proceeded similarly easy to clean-9 criteria were reweighted. To correct this problem
(Table 14). Each participant was provided with a prepopulated in experiment two, evaluation criteria and levels were adjusted to
morphological chart and solution form, in which participants reduce the weighting impact, and changed to easy to clean (9),
listed either eight (first experiment) or ten (second experiment) low cost (7), and number of parts (1). Evaluation solution values
integrated conceptual design solutions and rated them against pro- were very good (9), good (6), ok (3), and poor (1). Participants
vided criteria. The prepopulated morphological chart was used to using other values were not excluded and these nonstandard val-
control the variability of possible creativity within the students. ues were accepted as is. Participant ratings and criteria weightings
The only related activity tested here was the integration of means- were used in a decision matrix to determine final scores for each
to-form solutions. solution. From this matrix the top three integrated solutions,

Journal of Mechanical Design MAY 2012, Vol. 134 / 051004-5

Downloaded From: on 01/28/2016 Terms of Use:

Table 14 Experimental procedure

Experiment #1 Experiment #2
(burrito folder) (pet feeder)

Morphological hart training in advance Lecture on morphological charts presented class before the experiment
Morphological chart training on day of experiment Review of morphological charts presented at start of experiment
Time allowed to generate concepts 30 min
Number of concepts required to generate 8 10
Number of concepts chosen for judging 3 (or more if a tie existed) Exactly 3
Method for accommodating a tie in Evaluate all tied concepts. Participant score A set of exactly three concepts that included the
individual scoring of concepts determined using Eq. (3) highest ranked concepts was selected
Judge distribution system Concepts divided equally among five judges Concepts divided among two
judge panels of three judges
Judge evaluation method Evaluate the concepts using identical decision matrix and weightings as participants

defined and evaluated by participants, were collected for further the average of the top three solutions identified. In cases of no
analysis by judges. clearly identified third place, third place group scores were aver-
Participants were directed as to how to record and score their aged to create a composite third score. This score was averaged
chosen integrated conceptual design solutions. After the introduc- with first and second place scores (see Eq. 2).
tion, they were allowed 30 min to develop and evaluate their
design solutions, and instructed to form the highest quality con- solution3a þ solution3b
solution1 þ solution2 þ
ceptual combinations. The time limit for the concept generation score ¼ 2 (2)
phase was fixed by the need to conduct the experiment without 3
exceeding the allotted time. To choose the number of concepts for
the first experiment, participants generated and evaluated eight For the second experiment, three concepts were selected that
concepts in less than 30 min. Only one participant failed to gener- best represented the participant, including the three highest ranked
ate the full number of concepts; therefore, the second experiment concepts. Each judge ranked concepts using the same evaluation
required more concepts to encourage more novel solutions. levels as participants. The final participant scores were the aver-
age of nine scores, consisting of three judges’ scores for each of
the three selected concepts. The authors note that other evaluation
3 Analysis scales may be used with different results. This exploration of the
3.1 Jury. Each experiment employed a different system for role of evaluation scores and quality is beyond the scope of this
judging the integrated concepts; five individual judges or two study. A best practice baseline for creating a differentiated con-
groups of three individual judges. All judges were drawn from the cept set using the geometric scale was also used [30].
second graduate students in the CEDAR (Clemson Engineering The choice of evaluation criteria and their respective weights
Design Application and Research) Lab. To limit the time invested influenced the integrated concepts created by the participants. To
by each individual judge, results were divided amongst judges for ensure that all participants were working toward a common goal,
each experiment. In the first experiment, each judge was assigned they were required to score concepts against these criteria instead
approximately 60% of the concepts generated from each chart. of a personal preference. This experiment is used to study only the
Each integrated concept was also evaluated by three of the five process of exploring the design space to generate integrated con-
judges. While cumbersome, a uniform distribution of integrated cepts. While evaluation of the integrated concepts is not the focus
concepts among the judges was achieved. Intrarater consistency of this experiment, creating a consistent set of criteria for the par-
found no bias in judging [29]. ticipants and judges reduced the variation between what partici-
The second experiment used a simpler method for assigning pants identified as the best-integrated concepts without these
integrated concepts to the six judges. The collection of integrated specific criteria.
concepts was divided in half, with each evaluated by a three judge
panel working individually. While easier to establish, it was nec-
essary to elucidate any consistency between the two judging pan- 4 Results
els to determine the validity of combining evaluations from both
4.1 First Experiment Results. Of thirty participants in the
panels. An insufficient overlap in one morphological chart size
first experiment, four misunderstood the directions, creating their
prevented a full intra-rater analysis. The use of a system similar to
own means for each function instead of making selections from
the first experiment to spread the concepts evenly among judges
the morphological chart. They were excluded from the analysis.
may have mitigated the impact of differences between panels.
Another failed to rank all of their integrated conceptual design
Care must be taken when assigning evaluations with the ideal
solutions and was excluded also. As such, it can be inferred that
having each judge evaluate all integrated concepts. The logistical
there was ample time to complete tasks and that the scope of the
concern here is that a corresponding growth in the number of so
design problem was acceptable.
evaluations requires each judge to make that many more. Any
At the end of the exercise, all handouts were collected. The
assignment of evaluations among judges must minimally impact
three highest rated design solutions from each participant were
statistical analysis of the results. The additional effort to create
combined into a list of design solutions for evaluation. Absent no
evaluations in the first experiment eliminated the need to deter-
clear group of three or if there were a tie for third place, extra
mine if the judging could be pooled for statistical analysis, while
design solutions were added, and the list was purged of duplicate
reducing the uncertainty of the results.
solutions. This list of the three highest rated concepts covered
14% and 25% of the design spaces within the morphological
3.2 Score Calculation. Once the students had judged their charts (Table 15). The number of unique combinations generated
solutions, judges evaluated them accordingly. A final score was for the two charts were roughly equivalent (31 and 33). Further,
assigned by averaging the three scores provided by the panels, nearly half of student defined concepts were repeated by others in
which were used to evaluate participant-generated solutions. For each morphological chart (56% and 47%). The major difference
the first experiment, each participant was assigned the score from between the design space explorations was the actual coverage of

051004-6 / Vol. 134, MAY 2012 Transactions of the ASME

Downloaded From: on 01/28/2016 Terms of Use:

Table 15 Amount of design space evaluated by participants Table 17 First experiment quality score comparison

Morphological chart 35 53 Morphological chart #1 #2

Number of participants 12 13
Number of possible 125 243 Number of participants 12 13
Concept combinations Section 1 mean 59.9 51.1
Number of unique Quantity 55 70 Section 2 mean 59.3 46.0
Concept combinations % of possible concepts 44% 29% Combined mean 59.6 47.9
Generated by participants Standard deviation of combined group 11.8 10.9
Number of unique Quantity 31 33
Concept combinations % of possible concepts 25% 14%
In set of “top 3” % of generated concepts 56% 47%
4.2.2 Evaluation of Quality of Generated Concepts. Are the
Judging Panels Objective? One difficulty in evaluating engineer-
ing design methods regarding quality is subjectivity in measure-
the total concepts explored; only one-quarter of the concepts were ment, partially mitigated via judging panels of graduate students,
evaluated in the first chart and one-seventh were evaluated in the the results of which are shown in Table 19. Here, the values repre-
second. senting statistical analysis are not the geometric evaluation scores
Results from the two sections in the first experiment were of 1–3–6–9, but composite and averaged scores for complete con-
pooled. To ensure that mixing the class sections did not affect the cepts. Thus, scoring yields continuous scores, not the discrete and
analysis, an ANOVA statistical analysis was performed to deter- sparse scores of individual evaluations. Generated concepts were
mine the “p-value.” When the p-value was larger than a, the also provided to panels as simply integrated concepts and sorted
means of the analyzed groups were likely identical; when the randomly to avoid bias regarding slight increases in the level of
p-value was less than a, the means were identical [31]. Considera- detail in different number of functions. An unbalanced sample set
tion was an 80% confidence level, or a ¼ 0.20. Any p-value of of panel reviewed data was the result. As evident in Table 19,
0.10 or less indicated a statistically significant relation [32]. A while the means from both judge panels are not equal for each
p-value of greater than 0.10, but less than or equal to 0.20 indi- chart, the significant statistical overlap suggests a similarity if not
cated a likely relation [33]. equivalency between panels.
The means for the assessed quality of Chart #1 from each sec- Traditional student t-test and ANOVA was done under assump-
tion had a p-value of 0.93 (Table 16); the means for the assessed tions of normality (tested with t-test), independence, and equality
quality of Chart #2 from each section had a p-value of 0.44. The of variance. Therefore, further t-test and ANOVA is necessary to
total mean scores for Chart #1 and Chart #2 were not equivalent, determine statistical equivalency between panels. One performed
showing a significant difference in the quality of solutions that t-test assumed unequal variance, suggesting that the means were
each chart created. equal between the two judging panels for the 4  7 and the 8  5
The 3 function  5 mean morphological chart scoring produces chart sizes. The 4  5 chart size had nearly equal means, and simi-
higher quality concepts than the 5 function  3 counterpart, lar ANOVA results from the panel scoring for four different sized
Table 17, indicating that any improvement in the quality of morphological charts (Table 20). These results clearly show that
explored concepts must involve reducing the number of functions the judges evaluated only two of the four sizes of morphological
before reducing the number of means. charts identically (4  7 and 8  5). For the other two chart sizes,
one chart was evaluated inconsistently (4  5) whereas another
yielded inconclusive results due to a lack of judging overlap
4.2 Second Experiment Results
(8  7).
4.2.1 Quantity and Variety of Concepts Developed. Using the Table 21 shows the comparisons. First, all charts were com-
first experiment, a second experiment with 29 participants further pared with combined panels. Then, for cases in which one or both
explored the effect of function reduction on design space explora- panels lacked a p-value greater than a, the scores for judge panels
tion in morphological charts. As before, all handouts were col- #1 and #2 were compared. For cases relying upon the 8  7 size
lected. A set of the three highest rated design solutions were chart, no test was done with judge panel #1 autonomously, in that
identified, purged of duplicates, and combined into solutions for only one participant was evaluated by this panel at this size. The
evaluation. The list of the three highest rated concepts covered similarity of differences between the t-test and the ANOVA ne-
0.00024% to 1.3% of the design space (Table 18). cessitate further analysis exclusively with ANOVA. Note that
The numbers of unique concepts evaluated were similar for the very small sample sizes necessitate future inclusion of additional
four morphological chart sizes (19, 21, 24, and 18), encompassing a exercise replications with larger sample pools.
small percentage of combinations. Analysis of the concepts in vari- An examination of individual chart scores determined the fol-
ous configurations (18–24) found no significant improvement or lowing relationships for the relative concept quality generated
deficiency in ability to create different concepts (variety). Given from each morphological chart. In Fig. 1, the charts are in relative
identical time constraints, objectives, and charts with different in- position, with an axis of improving integrated conceptual design
formation, the designers explore the space similarly. From this quality from left to right. Vertical position in these figures is insig-
quantity and variety of concepts, design skills remain affected by nificant. Each chart is represented by a grouping of squares; height
the addition of means/functions to morphological charts regarding represents the number of functions and width represents the num-
the creation of unique concepts. This leaves the analysis of the ber of means. For example, the 4  7 morphological chart is four
impact on quality based on morphological chart content. squares tall and seven squares wide. Figure 1 shows the complete

Table 16 Summary of ANOVA results: comparing quality of means generated for 3 3 5 and 5 3 3 morphological charts

Groups p-value Evaluation

Chart #1 (3F  5M) Chart #2 (5F  3M) 0.018 Means are not equal
Section 1 Section 2 0.42 Means are equal
Chart #1 for section 1 only Chart #1 for section 2 only 0.93 Means are equal
Chart #2 for section 1 only Chart #2 for section 2 only 0.44 Means are equal

Journal of Mechanical Design MAY 2012, Vol. 134 / 051004-7

Downloaded From: on 01/28/2016 Terms of Use:

Table 18 Amount of design space evaluated by participants

Morphological chart size (F  M) 45 47 85 87 All 4 function All 8 function

Number of participants 8 7 8 6 15 14
Number of possible concept combinations 625 2401 390625 5764801 2401 5764801
Number of unique concept combinations Quantity 19 21 24 18 38 42
in set of “top 3” % of generated concepts 1.3% 0.29% 0.0020% 0.00010% 0.62% 0.00024%

Table 19 Second experiment raw participant scores as averaged within panels

Morphological chart size (F  M)

45 47 85 8 7

Number of participants 8 7 8 6
Judging panel #1 129.8 115 119.7 104.3
127.7 114.8 86.8
114.8 91.6 85.4
109.3 84.6 84.4
Judge panel #1 mean 120.4 101.5 91.1 104.3
Judge panel #1 std 9.94 15.73 16.24 NA
Judging panel #2 121.8 112.1 104.6 113.1
112.3 106.9 92.7 103.8
101.3 87.1 60.3 100.3
84.3 93.3
Judge panel #2 mean 104.9 102.0 85.9 100.6
Judge panel #2 std 16.1006 13.19141 22.92691 8.472131
Combined mean 112.7 101.7 89.1 101.2
Standard deviation of combined group 14.9 13.5 17.5 7.7

Table 20 Evaluation of judge panel scoring with t-test and ANOVA

Chart p-Value (t-test, two tail) p-value (ANOVA) Evaluation

45 0.16 0.15 Groups means might not equal

(Panels need to be examined separately)
47 0.96 0.96 Groups means are equal (Panels can be combined)
85 0.75 0.71 Groups means are equal (Panels can be combined)
87 N/A N/A Not evaluated due to lack of overlap between judge panels
(Panels need to be examined separately)

Table 21 Evaluation of morphological chart size relationships

Combined judges Judge panel #1 Judge panel #2

a b
Four functions p-value 0.16 0.09 0.81c
(4  5/4  7) Evaluation Means are not equal Means are not equalb Means are equalc
Five means p-value 0.01b 0.02b 0.25c
(4  5/8  5) Evaluation Means are not equalb Means are not equalb Means are equalc
Eight functions p-value 0.14a d
(8  5/8  7) Evaluation Means are not equala d
Means are equalc
Seven means p-value 0.93c d
(4  7/8  7) Evaluation Means are equalc d
Means are equalc
Rectangular p-value 0.11a d
(4  5/8  7) Evaluation Means are not equala d
Means are equalc
Nearly square p-value 0.15a e e

(4  7/8  5) Evaluation Means are not equala e e

0.10 < p  0.20.
p  0.10.
0.20 < p.
There were insufficient data for the 8  7 size to evaluate Judge panel #1.
It was unnecessary to evaluate individual judge panel relationships here. Panels were consistent for both sizes (see above).

set of notional relationships between the four morphological panels scored them consistently (Table 20), and they can be com-
charts. pared based upon combined scores. ANOVA comparison showed a
difference in the means of the two groups (Table 21). A comparison
4.2.3 Rectangular Morphological Charts. Consider the verti- of average scores for each chart shows that the vertical chart pro-
cal (4  7) and horizontal (8  5) charts above, because both judge duced higher scores than the horizontal chart. Therefore, reducing

051004-8 / Vol. 134, MAY 2012 Transactions of the ASME

Downloaded From: on 01/28/2016 Terms of Use:

does appear to be an improvement between the two sizes. There-
fore, removing means at a low level of functions may improve the
quality of concepts generated with morphological charts.
4.2.7 Morphological Charts With Seven Means. Consider the
two morphological charts that have seven means per function with
one chart having twice as many functions as the other (four and
eight functions). As previously indicated, both panels consistently
scored the 4  7 size chart while the 8  7 size lacked significant
data to determine the consistency of scoring. The ANOVA test
for combined groups showed that the means were identical
(Table 21), as with judge panel #2. Though both combined and
the individual judge panel evaluations match, there is no obvious
change between the two sizes. Therefore, adding functions at a
high level of means does not appear to change the quality of con-
cepts generated with morphological charts. Thus, there appears to
be a point where information reduction fails to influence posi-
Fig. 1 Morphological charts arranged by participants’ quality tively or negatively the quality of concepts generated.
scores as assessed by panel
4.2.8 Nearly Square Morphological Charts. Consider the
nearly square morphological charts of 4  5 and 8  7. As previ-
the number of functions and increasing the number of means pro- ously indicated, the 4  5 size morphological chart was not scored
duced higher quality concepts. consistently by the two judge panels. The 8  7 size morphologi-
cal chart had only a single participant evaluated by judge panel
4.2.4 Morphological Charts With Five Means. Consider two #1, so it was not possible to determine if the two judge panels
morphological charts with five means per function; one with twice evaluated this group consistently. A statistically significant
as many functions as the other (four and eight – Table 20). ANOVA comparison between the two sizes cannot be made only
Because the 8  5 size chart scored consistently between the two based on the combined judge panel scores. The ANOVA test for
panels and the 4  5 size did not, ANOVA comparison between the combined groups showed that the means were not the same, as
both is impossible from the combined judge panel scores. The is shown in Table 21. Judge panel #2 found no difference in chart
ANOVA test for the combined judge panels showed that the size. A statistically significant comparison cannot be made
means were not the same (Table 21). Means differed in charts for between these two sizes. Based on the average scores alone, there
judge panel #1 and were identical for judge panel #2. An average does appear to be an improvement between the two sizes. There-
of scores, shows no improvement between sizes. Therefore, reduc- fore, reducing the number of means and functions together may
ing the number of functions at a low level of means may improve improve the quality of concepts generated with morphological
the quality of concepts generated. This agrees with the inferences charts. In this manner, information reduction once again leads to
associated with the first experiment that reducing the number of easier exploration of the design space thereby resulting in higher
functions facilitates exploration of the design space. quality integrated solutions.

4.2.5 Morphological Charts With Eight Functions. Consider

the two morphological charts with eight functions but with different
5 Discussion
number of means per function (five and seven means). As previously The procedures for the two experiments are similar. The differ-
indicated, the 8  5 size morphological chart was scored consistently ences in procedure represent improvements made in an effort to
by the two judge panels while the 8  7 size did not have sufficient perform better analysis on the relationships between the number
data to determine if the two judge panels were consistent. A statisti- of means and the number of functions. The conclusions that can
cally significant ANOVA comparison between the two sizes cannot be jointly drawn from the two experiments are that (1) a chart
be made only based on the combined judge panel scores. The with more means than functions produces better concepts than
ANOVA test for the combined groups showed that the means dif- that of more functions than means and (2) adding functions to a
fered (Table 21). For judge panel #2, there was no difference morphological chart failed to improve results. It is important to
between chart sizes. A statistically significant comparison cannot be note that these conclusions are applicable to entry-level engineers
made between these two sizes. Based on the average scores alone, lacking design experience as opposed designers with significant
there does appear to be an improvement between the two sizes. experience. In addition, the scope of this research is limited to cer-
Therefore, adding means at a high level of functions may improve tain sizes of morphological charts and the conclusions drawn may
the quality of concepts generated with morphological charts. In this not be applicable to large-scale morphological charts or projects.
case, increasing the amount of information can improve the quality The hypothesis presented was that morphological charts that
of concepts generated. This runs contrary to the other findings where contained fewer possible integrated conceptual designs produce
information reduction leads to higher quality concepts. higher quality integrated concepts as this represents a smaller
design space to explore. From the experimental results, the hy-
4.2.6 Morphological Charts With Four Functions. Consider pothesis was not accepted in all cases. For example, during the
the two morphological charts with four functions but with differ- second experiment, the morphological chart with the most inte-
ent number of means per function (five and seven means). From grated conceptual designs (eight functions by seven means per
Table 20, one can see that the 4  7 size morphological chart was function) did not perform the worst. However, the shape of the
scored consistently by the two judge panels, while the 4  5 size morphological charts did impact the quality of concepts gener-
was not. A statistically significant ANOVA comparison between ated. Moreover, it appears that information reduction based on
the two sizes cannot be made based on the combined judge panel function yields higher quality concepts.
scores. The ANOVA test for the combined groups showed that the Two identified relationships have yielded better concepts;
means were not the same (Table 21). For judge panel #1, the charts with more means than functions and charts with fewer func-
means were also not found to be the same; however, for judge tions, the latter profoundly affecting what functions to include in a
panel #2, there was no difference found between the two chart morphological charts. By knowing that detailed functional decom-
sizes. A statistically significant comparison cannot be made position does enhance the selection of promising design solutions
between these two sizes. Based on the average scores alone, there from a morphological chart, designers can save time and effort

Journal of Mechanical Design MAY 2012, Vol. 134 / 051004-9

Downloaded From: on 01/28/2016 Terms of Use:

when decomposing the problem initially. Though results indicate [11] Aygun, M., 2000, “Comparative Performance Appraisal by Multiple Criteria
that reducing the number of functions improves the success of this for Design Alternatives,” Archit. Sci. Rev., 43, pp. 31–36.
[12] Bryant, C., Bohm, M., Stone, R., and McAdams, D., 2007, “An Interactive
task, the existence of a lower limit of the functions yielding prom- Morphological Matrix Computational Design Tool: A Hybrid of Two Meth-
ising integrated conceptual design solutions has yet to be clearly ods,” ASME Design Engineering Technical Conferences and Computers and
identified. Information in Engineering Conference, Las Vegas, NV.
Results show how the conceptual quality from participants [13] Bohm, M., Vucovich, J., and Stone, R., 2005, “Capturing Creativity: Using a
Design Repository to Drive Concept Innovation,” ASME Design Engineering
affects the morphological chart size and shape. The chart with Technical Conferences and Computers and Information in Engineering Confer-
which most students struggled was that with many functions and ence, Long Beach, CA.
few means per functions. As such, students should limit functional [14] Bryant, C., McAdams, D., Stone, R., Kurtoglu, T., and Campbell, M., 2006, “A
decomposition of a problem and conduct means generation when Validation Study of an Automated Concept Generator Design Tool,” ASME
Design Engineering Technical Conferences and Computers and Information in
using morphological charts during design. Two evaluation techni- Engineering Conference, Philadelphia, PA.
ques (judge panels and judge pools) were used to evaluate the [15] Bohm, M., Stone, R., and Szykman, S., 2003, “Enhancing Virtual Product Rep-
concepts of use in further study that builds on this research. resentations for Advanced Design Repository Systems,” ASME Design Engi-
neering Technical Conferences and Computers and Information in Engineering
Conference, Chicago, IL.
5.1 Design Guidelines. There are two requisite guidelines for [16] Tiwari, S., Teegavarapu, S., Summers, J., and Fadel, G., 2007, “Automating
morphological charts. Designers should expand the design space by Morphological Chart Exploration: A Multi-Objective Genetic Algorithm
expanding morphological chart width. They must then focus on the to Address Compatibility and Uncertainty,” Int. J. Product Dev., 9, pp.
primary and critical functions when constructing morphological [17] Weas, A., and Campbell, M., 2004, “Rediscovering the Analysis of
charts as their addition fails to improve the quality of the integrated Interconnected Decision Areas,” Artif. Intell. Eng. Des Anal. Manuf., 18, pp.
concepts. With these guidelines, and supporting experimental evi- 227–243.
dence, novice and student engineer designers, can effectively [18] Jin, Y., and Li, W., 2007, “Design Concept Generation: A Hierarchical Coevo-
lutionary Approach,” ASME J. Mech. Des, 129, pp. 1012–1022.
explore the qualitative design spaces within these charts. [19] Koch, P. N., Allen, J. K., Mistree, F., and Barlow, A., 1998, “Facilitating Con-
cept Exploration for Configuring Turbine Propulsion Systems,” ASME J.
5.2 Future Work. Future work entails improving concepts Mech. Des, 120, pp. 702–706.
[20] Barnum, G. J., and Mattson, C. A., 2010, “A Computationally Assisted Method-
generated with morphological charts. An understanding of the ology for Preference-Guided Conceptual Design,” ASME J. Mech. Des, 132,
influence on chart size on results can be used to help designers pp. 121003–121009.
using morphological charts develop better means for function and [21] Adelson, B., 1989, “Cognitive Research: Uncovering How Designers Design;
overall concept. The absence of predictability of study results in Cognitive Modeling: Explaining and Predicting How Designers Design,” Res.
Eng. Des., 1, pp. 35–429.
the literature requires creating surveys or interviews to create de- [22] Newell, A., 1990, Unified Theories of Cognition, First Harvard University
signer guidelines for morphological charts. Such survey and inter- Press, Cambridge, MA.
view results can elucidate improvements in the quality of [23] Koch, P., Allen, J., Mistree, F., and Mavris, D., 1997, “The Problem of Size in
concepts generated. Incorporating other design tools into morpho- Robust Design,” ASME Design Engineering Technical Conference, Sacra-
mento, CA, Papr No. DETC97DAC3983.
logical charts, specifically function structures, to improve the [24] Shah, J. J., Kulkarni, S. V., and Vargas-Hernandez, N., 2000, “Evaluation of
quality of generated concepts is currently being explored [25,26]. Idea Generation Methods for Conceptual Design: Effectiveness Metrics and
Design of Experiments,” ASME J. Mech. Des, 122, pp. 377–384.
[25] Richardson, J., Summers, J. D., and Mocko, G. M., 2011, “Function Represen-
References tations in Morphological Charts: An Experimental Study on Variety and Nov-
[1] Pahl, G., and Beitz, W., 1996, Engineering Design: A Systematic Approach, elty of Means Generated,” presented at the 21st CIRP Design Conference,
Springer-Verlag, New York. Daejeon, South Korea.
[2] Otto, K., and Wood, K., 2001, Product Design: Techniques in Reverse Engi- [26] Richardson, J., 2010, “Incorporating Function Structures into Morphological
neering and New Product Development, Prentice-Hall, NJ. Charts: A User Study,” MS Thesis, Clemson University, Clemson, SC.
[3] Ullman, D., 2002, The Mechanical Design Process, Pearson Education, Inc., [27] Ramachandran, R., 2012, “Understanding the Role of Functions and Interaction
Upper Saddle River, NJ. in the Product Design,” Clemson University, Clemson, SC.
[4] Pahl, G., and Beitz, W., 1966, Engineering Design: A Systematic Approach, [28] Ramachandran, R., Caldwell, B., and Mocko, G., 2011, “A User Study to Eval-
Springer-Verlag, New York. uate the Function Model and Function Interaction Model for Concept Gener-
[5] Dym, C., and Little, P., 2000, Engineering Design: A Project-Based Introduc- ation,” ASME Design Engineering Technical Conferences and Computers in
tion, John Wiley & Sons, Inc., New York. Engineering Conference, Washington, DC.
[6] Suh, N., 2001, Axiomatic Design: Advances and Applications, Oxford Univer- [29] Smith, G., 2007, “Morphological Charts: A Systematic Exploration of Qualita-
sity Press, New York. tive Design Space,” MS Thesis, Clemson University, Clemson, SC.
[7] Shah, J., 1998, “Experimental Investigation of Progressive Idea Generation Tech- [30] Pahl, G., Beitz, W., Feldhusen, J., and Grote, K. H., 2007, Engineering Design:
niques in Engineering Design,” ASME Design Engineering Technical Conferen- A Systematic Approach, 3rd. ed., Springer-Verlag, London.
ces and Computers and Information in Engineering Conference, Atlanta, GA. [31] Daniel, W., and Terrell, J., 1995, Business Statistics, Houghton Mifflin, Co Bos-
[8] Whiting, C. S., 1958, Creative thinking, Reinhold, New York. ton, MA.
[9] Ullrich, K., and Eppinger, S., 2000, Product Design and Development, [32] Ruder, J., and Sobek, D., 2007, “An Experiment on System Level Design,” J.
McGraw-Hill, Boston, MA. Eng. Design, 18, pp. 327–342.
[10] Teegavarapu, S., Snider, M., Summers, J. D., Thompson, L., and Grujicic, M., [33] Wetmore, W., Summers, J., and Greenstein, J., 2010, “Experimental Study of
2007, “A Driver for Selection of Functionally Inequivalent Concepts at Varying Influence of Group Familiarity and Information Sharing on Design Review
Levels of Abstraction,” J. Design Res., 6, pp. 239–259. Effectiveness,” J. Eng. Design, 21, pp. 111–126.

051004-10 / Vol. 134, MAY 2012 Transactions of the ASME

Downloaded From: on 01/28/2016 Terms of Use: