See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/282489172

Optimal difference test sequence and power for discriminating soups of

varying sodium content: DTFM version of dual-reference duo–trio with
unspecified tetrad tests

Article  in  Food Research International · June 2015

DOI: 10.1016/j.foodres.2015.06.019

CITATIONS READS

8 152

4 authors, including:

In-Ah Kim Hye-Seong Lee

Ewha Womans University Ewha Womans University
10 PUBLICATIONS   45 CITATIONS    73 PUBLICATIONS   1,279 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Cognitive decision strategies adopted in reminder tests View project

All content following this page was uploaded by Hye-Seong Lee on 17 February 2018.

The user has requested enhancement of the downloaded file.

 Food Research International 76 (2015) 458–465

Contents lists available at ScienceDirect

Food Research International

journal homepage: www.elsevier.com/locate/foodres

Optimal difference test sequence and power for discriminating soups of

varying sodium content: DTFM version of dual-reference duo–trio with
unspecified tetrad tests
In-Ah Kim, Hye-Lim Kim, Ha-Yeon Cho, Hye-Seong Lee ⁎
Department of Food Science and Engineering, College of Engineering, Ewha Womans University, Seoul 120-750, South Korea

a r t i c l e i n f o a b s t r a c t

Article history: As sodium reduction has become very important in the food industry, various types of unspecified duo–trio tests
Received 9 April 2015 have been studied to improve its efficiency for studying samples with high sodium content, and a constant-
Received in revised form 10 June 2015 saltier-reference duo–trio test with dual reference, one reference in the first position and the second reference in
Accepted 17 June 2015
the middle between the two test stimuli (DTFM), has been recommended. For the duo–trio test, a ‘comparison of
Available online 19 June 2015
distance’ (COD) strategy has been generally assumed. Yet, theoretically for DTFM, the 2-AFC reminder (2-AFCR)
Keywords:
τ-strategy is also possible, which would make DTFM more efficient than the unspecified tetrad test. In this study,
Sodium reduction the hypothesis was that when subjects are pre-exposed to two types of samples, the 2-AFCR τ-strategy can be
Sensory difference test adopted in a constant-reference DTFM using a fixed design experiment. In order to test this hypothesis, unspecified
Duo–trio tetrad tests involving categorization tasks were used as a means of pre-exposure to the two types of samples for
2-AFC DTFM, and a performance comparison was conducted. Two groups of 39 untrained/naive subjects performed
Unspecified tetrad both the unspecified tetrad and DTFM tests in varying orders for the purpose of discriminating two different
Optimal cognitive decision strategy soup samples of varying sodium content. A comparison of the d′ estimate across different methods supported the
hypothesis that the more efficient 2-AFCR τ-strategy was appropriate when the tetrad test preceded DTFM,
while when DTFM was performed first without pre-categorization of samples, the conventional duo–trio COD
strategy was appropriate for the constant-reference DTFM.
© 2015 Elsevier Ltd. All rights reserved.

1. Introduction
Sensory difference tests have played increasingly important roles in
the food industry. Food researchers have applied sensory difference
tests to evaluate the differences between confusable food products
for various business objectives including cost reduction and reformula-
tion. One of the major purposes for the latter is to reduce sodium
content in foods and beverages in order to produce healthier products.
Regarding the efficiency and robustness of these difference tests, evalu-
ating the sensory differences elicited from products of varying degrees
of high sodium content has been a challenge due to the sensory fatigue
and adaptation that saltiness easily causes, leading to considerable
problems in accurate analysis of sensory differences (Lee, Chae, & Lee,
2009; Lee & O'Mahony, 2007; O'Mahony & Goldstein, 1987; O'Mahony
& Odbert, 1985; Tedja, Nonaka, Ennis, & O'Mahony, 1994; Vié &
O'Mahony, 1989).
In the study of reformulation, unspecified sensory difference test
methods, including the triangle test (ASTM, 2004; ISO, 2004a; Peryam &
Swartz, 1950), the duo–trio test (ISO, 2004b; Peryam & Swartz, 1950),
the same–different test (Macmillan & Creelman, 2005), and the
⁎ Corresponding author.
E-mail address: hlee@ewha.ac.kr (H.-S. Lee).
unspecified tetrad test (Ennis & Jesionka, 2011; Masuoka, Hatjopoulos,
& O'Mahony, 1995), have been generally recommended. Yet, when used
for discriminating savory samples of reasonably high sodium content
using untrained/naive subjects, most of these methods have been
found to be susceptible to the operational hindrances caused by phys-
iological adaptation and cognitive problems. Problematic sequence ef-
fects have been reported for the triangle test (Kim, Chae, van Hout, &
Lee, 2014), the duo–trio test (Choi, Kim, Christensen, van Hout, & Lee,
2014; Kim & Lee, 2012; Kim, Lee, & Lee, 2010; Kim, Sim, & Lee, 2015),
and the same–different test (Choi et al., 2014) based on the balanced-
sample designs using all possible sample presentation sequences: i.e.
subjects performed better for some particular test sequences than
others. Sample presentation sequence effects have not yet been studied
for the tetrad tests.
In an attempt to identify the method with optimal test sequences for
studying taste equality in samples with reduced sodium content, vari-
ous versions of duo–trio tests have been actively studied (Choi et al.,
2014; Kim & Lee, 2012; Kim et al., 2010, Kim et al., 2014, Kim et al.,
2015; Lee & Kim, 2008). In these studies, without showing the sequence
effects when the duo–trio test was used as constant-reference mode,
subjects were found to perform better either using the saltier sample
as the reference (Choi et al., 2014; Kim & Lee, 2012; Kim et al., 2010)
or preferred one as the reference (Kim et al., 2014, Kim et al., 2015).

http://dx.doi.org/10.1016/j.foodres.2015.06.019
0963-9969/© 2015 Elsevier Ltd. All rights reserved.
 I.-A. Kim et al. / Food Research International 76 (2015) 458–465
Accordingly, for the evaluation of sodium-reduced samples against the
original high-sodium product, two versions of duo–trio tests using a
saltier sample as the constant-reference were recommended: the tradi-
tional duo–trio with a reference presented first (DTF) and the duo–trio
with the reference presented both first and in the middle between the
two alternative test samples (DTFM) (Choi et al., 2014; Kim & Lee,
2012). The effects of experimental conditions on these methods need
to be further studied to standardize and use these methods accurately
based on an appropriate model.
Signal detection theory (SDT) (Green & Swets, 1966; Macmillan &
Creelman, 2005) and Thurstonian modeling (Ennis, Rousseau, & Ennis,
2014) have been widely applied in difference tests in recent sensory
science fields. In these approaches, different psychometric models,
each assuming a certain cognitive decision strategy, have been devel-
oped for accurate discrimination analysis. Defining test methods and
constructing the experimental design appropriately are important factors
to validate such theoretical models. If the decision strategy used for a test
method in practice is discrepant from the generally assumed decision
strategy for a standard model for the test method, the analysis of the
results using the standard model can lead to wrong business decisions.
For DTF and DTFM, generally a ‘comparison of distance’ (COD) strategy
has been assumed when using untrained/naïve subjects (Choi et al.,
2014; Kim & Lee, 2012), for discriminating complex and not easily
describable sensory differences between samples. However, for the
constant-saltier-reference DTF and DTFM, it is needed to determine
whether a different cognitive decision strategy could be used for the
test depending on experimental conditions and test procedures. Since
the duo–trio COD strategy lacks statistical power, developing a model
with a standardized testing procedure that uses a more powerful cogni-
tive strategy and less affected by sequence effect, would be a suitable
way forward to increase economic efficiency.
Regarding the sample presentation sequences, the constant-saltier-
reference DTF is equivalent to the 2-AFC with reminder (2-AFCR) method
(Hautus, van Hout, & Lee, 2009; Lee, van Hout, & Hautus, 2007). Although
the unspecified tetrad test, for which a sorting process is assumed
no matter which of the β/τ-strategy might be involved in the decision
process, is theoretically most powerful among the unspecified sensory
difference test methods, its theoretical power is lower than the specified
tetrad and the 2-AFC method (Ennis, 1993; Ennis & Jesionka, 2011;
Garcia, Ennis, & Prinyawiwatkul, 2013). Based on SDT, 2-AFCR is
a variant of 2-AFC method assumed to be the most efficient, i.e.
more powerful than both specified and unspecified tetrad tests.
For the 2-AFCR method in SDT, either the 2-AFC β/τ-strategy or the 2-
AFCR τ-strategy were proposed (Hautus et al., 2009), and recently,
these models have been confirmed by using trained subjects who
evaluated the samples over several days to discriminate various simple
food systems in a fixed experimental design (Stocks, van Hout, &
Hautus, 2013, Stocks, van Hout, & Hautus, 2014; van Hout, Hautus, &
Lee, 2011).
In the same line, the constant-saltier-reference DTFM can be consid-
ered to have the same test sequence and procedure as the 2-AFC with
dual reminder (2-AFCDR). Yet, unlike DTF where the two test samples
are tasted sequentially, for DTFM, the reference is tasted before each
test sample. Thus, for DTFM, the 2-AFCR τ-strategy is also possible, as-
suming that subjects can learn to detect the difference between samples.
Thus, this study focuses on different possible cognitive decision strategies
of DTFM to investigate whether this method can be an alternative to the
unspecified tetrad test.
The goal of this study was to test whether untrained/naïve subjects
can use a most efficient 2-AFCR strategy for the constant-saltier-
reference DTFM in a 2-day experiment (not requiring several days of
training) for discriminating two different soup samples of varying sodi-
um content. It was hypothesized that when subjects are pre-exposed to
two types of samples, without extended training, the 2-AFCR τ-strategy
can be adopted in a constant-reference DTFM using a fixed design
experiment where only a pair of samples (two types of samples) is
459
discriminated. This hypothesis is based on the subjects' ability to learn
the sign of the sensory difference between samples and to use this infor-
mation in their decision process. Although theoretically the 2-AFC β/τ-
strategy is also possible for such a constant-reference DTFM, in this
study, as the most plausible 2-AFCR strategy for the DTFM, the 2-AFCR
τ-strategy was considered because the two test samples are not tasted
in comparative manner in DTFM and the 2-AFCR τ-strategy predicts
the same level of performance as the 2-AFC β/τ-strategy. In order to
test this hypothesis, unspecified tetrad tests involving categorization
tasks were used as a means of learning to identify sensory evidence
for differences between samples. This tetrad test was also performed
after DTFM, and the performance of DTFM in this case, without pre-
exposure to the sample classes, was compared with the performance
of the tetrad tests by utilizing two groups having equivalent
sensitivity based on an independent samples design. In order to create
the two subject groups, balanced-reference DTF was performed, and
results obtained from DTF using the saltier-reference were compared
to the DTFM and the tetrad methods. Thus, this study examines
sequence effects in DTF, unspecified tetrad tests, and the constant-
saltier-reference DTFM for the discrimination of savory soup samples
of high sodium content.
2. Materials and methods
2.1. Samples
Three corn soup samples with different salt concentrations were
prepared and served as 20 ml aliquots in 50 ml glass beakers. Sample
A was the original corn soup (Cup soup-corn cream flavor, Ottogi Co.
Ltd., Pyeongtaek, Gyeonggi-do, South Korea), sample B was the original
corn soup, but with 0.08% of more salt (Hanju Co. Ltd., Ulsan, South
Korea) content of original salt content, while sample C was the original
corn soup, but with 0.15% of more salt content of original salt content. All
samples were stored at constant temperature (53 ± 0.5 °C) in a thermos,
and they were presented on plate warmers to prevent cooling.
2.2. Subjects
Seventy-eight female students (age range: 21–26 years) were re-
cruited from Ewha Womans University in Seoul, South Korea. They
were untrained/naïve consumer subjects with pervious experiences in
consuming corn soup and had an interest in attending food sensory
testing. They were instructed not to eat any food, drink anything except
water, use mouthwash, or wear strong fragrant cosmetics and perfumes
for at least one hour before testing. After testing, all subjects were com-
pensated for their participation.
2.3. Experimental design and test procedures
The experimental design included three unspecified difference tests:
balanced-reference DTF, constant-saltier-reference DTFM, and the
unspecified tetrad test. Information regarding these test methods is
summarized in Table 1. The test performance of these three difference
tests were compared throughout the 2-day experiment design and the
overall experimental design is described in Fig. 1. The subjects were
divided into two groups based on the results of the initial sensitivity
test performed on Day 1. Balanced-reference DTF was first conducted
with 12 replicated tests for the ‘A–C’ sample pairs to check the baseline
of untrained/naïve subjects' sensitivity over two sub-sessions. The ‘A–C’
sample pairs were used instead of the ‘A–B’ sample pairs that was used
for the main test, in order to protect the subjects from becoming familiar
with the reference sample for DTFM to be investigated in the main test
on Day 2. Yet in order to be able to compare test performances of
the balanced-reference DTF and the constant-reference DTFM methods,
the DTF method was also conducted with 4 replicated tests for the ‘A–B’
sample pairs. Again, to minimize the learning effects that caused from
460 I.-A. Kim et al. / Food Research International 76 (2015) 458–465

Table 1
Description of the three different unspecified test methods used in this study.

Method Instruction No. of possible No. of Type of cognitive strategy

presentation stimuli assumed in SDT model
sequences (g)a in a test

Balanced-reference DTF Which sample was the same as the reference (R) presented first? g = 4; 3 Duo–trio ‘comparison of
① bAref-A-BN, distances’ (COD) strategy
② bAref-B-AN,
③ bBref-A-BN,
④ bBref-B-AN
Constant-saltier-reference Which sample was the same as the reference (R), presented before each test sample? g = 2; 4 ① Duo–trio COD strategy
DTFM ① bBref-A, Bref-BN, ② 2-AFCR τ-strategy
② bBref-B, Bref-AN
Unspecified tetrad There are four samples — two from one group and two from another. g = 6; 4 Categorization (sorting) based
Sort them into two groups of two based on similarity. ① bA-A-B-BN, on unspecified COD strategy
② bA-B-A-BN,
③ bA-B-B-AN,
④ bB-B-A-AN,
⑤ bB-A-B-AN,
⑥ bB-A-A-BN
a
Three different samples of corn soup of different salt content as follows; A: original corn soup; B: corn soup with 0.08% of more salt content of original salt content; C: corn soup with
0.15% of more salt content of original salt content.

the repeated exposure, only 4 tests were performed in this third sub-
session. All the untrained/naïve subjects took a 10 min break to reduce
fatigue and sensory adaptation between sub-sessions.
On Day 2, at least a week after Day 1, the constant-saltier-reference
DTFM (with sample B as reference) and the unspecified tetrad test
method were conducted for the ‘A–B’ sample pairs. DTFM presented
two references, one occurring first and the other occurring in the middle
between two test samples. Subjects were informed that the two refer-
ences remained constant throughout the replicated tests. Group 1 partic-
ipated in 6 unspecified tetrad tests first and 6 tests of DTFM later. The
reverse order was performed for Group 2. Both groups also participated
in 12 triangle tests using a set of visual stimuli between the two sub-
sessions during which the unspecified tetrad test and the DTFM methods
were conducted. In the visual triangle test, subjects were asked to choose
one different picture among three pictures presented on a computer
screen to reduce carry-over effects from the previous experimental
session on the following test session. Five-minute breaks were allowed
before and after the visual triangle tests.
The main goal of the experimental design was to investigate the
effect of the test procedures on the cognitive decision strategy used
for DTFM. The hypothesis for the different cognitive decision strategy
for DTFM before and after the unspecified tetrad tests (i.e. the effect of
the test procedures in DTFM which was the major interest in this
paper) was tested using an independent samples design. Thus, the per-
formance of DTF using a COD strategy and the unspecified tetrad test
using a sorting where samples might be compared based on either a β
or τ strategy, could act as comparisons to validate the SDT models as-
suming the two different decision strategies.
2.4. Data analysis
In the present study, the d′ estimate was used to compare test perfor-
mance across the three different unspecified test methods: balanced-
reference DTF, constant-saltier-reference DTFM, and the unspecified tet-
rad test. The d′ estimate was computed from the pooled data within
each group, reflecting the subjects' group discriminability under signal
detection theory/Thurstonian modeling. For the analysis of the DTF, the
conventional duo–trio model assuming the ‘comparison of distances’
(COD) strategy was used. For the analysis of DTFM, the duo–trio model as-
suming the COD strategy as well as the 2-AFC model assuming the 2-AFCR
τ-strategy were used to investigate the effects of pre-exposure to the two
types of the samples on the cognitive decision strategy used for DTFM
after the tetrad tests. The d′ values obtained from the different test
methods were compared based on two-sided 95% confidence intervals
computed based on the likelihood root statistic. Data analysis was con-
ducted using the R-package sensR (Christensen & Brockhoff, 2015),
which is freely available for the statistical software package R (R Core
Team, 2014).
3. Results
3.1. Group discrimination results obtained from the initial sensitivity test on
Day 1
In order to check the baseline discrimination performance of
untrained/naïve subjects to be used as independent samples for compar-
ison in the main test, the results obtained from the balanced-reference

Fig. 1. The overall experimental design. Number of tests performed per subject was noted for each sub-experimental session.
 I.-A. Kim et al. / Food Research International 76 (2015) 458–465 461

Table 2 2012), the results showed the same trend between the d′ estimate ob-
Estimate of d′ (95% CI) for the results obtained from the balanced-reference DTF, used as tained from the DTF test using a weaker-reference (d′ = 0.38, with
the initial sensitivity test.a
95% confidence intervals 0.00–1.29) and the DTF using a saltier-
Sample pair Test sequence Group 1 Group 2 reference (d′ = 1.52, with 95% confidence intervals 0.77–2.15).
‘A–C’ pair bAref-A-CN 1.47 (0.88–1.98) 1.42 (0.81–1.93)
bAref-C-AN 1.32 (0.66–1.83) 1.10 (0.20–1.64)
bCref-A-CN 1.42 (0.81–1.93) 1.52 (0.94–2.03)
3.2. Group discrimination results obtained from the main test on Day 2:
bCref-C-AN 1.77 (1.23–2.28) 1.67 (1.12–2.18) determination of the cognitive decision strategy for DTFM using different
Pooled (n = 468)b 1.49 (1.23–1.75) 1.43 (1.16–1.69) experimental procedures
‘A–B’ pair bAref-A-BN 1.21 (0.00–2.13) 1.52 (0.00–2.42)
bAref-B-AN 0.00 (0.00–0.98) 1.05 (0.00–1.98)
In the main test, the constant-saltier-reference DTFM and the
bBref-A-BN 1.21 (0.00–2.13) 0.87 (0.00–1.85)
bBref-B-AN 1.82 (0.77–2.75) 1.82 (0.77–2.75) unspecified tetrad tests were performed in different orders for the two
Pooled (n = 156)c 1.05 (0.30–1.53) 1.33 (0.79–1.78) experimental groups for discriminating the ‘A–B’ sample pair: Group 1
a
Two-sided 95% confidence interval for the d′ estimate was computed based on the
participated in the treatment condition that was hypothesized to use
likelihood root statistic. the alternative 2-AFCR τ-strategy, potentially leading to higher test
b
Total observations for each test sequence = 39 subjects × 3 replications = 117; total power and efficiency of DTFM than the conventional duo–trio test or
observations for pooled data over all test sequences = 39 subjects × 3 replications × 4 the unspecified tetrad test. To compare test performance across these
test sequence = 468.
c three methods, the d′ estimates obtained from these three methods are
Total observations for each test sequence = 39 subjects × 1 test = 39; total
observations for pooled data over all test sequences = 39 subjects × 4 test shown in Table 3. For the conventional duo–trio, DTF, only the results of
sequence = 156. the saltier-reference DTF were used to make fair comparisons between
DTF and DTFM. According to the SDT and the Thurstonian model, for
the discrimination of the same set of samples using the same subjects,
all methods would result in the same d′ estimates if the model assump-
DTF, used as the initial sensitivity test, were examined for the two groups
tion for cognitive decision strategy is appropriate. Confirming this theory,
(Table 2). Based on the pooled data, no significant difference was ob-
the unspecified tetrad test for both groups resulted in similar d′ estimates
served between the two groups for both ‘A–C’ and ‘A–B’ sample pairs,
for the results of DTF. Based on this theory, it was possible to identify
confirming that the two independent experimental groups used in this
which model was appropriate for each group, assuming different cogni-
study had equivalent group discriminability.
tive decision strategies for the constant-saltier DTFM, by comparing d′ es-
Results were analyzed for the segregated data of each possible test
timate results to the other two methods. As shown in Table 3, for Group 1,
sequence to confirm the sequence effects that occurred by the nature
DTFM assuming the 2-AFCR τ-strategy resulted in a d′ estimate that was
of the reference sample (weaker-reference vs. stronger-reference) and
equivalent to values obtained from DTF and the unspecified tetrad test,
shown in Table 2. For the ‘A–C’ and ‘A–B’ sample pairs, samples ‘C’ and
while for Group 2, DTFM assuming the duo–trio COD strategy resulted
‘B’ were the stronger (saltier) of each pair, respectively. d′ estimates
in a d′ estimate that was equivalent to the values obtained from the
were compared based on 95% confidence intervals across the different
other methods, confirming the hypothesis. Comparing the d′ estimates
test sequences, in correspondence with the previous studies (Choi
and their 95% confidence intervals determined for DTFM and DTF with
et al., 2014; Kim & Lee, 2012). When sample ‘A’ was discriminated
those obtained from the unspecified tetrad test in both groups, it was
from sample ‘C’, the sample with the highest salt concentration, there
found that the lower confidence intervals of d′ estimates determined
was no difference. However, when sample ‘A’ was discriminated from
for DTFM and DTF were higher than the unspecified tetrad test. This
sample ‘B’, some discrepancy was observed. Namely, when looking
suggests that regardless of the decision strategy used, the two versions
at the data of each test sequence, d′ estimates were quite similar for
of constant-saltier-reference duo-trio tests — DTF and DTFM — were
‘A–C’ sample pairs across difference test sequences, ranging from 1.32
not inferior to the unspecified tetrad test.
to 1.77 for Group 1 and from 1.10 to 1.67 for Group 2, and all with a
higher value than zero based on 95% confidence intervals. In contrast,
for the ‘A–B’ pairs, the saltier-reference test sequence, b Bref-B-A N, 3.3. Sequence effects of the DTFM and the unspecified tetrad test
showed larger d′ estimates for both groups, and in only this case, they
were higher than zero based on 95% confidence intervals. On the According to appropriate model assumptions for the cognitive deci-
other hand, for the ‘A–B’ pairs in Group 1, the weaker-reference test se- sion strategies for DTFM determined in Section 3.2, results were also
quence, b Aref-B-AN, showed approximately chance-level performance analyzed for the segregated data of each possible test sequence to inves-
(d′ = 0.00, Pc = 0.41). Due to these sequence differences, for Group tigate sequence effects for the constant-saltier-reference DTFM and the
1, when the four test sequences were divided into two sets of tests unspecified tetrad tests (Table 4). As seen in Table 4, the two test
according to the nature of the reference (weaker-reference vs. stronger- sequences for DTFM resulted in equivalent d′ estimates for both groups,
reference) as reported in previous studies (Choi et al., 2014; Kim & Lee, based on the total observation number (sample size), n = 117, although

Table 3
Estimate of d′ (95% CI) a for the results obtained from the saltier-reference DTF, the constant-saltier-reference DTFM, and the unspecified tetrad methods for discriminating ‘A–B’ sample
pair (n = 234).b

Methods SDT/Thurstonian model Group 1 Group 2

(Tetrad → DTFM) (DTFM → Tetrad)

Saltier-reference DTF Duo-trio COD strategy model 1.52 (1.13–1.88) 1.37 (0.96–1.73)
Constant-saltier-reference DTFM Duo–trio COD strategy model 2.48 (2.09–2.87) 1.49 (1.10–1.85)c
2-AFCR τ-strategy model 1.27 (1.00–1.55)c 0.59 (0.35–0.84)
Unspecified tetrad Unspecified categorization (sorting) model 1.16 (0.92–1.38) 1.16 (0.92–1.38)
a
Two-sided 95% confidence interval for the d′ estimate was computed based on the likelihood root statistic.
b
Total observations for each method = 39 subjects × 6 test sequence = 234; for the saltier-reference DTF, the variance of d′ estimates were re-computed according to the proportion of
correct responses shown in Table 1, assuming N = 234.
c
For the DTFM, d′ estimate in bold is equivalent value to d′ estimate for both DTF and tetrad methods, considering the 95% confidence interval for the d′ estimate.
462 I.-A. Kim et al. / Food Research International 76 (2015) 458–465

Table 4 modified data analysis is possible according to the determined cognitive

Estimate of d′ (95% CI)a for all possible test sequences in the constant-saltier-reference decision strategy.
DTFM and the unspecified tetrad methods for discriminating ‘A–B’ sample pair, based on
the appropriate SDT/Thurstonian model examined in Table 3.
In addition, it is critical to optimize the different test methods in a
way less susceptible to operational hindrances, such as problematic sen-
Method Test sequence Group 1 Group 2 sory adaptation and memory loss, in order to be as efficient as the theory
(Tetrad → (DTFMc →
predicts. This is because the predictions using the SDT or Thurstonian
DTFMb) Tetrad)
approaches are tied to the precondition of no deficiency in the clarity
Constant-saltier-reference bBref-B, Bref-AN 1.39 (0.99–1.81) 1.82 (1.29–2.34) of sensory information transformed to the human brain and perfect
DTFM bBref-A, Bref-BN 1.17 (0.78–1.55) 1.16 (0.37–1.69)
memory for employing a cognitive decision strategy (Lee & O'Mahony,
Pooled (n = 234)d 1.27 (1.00–1.55) 1.49 (1.10–1.85)
Unspecified tetrad bB-B-A-AN 1.07 (0.28–1.62) 1.41 (0.82–1.95) 2007).
bA-A-B-BN 1.07 (0.28–1.62) 0.98 (0.00–1.54) In this context, this paper presented and confirmed the hypothesis
bB-A-B-AN 1.33 (0.72–1.87) 0.98 (0.00–1.54) for the most efficient SDT or Thurstonian model (2-AFC model) for a
bA-B-A-BN 1.33 (0.72–1.87) 0.98 (0.00–1.54) modified version of a duo–trio test, with a reference presented both
bB-A-A-BN 1.24 (0.60–1.78) 1.24 (0.60–1.78)
bA-B-B-AN 0.88 (0.00–1.46) 1.33 (0.72–1.87)
first and between the two alternative test samples (DTFM), using a
Pooled (n = 234)e 1.16 (0.92–1.38) 1.16 (0.92–1.38) constant-saltier-reference, for discriminating two different savory
a soup samples of high salt content. The samples were prepared to vary
Two-sided 95% confidence interval for the d′ estimate was computed based on the
likelihood root statistic. in salt content to simulate sodium reduction experiments in business
b
d′ estimates for DTFM of Group 1 were computed assuming the 2-AFCR τ-strategy situations. Although the differences in sample formulations were only
model. at the level of salt content, salt level can greatly affect overall sensory
c
d′ estimates for DTFM of Group 2 were computed assuming the duo–trio COD strategy
characteristics and is not limited to effects on saltiness or overall taste
model.
d
Total observations for each test sequence = 39 subjects × 3 replications = 117; total
intensity in food (Keast & Breslin, 2002). The test procedures required
observations for pooled data over all test sequences = 39 subjects × 3 replications × 2 for transforming this unspecified DTFM test to the specified 2-AFCR
test sequence = 234. method having the optimal test sequence as well as the highest test
e
Total observations for each test sequence = 39 subjects × 1 test = 39; total observations power were 1) the pre-tests of samples using the unspecified tetrad
for pooled data over all test sequences = 39 subjects × 6 test sequence = 234.
tests prior to DTFM and 2) keeping a saltier sample constantly as a
reference for DTFM. Other experimental condition used to confirm the
the d′ estimates for the test sequence bBref-B, Bref-AN tended to be slightly present results was a fixed experimental design where only a sample
bigger than the test sequence bBref-A, Bref-BN. For the tetrad test, similar pair was tested.
to the balanced-reference DTF, there was some discrepancy in the d′ esti- When using untrained/naïve consumer subjects for measuring subtle
mates across the different test sequences. In this case, the total observa- sensory differences between samples, it is generally recommended to use
tion number (sample size) of the unspecified tetrad test and the DTF an unspecified difference test, not having to describe what constitutes the
was n = 39 for both, and thus it is possible to compare each. Based on difference. When explicitly specified difference tests are used for food dis-
95% confidence intervals, for the tetrad test, only the test sequences crimination, it is possible for untrained/naïve consumer subjects to focus
bA-B-B-AN of Group 1 and bA-A-B-BN, bB-A-B-AN, and b A-B-A-BN of on different sensory attributes with concept confusion during repeated
Group 2 were not different from zero, indicating that more errors tests, leading to higher rates of response error (McClure & Lawless,
occurred in these sequences than in other test sequences (Table 4). 2009, 2010). However, statistically, specified difference test methods re-
quiring explicit specification or description of sensory evidence to be
4. Discussion identified by the subject are more powerful than unspecified methods
(cf. Bi, Lee, & O'Mahony, 2010; Bi, Lee, & O'Mahony, 2010, 2014; Byer &
In food sensory science, the application of signal detection theory Abrams, 1953; Ennis, 1990, 1993; Ennis & Jesionka, 2011; Frijters,
(SDT) or Thurstonian modeling approaches has been broadly accepted 1979). Thus it is useful to establish a standardized test procedure and ex-
for the analysis of various sensory difference tests as a useful measure- perimental conditions that can successfully transform the unspecified
ment science (Ennis et al., 2014; van Hout, 2014). Using this approach is duo–trio to the specified 2-AFC model required to implicitly define the
advantageous because it takes into account the use of different cognitive decision space based on which untrained/naïve subjects draw their re-
decision strategies adopted for different experimental designs and the sponses. This can be done by adopting the more powerful 2-AFC strategy
formats of difference tests. Analysis based on this approach should re- without explicitly describing the sensory difference. In this paper, 6 re-
sult in absolute indices, d′ estimates, indicating the degree of sensory peated tests of the unspecified tetrad tests were found to be sufficient
differences between samples, independent of the test methods used to for this transformation in a one-day experiment. This DTFM version of
discriminate them. However, for such d′ estimates, to be computed ac- the constant-reference duo–trio method assuming the 2-AFC with
curately and used as an absolute measure in business decision-making reminder (2-AFCR) τ-strategy thus could be an alternative method to
with low error (i.e. product reformulations, ingredient changes, etc.), the unspecified tetrad which has been most advocated due to its high
the cognitive decision strategy for the difference test assumed in each test power (Ennis, 2012; Ennis & Christensen, 2014; Ennis & Jesionka,
SDT or Thurstonian analysis should be appropriate. Therefore, in order 2011; Garcia, Ennis, & Prinyawiwatkul, 2012; Garcia et al., 2013; Ishii,
to validate the SDT or Thurstonian approaches in sensory discrimination O'Mahony, & Rousseau, 2014; Rousseau & Ennis, 2013). In this study, in
analyses in the food industry and academia, empirical research investi- order to make performance comparisons across DTFM, DTF and the un-
gating the cognitive decision strategy used for the different types of specified tetrad test methods using a related samples design, all possible
difference tests in different experimental designs with procedural test sequences were tested for each method. Thus, 6 tests were used for
variations is needed. In practice, in order to meet the various business the tetrad pre-tests. Further research should be done to find the mini-
objectives, often it is necessary to make some modifications to test pro- mum number of repeated tetrad tests that could work as pre-tests for
cedures and experimental designs, to use the difference test method the DTFM method, satisfying the pre-requisite for adopting the 2-AFCR
more effectively, dealing with the given practical constraints. Thus, re- τ-strategy.
searchers need to be aware of the consequences of the modifications When conducting the sensory difference test, it has not been a com-
of test procedures on the use of the cognitive decision strategy adopted mon practice to use two types of difference test methods in one exper-
for the test methods. Based on the information of subjects' decision pro- iment. However, for using untrained/naïve consumer subjects, it could
cess, researchers need to optimize the modifications to test procedures be beneficial to use at least two methods as a comparison to validate
in such a way that subjects' cognitive decision strategy is known and the the accuracy of the test performance and the appropriateness of the
 I.-A. Kim et al. / Food Research International 76 (2015) 458–465
SDT or Thurstonian model used for the analysis. If the experiment does
not include a comparison (control) method, it would be impossible to
examine whether or not some subjects failed to perform the given
method or used different decision strategy. When there is a reference
sample and conducting repeated tests is feasible given practical con-
straints and consequences that may occur to the subjects' state including
fatigue, we recommend the use of constant-reference DTFM preceded by
the unspecified tetrad tests. This method not only allows the most effi-
cient discrimination testing in the 2-AFCR design, but also allows a com-
parison check using the tetrad tests.
In this study, for the determination of the validity of the 2-AFC model
assuming the 2-AFCR τ-strategy for the constant-saltier-reference
DTFM preceded by the unspecified tetrad tests, the balanced-reference
DTF (the conventional duo–trio with the reference presented at first)
and the unspecified tetrad test methods were used as control methods
as well as DTFM without the tetrad pre-tests. From comparisons of d′
estimates among these test methods, it was also confirmed that when
untrained/naïve consumer subjects are used without several days of
familiarization to the test samples, although DTF and DTFM were used
with a constant-reference mode, as in the 2-AFCR method, the conven-
tionally assumed duo-trio ‘comparison of distances’ (COD) strategy was
used. This finding corresponded with previous studies using DTF and
DTFM for discriminating tomato juices (Choi et al., 2014; Kim & Lee,
2012), and experiments using DTF for discriminating corn soup samples
(Kim et al., 2014) and soymilk samples (Jeong et al., 2015). However, in
these previous studies, there were some variations in the reference pre-
sentation. For example, in Kim et al. (2014) presented the reference
twice in the first two positions, while Jeong et al. (2015) allowed tast-
ings of the reference samples prior to the main test, enhancing sensory
memory for the reference. Also, for these two studies, instead of a fixed
design using one sample pair, multiple samples designs were used
where multiple test samples were compared from a reference in an ex-
perimental session. Thus further investigation of the effects of the differ-
ent reference presentations and tastings prior to or during the duo–trio
tests as well as the experimental design (fixed sample pair vs multiple
sample pairs) is needed to optimize the test procedures, both for DTF
and DTFM using the 2-AFCR τ-strategy and for DTF and DTFM using
the duo–trio COD strategy, that could be selectively applied according
to the different requirements of various experimental contexts under
various business and/or academic objectives.
Comparisons of d′ estimates across DTFM, DTF, and the unspeci-
fied tetrad test in terms of a single test sequence shown in Tables 2
& 4 also confirmed previous findings that when a stronger sample
is used as a reference for DTF, subjects perform better on tests for
discriminating confusable samples having only a subtle difference
(Choi et al., 2014; Kim & Lee, 2012). Results suggest that in the case of
only one test would be performed per subject, using the test sequence
Fig. 2. Comparison of d′ estimates and 95% likelihood confidence intervals obtained from each method with an example of similarity limit (action standard) of d′ = 1.
463
bBref-B, Bref-AN of DTFM, and bBref-B-AN of DTF would be advantageous
for the duo–trio design. These test sequences appeared to be less sus-
ceptible to the operational hindrances than the unspecified tetrad test.
To examine the operational test power of these test sequences further,
we re-computed d′ estimates based on the responses obtained from
these two test sequences by assuming increased sample size equal to
the tetrad test (n = 234). The results are illustrated in Fig. 2 with an
arbitrary action standard d′ = 1 as a similarity limit as suggested by
Kuesten (2001): d′ estimates (with 95% confidence intervals) for the
constant-reference DTFM using the 2-AFCR τ-strategy, the constant-
reference DTFM using the COD strategy, and the DTF using the COD strat-
egy were d′ = 1.39 (1.11–1.68), d′ = 1.82 (1.45–2.18), and d′ = 1.82
(1.45–2.18), respectively. As shown in Fig. 2, the lower confidence inter-
vals of these d′ estimates obtained from DTF and DTFM were larger
than the tetrad test results obtained from both groups, d′ = 1.16 (0.92–
1.38). The level of the lower confidence intervals of d′ estimates are the
critical value used for the similarity test. For example, if a similarity test
was performed using an action standard d′ = 1, in the present study,
the four methods would lead to a different conclusion: the results of d′
estimates for only three duo–trio methods and not the tetrad test were
exceeding the action standard (d′ = 1) considering the lower confidence
interval, concluding that the subjects perceived the samples as different.
For these three methods using duo–trio presentations, two different
major sources can be considered as reasons for such high test efficiency
in savory food discrimination. The efficiency of the constant-reference
DTFM using the 2-AFCR τ-strategy could be attributed mainly to higher
statistical power, while the efficiency of the constant-reference DTFM
and DTF using the COD strategy could be attributed mainly to high
test sequence advantage. The variations in the latter test efficiency
have sometimes been described as either sequence effects or task diffi-
culty due to increase cognitive load and memory decay (Kim, Jeon, Kim,
& O'Mahony, 2006; Rousseau & O'Mahony, 1997; Dessirier & O'Mahony,
1998; Rousseau, Rogeaux, & O'Mahony, 1999; Rousseau, Stroh, &
O'Mahony, 2002; Lau, O'Mahony, & Rousseau, 2004; Lee & O'Mahony,
2007). These effects were later referred to as components of ‘operational
test power’ in a broader concept (Choi et al., 2014; Garcia et al., 2013; Kim
& Lee, 2012; Kim et al., 2014; van Hout et al., 2011). The test methods that
have higher ‘operational test power’ would result in the larger d′ esti-
mates with higher lower confidence limit for discrimination of the
same samples, than other test methods, which is important for business
decision making using the d′ results. Namely, The term ‘operational test
power’ reflects the fact that the high statistical power and precision of
the d′ estimates predicted from the theory under the precondition of
perfect memory and no deficiency in the clarity of the sensory informa-
tion is necessary but not sufficient factor to determine the accuracy of
the data analysis and business decision making using d′ estimations.
(Choi et al., 2014; Garcia et al., 2013; Kim & Lee, 2012; Kim et al.,
464 I.-A. Kim et al. / Food Research International 76 (2015) 458–465
2014; van Hout et al., 2011). Yet, since the relativeness in operational
test power of various difference test methods tends to be variable to
the experimental contexts, caution should be taken before the empirical
results with food discriminations are generalized. For example, for dis-
crimination between more complex food samples, or discrimination of
multiple samples from a reference, some test method might be more
susceptible to the operational hindrance than others, leading to less
robust results. This aspect also highlights the advantages of the two se-
quential test approaches such as the constant-reference DTFM preceded
by the unspecified tetrad tests presented in this paper. If the discrimina-
tion tests are performed as a comparison experiment, this design will
not only allows the checking for the appropriate cognitive decision
strategy but also reveals the information how much the predicted test
power of the discrimination methods used for investigation were
vulnerable to the experimental contexts. This motivates researchers to
use more standardized methods with a comparison and describe
more details about experimental design, test procedures, and character-
istics of the samples used for sensory discriminations in the literature. It
also motivates more empirical research based from which extended
theories, taking into account additional factors such as cognitive load
factors and the sequence advantage, can evolve.
5. Conclusions
In the present study, the constant-saltier-reference DTFM preceded by
the unspecified tetrad tests was confirmed to adopt the 2-AFCR τ-strategy
by untrained/naïve subjects, while for the same test design, without
tetrad pre-tests, the duo-trio COD strategy was adopted. This suggests
that the tetrad test could be used as a familiarization procedure for sub-
jects to learn the signs of sensory difference that is required to develop
the decision space in a way that could be most efficiently dealt with by
subjects and thus subjects can use an optimal decision strategy for the
DTFM as in the specified 2-AFC method. This DTFM using the most pow-
erful decision strategy can be an alternative to the triangle or unspecified
tetrad tests that has been commonly used. But the tetrad test used as pre-
tests prior to this DTFM version of the 2-AFC, can be used as a perfor-
mance check for checking the cognitive strategy and relative operational
power of the various versions of discrimination test method, as a design
of scientific comparison experiments for conducting sensory difference
tests. Such comparison experiments are critical for using untrained/
naïve consumer subjects and using more complex experimental design
for discriminations of larger number of or complex samples. This paper
not only contributes to the extension of the theory and models of the
duo–trio tests, but also provides information to aid sensory practitioners
in using the duo–trio test in the most efficient way when there is a
reference sample and the pre-familiarization of sensory differences of
samples under investigation is feasible. Regarding sodium reduction
issues, the saltier original sample should be used as the constant-
reference, because not only subjects are more sensitive to reductions in
salt than increases in salt, but also constant-saltier reference reflects the
evaluation situation ecologically valid for food industries.
Acknowledgments
This research was supported by the Basic Science Research
Program through the National Research Foundation of Korea (NRF)
funded by the Ministry of Science, ICT & Future Planning (No.
2015R1A1A1A05001170).
References
ASTM E1885-04 (2004). Standard test method for sensory analysis — Triangle test. West
Conshohocken, PA: ASTM International.
Bi, J., Lee, H. -S., & O'Mahony, M. (2014). Estimations of Thurstonian models for various
forced-choice sensory discrimination methods as a form of the ‘M+N’ test. Journal
of Sensory Studies, 29, 325–338.
Bi, J., Lee, H. -S., & O'Mahony, M. (2010). d′ and variance of d′ for four-alternative forced
choice (4-AFC). Journal of Sensory Studies, 25, 740–750.
Byer, A., & Abrams, D. (1953). A comparison of the triangular and two-sample taste test
methods. Food Technology (Chicago), 7, 185–187.
Choi, Y. -J., Kim, J. -Y., Christensen, R.H.B., van Hout, D., & Lee, H. -S. (2014). Superior
performance of constant-saltier-reference DTF and DTFM to same–different tests
by consumers for discriminating samples varying sodium contents. Food Quality
Preference, 37, 100–108.
Christensen, R.H.B., & Brockhoff, P.B. (2015). sensR: An R-package for sensory discrimination.
R-package version 3.1.1 (http://www.cran.r-project.org/web/packages/sensR).
Dessirier, J.M., & O'Mahony, M. (1998). Comparison of d′ values for the 2-AFC (paired
comparison) and 3-AFC discrimination methods: Thurstonian models, sequential
sensitivity analysis and power. Food Quality Preference, 10, 1–8.
Ennis, D.M. (1990). Relative power of difference testing methods in sensory evaluation.
Food Technology, 44, 114–118.
Ennis, D.M. (1993). The power of sensory discrimination methods. Journal of Sensory
Studies, 8, 353–370.
Ennis, J.M. (2012). Guiding the switch from triangle testing to tetrad testing. Journal of
Sensory Studies, 27, 223–231.
Ennis, J.M., & Christensen, R.H.B. (2014). Precision of measurement in tetrad testing. Food
Quality and Preference, 32, 98–106.
Ennis, J.M., & Jesionka, V. (2011). The power of sensory discrimination methods revisited.
Journal of Sensory Studies, 26, 371–382.
Ennis, J.M., Rousseau, B., & Ennis, D.M. (2014). Sensory difference tests as measurement
instruments: A review of recent advances. Journal of Sensory Studies, 29, 89–102.
Frijters, J. (1979). Variations of the triangular method and the relationship of its unidi-
mensional probabilistic models to three-alternative-forced-choice signal detection
theory models. British Journal of Mathematical and Statistical Psychology, 32, 229–241.
Garcia, K., Ennis, J.M., & Prinyawiwatkul, W. (2012). A large-scale experimental comparison
of the tetrad and triangle tests in children. Journal of Sensory Studies, 27, 217–222.
Garcia, K., Ennis, J.M., & Prinyawiwatkul, W. (2013). Reconsidering the specified tetrad
test. Journal of Sensory Studies, 28, 445–449.
Green, D.M., & Swets, J.A. (1966). Signal detection theory and psychophysics. New York,
USA: John Wiley & Sons Ltd.
Hautus, M.J., van Hout, D., & Lee, H. -S. (2009). Variants of A Not-A and 2AFC tests: Signal
detection theory models. Food Quality Preference, 20, 222–229.
Ishii, R., O'Mahony, M., & Rousseau, B. (2014). Triangle and tetrad protocols: Small sensory
differences, resampling and consumer relevance. Food Quality Preference, 31, 49–55.
ISO (2004a). Standard 4120: Sensory analysis — Methodology — Triangle test. Geneva,
Switzerland: International Organization for Standardization.
ISO (2004b). Standard 10399: Sensory analysis — Methodology — Duo–trio test. Geneva,
Switzerland: International Organization for Standardization.
Jeong, Y. -N., Kang, B. -A., Song, M. -J., Jeong, M. -J., Hautus, M.J., & Lee, H. -S. (2015). Sen-
sory discrimination by consumers of multiple stimuli from a reference: Stimulus con-
figuration in A-Not AR and constant-ref. duo-trio superior to triangle and unspecified
tetrad? Food Quality and Preference, http://dx.doi.org/10.1016/j.foodqual.2015.06.021.
Keast, R.S.J., & Breslin, P.A.S. (2002). An overview of binary taste-taste interactions. Food
Quality and Preference, 14, 111–124.
Kim, M. -A., Chae, J. -E., van Hout, D., & Lee, H. -S. (2014). Higher performance of constant-
reference duo–trio test incorporating affective reference framing in comparison with
triangle test. Food Quality and Preference, 32, 113–125.
Kim, H. -J., Jeon, S.Y., Kim, K. -O., & O'Mahony, M. (2006). Thurstonian models and variance I:
Experimental confirmation of cognitive strategies for difference tests and effects of per-
ceptual variance. Journal of Sensory Studies, 21, 465–484.
Kim, M. -A., & Lee, H. -S. (2012). Investigation of operationally more powerful duo–trio
test protocols: Effects of different reference schemes. Food Quality and Preference,
25, 183–191.
Kim, M. -A., Lee, Y. -M., & Lee, H. -S. (2010). Comparison of d′ estimates produced by three
versions of a duo–trio test for discriminating tomato juices with varying salt concen-
trations: The effects of the number and position of the reference stimulus. Food
Quality and Preference, 21, 504–511.
Kim, M. -A., Sim, H. -M., & Lee, H. -S. (2015). Affective discrimination methodology: Deter-
mination and use of a consumer-relevant sensory difference for food quality mainte-
nance. Food Research International, 70, 47–54.
Kuesten, C.L. (2001). Sequential use of the triangle, 2-AC, 2-AFC, and same–different
methods applied to a cost-reduction effort: consumer learning acquired throughout
testing and influence on preference judgements. Food Quality and Preference, 12,
447–455.
Lau, S., O'Mahony, M., & Rousseau, B. (2004). Are three-sample tasks less sensitive than
two-sample tasks? Memory effects in the testing of taste discrimination. Perception
& Psychophysics, 66, 464–474.
Lee, Y. -M., Chae, J. -E., & Lee, H. -S. (2009). Effects of order of tasting in sensory difference
tests using apple juice sample: Development of a new model. Journal of Food Science,
74, S268–S275.
Lee, H. -S., & Kim, K. -O. (2008). Difference test sensitivity: Comparison of three versions
of the duo–trio method requiring different memory schemes and taste sequences.
Food Quality and Preference, 19, 97–102.
Lee, H. -S., & O'Mahony, M. (2007). The evolution of a model: A review of Thurstonian and
conditional stimulus effects on difference testing. Food Quality and Preference, 18,
369–383.
Lee, H. -S., van Hout, D., & Hautus, M.J. (2007). Comparison of performance in the
A-Not A, 2-AFC, and same–different tests for the flavor discrimination of marga-
rines: The effect of cognitive decision strategies. Food Quality and Preference, 18,
920–928.
Macmillan, N.A., & Creelman, C.D. (2005). Detection theory, a user's guide (2nd ed.).
Mahwah, New Jersey: Lawrence Erlbaum Associates.
 I.-A. Kim et al. / Food Research International 76 (2015) 458–465
Masuoka, S., Hatjopoulos, D., & O'Mahony, M. (1995). Beer bitterness detection: Testing
Thurstonian and sequential sensitivity analysis models for triad and tetrad methods.
Journal of Sensory Studies, 10, 295–306.
McClure, S., & Lawless, H. (2009). Examination of the subject defined 2-AFC (SD-2-AFC).
Journal of Sensory Studies, 24, 14.
McClure, S., & Lawless, H. (2010). Comparison of the triangle and a self-defined two alter-
native forced choice test. Food Quality and Preference, 21, 547–552.
O'Mahony, M., & Goldstein, L. (1987). Tasting successive salt and water stimuli: The roles
of adaptation, variability in physical signal strength, learning, supra-and subadapting
signal detectability. Chemical Senses, 12, 425–436.
O'Mahony, M., & Odbert, N. (1985). A comparison of sensory difference testing procedures:
Sequential sensitivity analysis and aspects of taste adaptation. Journal of Food Science, 50,
1055–1058.
Peryam, D.R., & Swartz, V.W. (1950). Measurement of sensory differences. Food
Technology, 4, 390–395.
R Core Team (2014). R: foundation for statistical computing. (URL http://www.R-project.
org/).
Rousseau, B., Stroh, S., & O′Mahony, M. (2002). Investigating more powerful discrimina-
tion tests with consumers: Effects of memory and response bias. Food Quality and
Preference, 13, 39–45.
Rousseau, B., & Ennis, J.M. (2013). Importance of correct instructions in the tetrad test.
Journal of Sensory Studies, 28, 264–269.
Rousseau, B., & O'Mahony, M. (1997). Sensory difference tests: Thurstonian and SSA pre-
dictions for vanilla flavored yogurts. Journal of Sensory Studies, 12, 127–146.
View publication stats
465
Rousseau, B., Rogeaux, M., & O'Mahony, M. (1999). Mustard discrimination by same–
different and triangle tests: Aspects of irritation, memory and τ criteria. Food Quality and
Preference, 10, 173–184.
Stocks, M.A., van Hout, D., & Hautus, M.J. (2013). Cognitive decision strategies adopted in
reminder tasks by trained judges when discriminating aqueous solutions differing in
the concentration of citric acid. Journal of Sensory Studies, 28, 217–229.
Stocks, M.A., van Hout, D., & Hautus, M.J. (2014). Cognitive decision strategies adopted by
trained judges in reminder difference tests when tasting yogurt, mayonnaise, and ice
tea. Food Quality and Preference, 34, 14–23.
Tedja, S., Nonaka, R., Ennis, D.M., & O'Mahony, M. (1994). Triadic discrimination testing:
Refinement of Thurstonian and sequential sensitivity analysis approaches. Chemical
Senses, 19, 279–301.
van Hout, D. (2014). Measuring meaningful differences: Sensory testing based decision making
in an industrial context; applications of signal detection theory and Thurstonian modelling.
Doctoral dissertation No. EPS-2014-304-MKT. Erasmus Research Institute of Manage-
ment (ERIM) (Retrieved from http://hdl.handle.net/1765/50387).
van Hout, D., Hautus, M.J., & Lee, H. -S. (2011). Investigation of test performance over re-
peated sessions using signal detection theory: Comparison of three nonattribute-
specified difference tests 2-AFCR, A-Not A and 2-AFC. Journal of Sensory Studies, 26,
311–321.
Vié, A., & O'Mahony, M. (1989). Triangular difference testing: Refinements to sequential
sensitivity analysis for predictions for individual triads. Journal of Sensory Studies, 4,
87–103.