SENSORY ANALYSIS - DISCRIMINATIVE TESTS - "2 out of 5"

Addressees:
All members of the GQD
Quality Directors, Sensory relays, Sensory coordinators
Available on Intranet "GQD Documentation" Distribution level:
Group's Sensory Departments / Quality Departments

Global modification of the initial document MOSQC039(A).

1. General introduction on global discriminative tests

Definition
The purpose of global discriminative tests is to detect the presence or the absence of sensory differences
between two products. The sensory characteristic that is concerned by the possible differences is not generally
known to the experimenter, and never to the subjects.

Global discriminative tests cover various tests such as the A-not A test, the duo-trio test, the triangle test, the 2 out of
5 test, the p out of n test, tetrad, the simple difference test.
The principle is as follows: the subjects must compare several samples and recognize those that are identical
between them or different.

These methods make it possible to compare products in a global manner without identifying the nature of the
difference perceived or even quantifying it.
They require, as a prerequisite, to have homogeneous samples, i.e. without intra-batch variability.
These are forced choice tests which implies that a response is imposed on the subjects and that not responding is
not allowed.
We recommend the non-repetition by the subjects of the evaluation.
The protocol has to test the hypothesis H0 (= the products are perceived as identical). This hypothesis is rejected if
the proportion of correct responses is significantly greater than what would be expected by chance.

Scope
These tests must imperatively be set up with the assistance of the Division/Country sensory relay or
otherwise with the GQD's sensory department, in order to validate the methodology chosen, setting it up and
the interpretation of the test. They apply each time that it is desired to compare two or more products between
which the differences are unknown and low. In the case of differences that are easy to perceive, it is preferable to
use classification tests or sensory profiles.

The "p among n" test is a global difference test that has the same principle as the triangle test. It has many
possible combinations of which the most common tests are the "1 out of 4" and the "2 out of 5" tests. When it
is possible to use them, these tests are to be favored with respect to the triangle test.

Advantages of the "2 out of 5" test:

Lower number of subjects required
For tasters, does not require any training except for the method with a blank test (for example, water vs.
slightly sweetened water)
Robust statistical method (probability of only 1 chance out of 10 to find the correct response through random
chance).

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
1/16
Disadvantages of the "2 out of 5 test":
Sensory fatigue of the subjects
Not suitable for saturating products
Cumbersome to set up in practice; needs more products; requires sensory statistics software to process
the results if a similarity test is carried out
No identification of the nature of the difference
No quantification of the difference

Lactalis recommendation: if the type of product to be tested allows, using the "2 out of 5" is favored with respect to
the triangle test in case of a reduced number of subjects available.

To assist you, two record sheets are available: FESQC396(A) 2 out of 5 test questionnaire.docx and
FESQC397(A) File for conducting a discrimination test.xlsx.

Reference to standards for the "2 out of 5" test: no existing standard.

2. Prerequisite

The person who is in charge of a discriminative test must imperatively master the following statistics notions:
- Notions of population and samples
- Hypothesis tests and the notion of risk (α and β)
- Probabilities, binomial distribution, normal distribution
- The notion of variability

Warning, in the framework of a "2 out of 5" similarity test, you must have sensory statistics software to be able to
process the results. You must therefore contact your Division/Country sensory relay beforehand in order to validate
this point (or the sensory relay can contact the GQD sensory department).

3. Principle of the "2 out of 5" test

The "2 out of 5" test complicates the taster's task by decreasing the probability of randomly giving a correct response
by increasing the number of samples. This test consists in presenting five samples to the subjects, of which two come
from one lot and three come from another lot. This entails grouping together the samples that are perceived as
identical. (Cf. – Appendix 1 Sample questionnaire and FESQC396 (A)).

This test makes it possible to determine, according to the objective of the project that is submitted to us,

- that there is a perceptible difference between the two products difference "2 out of 5"

Example 1: Is a change in a ferment perceptible on my cheese?

It is desired that it be perceived as more intense at the aromatic level.

OR - that there is no perceptible difference between the two products similarity "2 out of 5"

Example 2: Does the modification to the process at the plant impact my kefir?
It is desired that consumers do not perceive the change in the process made in the manufacturing of kefir.

From a statistics standpoint, the sensitivity of the test is defined by 3 parameters: the α-risk, the β-risk and the
proportion of Pd discriminators (Cf.- Appendix 2 Additional information on statistics parameters).

These 3 parameters are to be set absolutely before conducting the test because they influence the number of
tasters to survey.

α−risk:
α− this is the risk of concluding that the 2 products are different when they are actually not.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
2/16
β-risk: this is the risk of concluding that the 2 products are similar when they are actually not.

Pd (proportion of discriminators): proportion in the population of the subjects that can make the distinction between
the two products.

Finally, it can generally be observed that the proportion of subjects who perceive a difference can be divided into 3
ranges:

- Pd ≤ 20% represents a low proportion,

- 20% < Pd ≤ 30% represents a moderate proportion,
- Pd > 30% represents a moderate to strong proportion (50%).

4. Experimental procedure

4.1 Choosing the statistical parameters

difference "2 out of 5"

The α-risk must be set with the sponsor of the study and before running the test. Through experience, the α-risk
is set to 5% (generally accepted average risk). But, according to the product tested (importance for the business) and
of the project (more or less risky), we can position ourselves at another level.

similarity "2 out of 5"

The α- and β- risks as well as Pd must be set with the sponsor of the study before running the test. By default,
Pd is set to 20% (case with people who consume and/or evaluate the product but not on a weekly basis) or 30% (case
with people who evaluate the product for release-based tasting, or on a weekly basis on a trained panel or on an
expert panel for example) and the β-risk at 10%. The α-risk is generally neglected and set to 20% (given that
concluding that there is a difference when there is no difference is not essential with respect to the project).

4.2 Number of tasters to survey

Based on the α- and β-risks, as well as Pd set with the sponsor of the study, the minimum number of subjects to be
surveyed can be identified using a sensory statistics software such as Fizz traitements ® (by Biosystèmes) for those
who have a license (Cf. – Appendix 3 Calculation of the number of tasters to be surveyed using the Fizz traitements ®
software, version 2.61).

Moreover, the global recommendations of the Lactalis Group are as follows:

difference "2 out of 5"

Minimum 14 tasters
α = 5%

similarity "2 out of 5"

β = 10%, Minimum 22 tasters
Pd at 20% or 30%

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
3/16
Note that the more tasters are surveyed, the more randomly-given responses is limited. On the contrary, if few tasters
are surveyed, the responses obtained are highly linked to chance. When it is sought to demonstrate a similarity, more
tasters must then be surveyed than when trying to demonstrate a difference.

4.3 Choosing samples

The two products tested (for example, Test and Control) must be representative of their production and be
homogeneous (no intra-batch variabilities). They must not be different visibly. Their difference must be very low (to be
checked before running the test).

Note: in case of a slight difference in color between the two products, the latter can be masked by using a red filter or
a green filter if the test is conducted in a sensory box, or otherwise using tinted glasses or using covers, for example.

4.4 Preparation and unfolding of the "2 out of 5" test

- Store the two products at the same temperature.

- Provide, per product, the required quantity (= 2.5 x quantity per taster x the total number of tasters).

- Generate 3 codes (3-digit codes) for the product A and 3 codes for the product B.

- Simultaneously present the 5 samples at the same temperature and in the same way (same size and portion form,
same color, same overall aspect except, of course, if the difference concerns an aspect criterion).

- Present the 5 samples to each taster in an imposed order (to be evaluated from left to right) according to the
presentation plan defined hereinafter: repeat this according to the number of tasters to survey.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
4/16
 Taster Order 1 Order 2 Order 3 Order 4 Order 5
1 A A B B B
2 B B A A A
3 A B A B B
4 B A B A A
5 A B B A B
6 B A A B A
7 A B B B A
8 B A A A B
9 B A A B B
10 A B B A A
11 B A B A B
12 A B A B A
13 B A B B A
14 A B A A B
15 B B A A B
16 A A B B A
17 B B A B A
18 A A B A B
19 B B B A A
20 A A A B B

5. Statistical interpretation of the results

Use of the binomial probability distribution = p ! (n-p) ! / n !

p is the number of samples among a larger population of n samples

For the "2 out of 5" test, the probability is 2 ! (5-2) ! / 5 ! = 1/10.

difference "2 out of 5"

Cf. – Appendix 4 Statistical table for difference "2 out of 5".

- Count the number of correct responses obtained (RC).

- For the α-risk chosen and the number of tasters questioned, look up the expected minimum number of RCs in the
table:
• if the number of RCs obtained > or = minimum number of RCs read in the table, the products are considered
to be significantly different at threshold α.
• if the number of RCs obtained < minimum number of RCs read in the table, the products are considered as
not significantly different at threshold α.

Note: For those that have the Fizz ® software, the session can be created with Fizz Acquisition ®. The responses
from the tasters are automatically collected and the statistical interpretation of the results is then done with Fizz
Traitements ®. Cf. – Appendix 5 Interpretation of the results with the assistance of the Fizz traitements ® software,
version 2.61

similarity "2 out of 5"

There is no statistical table for a similarity "2 out of 5" test. Interpreting the results has to be done using sensory
statistics software such as Fizz traitements ® for example.
Cf. – Appendix 5 Interpretation of the results with the assistance of the Fizz traitements ® software, version 2.61

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
5/16
Note 1: demonstrating an absence of a global difference or even a similarity between two products using a "2 out of 5"
test does not imply that all their sensory characteristics are identical.

Note 2: it is not possible to extrapolate the result of a "2 out of 5" test as a consumer preference.

Note 3: there is no way to validate the performance of the tasters who are participating in a "2 out of 5" test.

Note 4: repeating a "2 out of 5" test is not recommended at all due to the excessively large number of samples to
taste.

Note 5: in case of several modifications in progress on a product, it is important to take account of all these
modifications in order to validate the new recipe. Indeed, the adding of several changes that are not significantly
perceptible can lead to a significant difference.

Reduction in fat

Similarity of the 2 recipes

Current New recipe

Similarity of the 2 New pack
Reduction in fat
recipes New pack
Change in aroma supplier

Similarity of the 2 recipes

Is there a significant
difference between the
current recipe and the
new recipe?
Change in aroma supplier

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
6/16
Appendix 1 – Sample Questionnaire – 2 out of 5 Test

Date: ……………………

Last name, First name: ……………………

You will be given five samples: 425 – 735 – 256 – 849 and 333.
Examine them from left to right.

You must separate them into 2 groups (one with 3, and the other with 2 identical samples).
An answer must be provided.

Group A:

Group B:

Which sensory characteristic(s) enabled you to tell the difference?

……………………………………………………………………………………………………………….
……………………………………………………………………………………………………………….
……………………………………………………………………………………………………………….

The intensity of the difference according to you is?

Low Moderate Medium Strong Very strong

Note 1: the responses to the additional questions are taken into account only for the tasters who found the correct
response to the test.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
7/16
Appendix 2 – Additional information on statistics parameters

Recall:

Example 1: difference "2 out of 5" test

Is a change in a ferment perceptible on my cheese?
It is desired that it be perceived as more intense at the aromatic level.

Example 2: similarity "2 out of 5" test

Does the modification to the process at the plant impact my kefir?
It is desired that consumers do not perceive the change in the process made in the manufacturing of kefir.

We want
- either to validate the null hypothesis, H0 = the two products are identical (case of example 2)
- or to validate the opposite hypothesis, H1 = the products are different (case of example 1)

Decision based on the test: I accept…

H1: the products are different H1: the products are identical
Case of example 1 Case of example 2

We conclude that the products are

α-risk: We conclude that the change in
H0 is similar and we were right. The
ferment is perceptible although the two
true manufacturing process of kefir can be
cheeses are similar.
changed.
Reality:

β-risk: We conclude that the products

We conclude that the cheeses are
H1 is are similar although they are not.
different and we were right. The recipe
true There is a risk of receiving consumer
can be changed with the new ferment.
complaints.

α-Risk (first order risk, or type I or the false positive rate): risk of concluding that the 2 products are different when
they are actually not. The smaller the α-risk is set, the stronger the proof will be that the difference is perceptible
between the 2 products. In sensory analysis, we are often positioned at 5%.

β-risk (second order risk, or type II or the false negative rate): risk of concluding that the 2 products are similar when
they are actually not. The smaller the β -risk is set, the stronger the proof will be that the difference is not perceptible
between the 2 products and that they can be considered as similar. We are generally positioned at 10%. Taking this
risk into account requires estimating a proportion of discriminators Pd.

Pd (proportion of discriminators): proportion in the population of the subjects that can make the distinction between
the two products. The subjects who participate in this test are considered to be a random sample drawn from a wider
population. Pd is linked to the complexity of the product universe and to the intensity of the difference between the two
products but also it is linked to the consumption frequency of the product by the subjects, i.e. to their degree of
expertise on the product. The value of Pd depends on the test used. It is said to be "test-dependent". To estimate Pd,
it is possible however to consult the existing bibliography on the product category tested or to use the experience of
one's colleagues as a basis or to build a database (for this, in the questionnaire of the "2 out of 5" tests, ask a question
for example in order to know if the taster is answering randomly or not).

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
8/16
Appendix 3 – Calculation of the number of tasters to survey using the Fizz traitements ® software, version
2.61

In Fizz Traitements, go to the Statistics (1) menu then select Determination Number of Subjects/ Power
(Discrimination test) (2):

Mode: Click on Percentage of discriminators

Parameters:
- Choose from the drop-down menu: Two-Five Test
- Specify the threshold of the α-risk, chosen with the sponsor of the test
- Specify Pd (proportion of discriminators)
- Specify the power desired for the test (recall = 1-β)
- Then click on the Calculate Number of Subjects button

The calculated number of subjects is displayed in the white window immediately below the parameters.

In the example shown hereinabove: For a similarity "2 out of 5" test, it was chosen to take a β-risk at 10%. We
therefore specify that we want a power of 90%. Pd is estimated at 20%. And in the context of the test (we especially
do not want to incorrectly conclude that the 2 products are similar), the α-risk is neglected (concluding incorrectly that
a difference exists between the two products when there is no difference is not important for our sponsor) and set to
20%.

22 tasters are to be surveyed.

The test can indeed be conducted with 22 tasters because it also corresponds to the minimum recommended by
Lactalis.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
9/16
Appendix 4 Statistical table for difference "2 out of 5"

n = number of subjects surveyed and significance threshold (%) = α-risk

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
10/16
Appendix 5 Interpretation of the results with the assistance of the Fizz traitements ® software, version 2.61

Following the creation of a "2 out of 5" session with Fizz Acquisition or at last the creation of a results file (FRS), the
data collected can be interpreted with the software.

In Fizz Traitements, go to the Processing – Choice between samples menu, then select the Two-Five Test method
and choose the results file (FRS) that contains the data to be processed.

The column of your FRS complies with the coding: 0 for an incorrect response and 1 for a correct response.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
11/16
Click on Prepare so as to have the possibility of providing a Title (Corona Project), modify the Product names and
select the Test to be processed using the drop-down menu (P2_25_1).
Save as the modifications made.
Click on OK.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
12/16
By default, all tasters who have done the test are selected for the processing but you can remove tasters by clicking
on Choice.

Finally, three types of processing are proposed:

- Difference test only: when interest is given solely to revealing a difference between the two products.

Confirm that the correct responses are coded as 1 and incorrect responses are coded as 0, by clicking on OK.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
13/16
 The results page is displayed, as hereinbelow:

There is a summary of all the responses (one per taster).

There is the statistical interpretation of the test: with 5

correct responses out of a total of 22, the calculated α-
risk is 6.21% therefore we are above the α threshold
set to 5%: the two products are not significantly
different at the threshold of 5%.

Note: taking the statistical table from appendix 3, for n = 22 and α = 5%, it is observed that a minimum of 6
correct responses is required in order to conclude a difference between the two products. With 5 correct
responses in the example, it is also concluded that the two products are not significantly different at the 5%
threshold.

- Difference test and β-risk: when interest is given only to revealing a difference and in the power of the test
carried out (have we surveyed enough tasters if we do not observe any difference between the two
products?).

Confirm that the correct responses are coded as 1 and incorrect responses are coded as 0 on the first tab and
click on the tab "Beta" Risks.
__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
14/16
 There are different options to choose from:
- Always calculate the β-risk or only if the difference test is not significant (in
the example if α > 5%)
- Ask for the calculation of the β-risk for different values of Pd (in the
example: 30%)
- Inversely, ask for the calculation of Pds according to the β-risks chosen (in
the example: 10%)

Click on OK.

There is the statistical interpretation of the test: with 5 correct responses

out of a total of 22, the calculated α-risk is 6.21% therefore we are
above the α threshold set to 5%:
The two products are not significantly different at the threshold of
5%.
With a Pd at 30%, the calculated power is 95.09% therefore greater than
80%.
Enough people were surveyed.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
15/16
- Similarity test: when interest is given solely to revealing a similarity between the two products.
Confirm that the correct responses are coded as 1 and incorrect responses are coded as 0 on the first tab and
click on the tab Similarity tests.
There are different options to choose:
- The calculation of the β-risk for different values of Pd (in the example:
30%)
- And the possibility of asking for the calculation of Pds according to the β-
risks chosen (in the example: 10%)

Click on OK.

If in our example, before the test, the β-risk was set to 10% and Pd to 30%:

- With a Pd at 30%, the calculated -risk 12.00% therefore > 10%.

or
- With a β-risk at 10%, the calculated Pd is 31.30% therefore > 30%.

It is not possible to significantly conclude that the two products are similar.

__________________________________________________________________________________________
MOSQC039(B) - Sensory analysis - Discriminative tests- 2 out of 5-US.docx
Written by I. Paillard on July 1, 2008
Modified by E. Gilbert on 6/11/2020
16/16