Sensory Evaluation Manual | Taste | Olfaction


Associate Professor Richard Mason The University of Queensland and Stephen Nottingham

Sensory Evaluation ACKNOWLEDGEMENTS These notes form the basis of a practical workshop presented for personnel at Naresuan University, Phitsanulok, Thailand in July, 2002. We would like to thank Michael O’Mahony for his permission to include copies of the statistical tables from his book “Sensory Evaluation of Food: Statistical Methods and Procedures” and material supplied by the Centre for Food Technology, DPI, Brisbane.



R L Mason and S M Nottingham

Sensory Evaluation TABLE OF CONTENTS ACKNOWLEDGEMENTS ....................................................................................................2 PROGRAM ............................................................ ERROR! BOOKMARK NOT DEFINED. INTRODUCTION....................................................................................................................5 THE HUMAN SENSES IN SENSORY EVALUATION .....................................................7 THE SENSES - AN INTRODUCTION .................................................................................7 SENSE OF SIGHT- .................................................................................................................9 THE SENSE OF SMELL......................................................................................................13 THE SENSE OF TASTE.......................................................................................................15 THE SENSE OF HEARING.................................................................................................19 THE SENSE OF TOUCH .....................................................................................................20 SENSORY INTERACTION .................................................................................................21 OPERATIONAL PRINCIPLES OF SENSORY TESTING .............................................23 DESIGN OF A SENSORY TESTING AREA.....................................................................31 STATISTICAL PRINCIPLES .............................................................................................34 SENSORY EVALUATION METHODS .............................................................................38 AFFECTIVE TESTS .............................................................................................................38 SPECIFIC TEST METHODS ..............................................................................................39 PAIRED PREFERENCE TEST.......................................................................................39 RANKING FOR PREFERENCE.....................................................................................41 RATING FOR PREFERENCE ........................................................................................44 SENSORY EVALUATION IN CONSUMER TESTING ..................................................46 ANALYTICAL SENSORY TESTS: ....................................................................................53 DIFFERENCE TESTING.....................................................................................................53 SIMPLE DIFFERENCE TEST ............................................................................................53 TRIANGLE TEST .............................................................................................................53 DUO-TRIO TEST..............................................................................................................55 TWO-OUT-OF-FIVE TEST.............................................................................................59 COPYRIGHT R L Mason and S M Nottingham 3

.............................94 STATISTICAL TABLES..................................76 STATISTICS FOR SENSORY: DESCRIPTIVE TESTING ...........................................................................................................73 DESCRIPTIVE TESTING .........................................................65 PAIRED COMPARISON TEST .................65 RANKING TEST ......................................................................................................................................91 SELECTED BIBLIOGRAPHY........................92 JOURNALS ............................................59 DIFFERENCE-FROM-CONTROL TEST (DFC) ...........................95 COPYRIGHT 4 R L Mason and S M Nottingham ...........................................................................................86 REPORTING .................Sensory Evaluation “A” – “NOT A” TEST.........................67 RATING TEST ................................................................................................................59 DIRECTIONAL DIFFERENCE TESTS ...............................................................................................................................................................81 SELECTION............................................................................. TRAINING AND MOTIVATION OF A PANEL ......................................................................................................................................69 STATISTICS FOR SENSORY: DIFFERENCE TESTING .............................................................................................................................

taste. Sensory evaluation was one of the earliest methods of quality control and it is still widely used in industry. accurate and reproducible. measure. touch and hearing. the level of application depends on the situation (e. However. analyse and interpret reactions to those characteristics of foods and materials as they are perceived by the senses of sight. Disadvantages • • • • • Time consuming Expensive to run Method selection Analysis Interpretation Industry applications of sensory evaluation • • • Product development Product matching Product improvement 5 R L Mason and S M Nottingham COPYRIGHT .Sensory Evaluation INTRODUCTION Sensory evaluation . smell.g. they must be correlated to sensory evaluation to indicate a consumer response. However.A scientific discipline used to evoke. beer and wine tasting to operators sampling of products from production line). Four variables affect sensory evaluation: • • • • The Food The People The Testing Environment Methods Sensory evaluation terminology • • • • • • Sensory evaluation Sensory Analysis Organoleptic Analysis Taste Testing Psychophysics Subjective Evaluation Advantages • • • • • Gives real answer regarding consumer quality Relatively cheap process (depending on how it is done) Rapid Many applications Objective methods are more reliable.

General guide to methodology Types and choice of test Sensory analysis of foods .General guide to methodology General requirements Sensory analysis of foods .Specific methods .Specific methods .3 AS 2542.3 Year 1995 1984 1984 1995 1982 1983 1988 1988 1991 1995 1989 Title Sensory analysis of foods .2.2.General guide to methodology Selection of assessors Sensory analysis of foods .2.Triangle test Sensory analysis of foods .6 AS 2542.Duo-trio test Sensory analysis of foods .0 AS 2542.2 AS 2542.Glossary of terms COPYRIGHT 6 R L Mason and S M Nottingham .1.Specific methods .Specific methods .1.Paired comparison test Sensory analysis of foods .2.Introduction and list of methods Sensory analysis of foods .Specific methods .'A not A' test Sensory analysis of foods .Rating Sensory analysis of foods .2.3 AS 2542.2.Ranking Sensory analysis of foods .1 AS 2542.1.1 AS 2542.5 AS 2542.Specific methods .2 AS 2542.Sensory Evaluation • • • • • • • • • • Process change Cost reduction New raw materials selection Quality control Storage stability Product grading / rating Consumer acceptance Consumer preference Panel selection / training Correlation subjective / objective Sensory Standards Aus Standard AS 2542.4 AS 2542.

and Texture . size. COPYRIGHT 7 R L Mason and S M Nottingham . It is impossible to rate each one individually unless special precautions are taken. coloured lights. e. Flavour . taste. These attributes are expressed as a continuum and not as finite properties.colour.Sensory Evaluation THE HUMAN SENSES IN SENSORY EVALUATION THE SENSES . purees.odour. blindfolds. nose clips. viscosity and hearing.mouth feel.g.AN INTRODUCTION The sensory properties of foods are related to three major attributes: • • • Appearance . shape.

ear is receptor for sound. An effective stimulus produces a sensation. and Hearing. Smell.g. e. light receptors in retina of eye. e. the dimensions of which are: • • • • Intensity/strength. the sour taste of lemons is the perception of the sensation received by the receptors (taste buds) from a chemical stimulus (citric acid). Touch.g. e. taste buds on tongue. However. and Hedonics/like-dislike. Duration/retention. Receptors Receptors are the stimuli detecting cells of the sense organ.Sensory Evaluation Humans possess about 30 different senses. the sensory properties of foods are perceived through the senses of: • • • • • Sight. Extent/separation. COPYRIGHT 8 R L Mason and S M Nottingham . Perception Perception is the psychological interpretation of sensations determined by comparison with past experiences.g. eye is receptor for light. Stimuli A stimulus is any chemical or physical activator that causes a response in a receptor. Taste.

COPYRIGHT 9 R L Mason and S M Nottingham .Sensory Evaluation SENSE OF SIGHTThe appearance of food Stimuli = visible light Receptor= retina of the eye Perception=sight. The rods are responsible for vision in dim light and the cones are responsible for colour vision. The retina is the receptor of vision and contains two types of cells. appearance The appearance of foods is a major factor governing its acceptability and can be subdivided into three main categories: • Optical properties. mostly males. from an external source interacts with the object and is brought to focus on the retina of the eye. light. vision. Light incident on these cells causes a photochemical reaction that generates an electrical impulse which is transmitted to the brain via the optic nerve. gloss and translucency • Physical form-shape and size • Mode of presentation-lighting packaging etc Optical properties Vision Vision is a complex phenomena consisting of several basic components.colour. A stimulus. Approximately 8% of the population have some defect with relation to colour. Colour blindness is caused by loss or lack of colour receptor cells in the cones.

and Gloss. Natural light is too variable for use in evaluating appearance of foods. Colour/hue.] Light sources Incandescent lights consist of a tungsten filament which is heated in an inert gas. The relationship between and within each of these components is responsible for the colour and gloss characteristics of the food. The higher the temperature.475 nm . and Refracted.Sensory Evaluation Light Visible light is that part of the electromagnetic spectrum which radiates between wavelengths of 380 .575 nm — 590 nm — 770 nm =violet =blue =green =yellow = red [NOTE: All electromagnetic radiations are physically the same. the more light produced. Chroma/purity.Object interactions Light incident on an object may be: • • • • Absorbed.450 nm . Physical form The second class of product appearance is physical form that can be subdivided into three parts: COPYRIGHT 10 R L Mason and S M Nottingham . Light . Transmitted. Light produced is softer but can produce colour distortion at particular wavelengths. the optical system of the eye is such that only the visible range of wavelengths is absorbed by the lens. Light from this source tends to be harsh and tends to highlight the red end of the spectrum. However. Fluorescent lights operate by electrical excitation of atoms that produces spectral lines at specific wavelengths which then impinge onto fluorescent materials which convert the incident light into light at a longer wavelength. Different wavelengths produce different colours 380 450 500 575 590 . The main light/object interactions produced are: Lightness/value.770 nm. Reflected.

Contrast . Some examples include: • • • • Sliced. ingredient. Packaging . Mode of presentation is applicable on the supermarket shelf (at retail level) and also in terms of presentation at the table (home and restaurant). Some examples include: • • • Open dry structure of meat Wrinkling of peas Wilting of lettuce Visual consistency can indicate product viscosity as in: • • • Setting of a jelly Syrups of different concentrations Pastes and purees Mode of presentation This aspect should be considered from a marketing point of view and is important because it influences sales.affects apparent product colour.phenomena of adjacent colours. design. pieces whole Length of frozen French fries Cut of beans Extrusions Surface texture can indicate product price. and Illumination . Shape and size are important from a food technologist's point of view because these can be altered during the manufacture of processed products. etc. and Visual consistency. colour. Factors to be considered are: • • • • Product description . diced.Sensory Evaluation • • • Shape. COPYRIGHT 11 R L Mason and S M Nottingham .shape. Surface texture.

Sensory Evaluation Summary Appearance is an important aspect of food quality as it is the first subjective evaluation made of food quality. The product has to pass the visual assessment before the consumer can or will consider the other parameters such as taste and texture. Factors that should be considered in evaluating product appearance include: • • • • • • • • use of standard conditions: light source (type, intensity, colour); background; and style of presentation (unless tested). selection of appearance attribute(s) for inclusion on scoresheet; using appearance to reduce tasting load; should be masked to eliminate unwanted interactions when assessing parameters involving other senses; and colour charts/standards help rating.



R L Mason and S M Nottingham

Sensory Evaluation THE SENSE OF SMELL (Odour/olfaction) Stimuli = volatile chemicals Receptors= olfactory cells in the nose Perception=smell, odour, aroma, flavour Smell is one of our most primate senses. influenced by smell than other senses. Supposedly prehistoric people were more

The human nose is capable of detecting thousands of different odour substances. However, our sensitivity is much less than other animals. (Animals use smell - food, mating, territory etc). Smell is detected both before and during eating. Smell is an important aspect of flavour. There are 20x106 olfactory receptors, but only about 1000 taste receptors. Odour description requires the development of an odour/flavour memory, e.g. fishy, flowery, woody. This is the basis of flavour/odour memory development by wine judges and milk/cheese graders. Individuals vary a great deal in their sensitivity to different odours/aromas. Anatomy of olfactory system

From the diagram it can be seen that most of air misses the olfactory area. Only 5-10% of inspired air passes over olfactory receptors. However, this amount can be increased by sniffing harder; obviously the more air which passes over the receptors the better the COPYRIGHT 13 R L Mason and S M Nottingham

Sensory Evaluation response. The large number of olfactory receptors (20x106) enable detection of : • • • More odours than tastes; A greater variety of odours; and Odours at much lower concentration (10 molecules/mL).

In order for odour to register: • • • • Substance needs to be volatile enough to get into air in the sensory region. Substance needs to be partially soluble in mucus covering of receptors. Minimum number of odorous molecules need to be present. Need to be in contact with receptors for minimum time.

Olfactory intensity Human nose is about 10-100 times more sensitive to odours than any physico-chemical analysis (e.g. gas chromatography). It has been demonstrated that human nose is capable of detecting ethyl mercaptan at a concentration of 0.01 mg/230m3 of air, which is equivalent to about 8 molecules/receptor. Olfactory threshold Detection threshold is the concentration where smell is detected. Recognition threshold is the concentration where the smell is recognised. Olfactory interactions Nature of the response may change with concentration (e.g. perfumes at low concentration are pleasant but at strong concentration may be unpleasant). Interaction of odours: • Additive - increase intensity; • Suppressive - decrease intensity; and • Blending - when new odour unrelated to originals. Olfactory adaptation Initial sensation maybe strong - but weakens and makes identification difficult; this is due to adaptation of olfactory receptors. In testing we therefore need to allow for this by: • Taking first impression of odour and/or • Waiting between tests to allow receptors to recover.



R L Mason and S M Nottingham

g. Therefore. and Bitter. strong peaks may produce weak odour whereas weak peaks may produce a strong odour). Human nose is much more sensitive than analytical instruments. Smell. Foods contain numerous compounds of varying volatility that can make analytical interpretation difficult (e. The tongue is important as it brings the food into contact with the taste buds and also provides a mixing action which enables an even distribution of food about the taste buds as well as preventing the development of concentration gradients. analytical testing does not. Strictly speaking taste involves only those sensations mediated by the Gustatory Nerve Fibres and these sensations have five (5) basic qualities: • • • • • Salt. • THE SENSE OF TASTE (Gustation) Stimuli = soluble chemicals or chemicals which are solublised during chewing Receptors= taste buds in mouth Perception=taste. Saliva production is generally stimulated by chewing. Touch. Umami Taste stimuli Taste response requires an aqueous solution of the substance (stimulus) to contact the taste buds. Smell measures perception of a mixture. as well as the appearance and odour of the food. Sweet. flavour What is commonly referred to as taste/flavour is actually a combination of: • • • • Taste. saliva secretions are important in terms of ensuring contact between the product and the taste buds. and Temperature. Sour.Sensory Evaluation Summary • • • Smell is a major component of food flavour. COPYRIGHT 15 R L Mason and S M Nottingham .

COPYRIGHT 16 R L Mason and S M Nottingham .Sensory Evaluation Taste receptors The receptors for taste are the taste buds and these are mounted on papillae (folds in the skin of the tongue). Taste buds are mainly located at the tip. lips. Different areas of the tongue are most responsive to different sensations. The area of greatest response is the top of the tongue. Their life cycle is 10 days and they are easily destroyed by heat. The tongue itself is important as it brings the food into contact with the taste buds and also provides a mixing action which enables an even distribution of food about the taste buds as well as preventing the development of concentration gradients. cheeks. underside of tongue and floor of mouth. pharynx. epiglottis. sides and rear of tongue. There is very little response in the centre of the tongue. larynx. tonsils. • • • • Tip Sides Sides Rear sweet salty sour bitter Taste cells constantly degenerate and regenerate. Other areas in the mouth and throat where taste buds are situated include: palate.

but only sodium chloride gives a pure salty taste. Bitterness is generally perceived at very low concentration and a relationship appears to exist between sweet and bitter as many sweet substances produce a bitter aftertaste (saccharin). sourness of aliphatic organic acids relates to chain length. Bitterness Many chemically different compounds have a bitter taste. Bitterness is the taste which most people have difficulty in detecting and response level varies greatly from individual to individual. Saltiness Many crystalline water-soluble salts yield a salty taste. COPYRIGHT 17 R L Mason and S M Nottingham . sweet and salt in various combinations. proteins. However.g. quinine. some amino acids are sweet (aspartane) picric acid is bitter sugar may enhance/depress sourness sourness is also affected by pH and acid presence of buffers affects sourness Sweetness The common substances that produce the sweet taste are the sugars and other hydroxy compounds such as alcohols and glycols. codeine) and many other bitter substances are harmless (glycosides. Other substances taste salty but also bitter. strychnine and nicotine. alkaline. esters and aldehydes and tannins in wines and tea).Sensory Evaluation The five basic tastes A basic taste is one for which specific taste buds have been identified as being physiologically responsible for the particular taste sensation. Originally it was thought that bitterness was an indication of danger (poison). Other substances such as lead salts. non-nutritive sweeteners (cyclamates. saccharin and aspartame ) also taste sweet. amino acids. Sourness This is the simplest taste as only acids (H+) produce sourness and as the (H+) increases the sourness increases However there are some anomalies to this: • • • • • • • organic acids are more acidic than expected. However. bitterness is mainly associated with alkaloids such as caffeine. many alkaloids are used as drugs (e.

and • Sex. there are many other compounds which contain glutamate and which are capable of producing the savoury. The most notable example is mono-sodium glutamate (MSG).6 seconds and therefore if there is no response within this time the level is sub-threshold. Taste thresholds and sensitivity There is great variability between individuals in their levels of sensitivity. Sensitivity is affected by: • Temperature. spicy. • Sleep. • Hunger. Taste interactions Having described the 5 basic tastes it is obvious that foods are a very complex system which contain many different taste compounds and therefore many different tastes. This loss in sensitivity varies considerably with the taste (sweet. For example. recognition times vary between the basic tastes COPYRIGHT 18 R L Mason and S M Nottingham .Sensory Evaluation Umami Umami is the taste that has been shown to be associated with substances that contain glutamate.0. sour. sensitivity decreases due to adaptation and fatigue. MSG is well known as a flavour enhancer and can cause adverse reactions in some sensitive individuals. salty or bitter) and also with the compound.2 . However. tasting a series of acids causes the sensitivity to be reduced by the preceding acids. brothy taste associated with MSG. The fact that there are only 5 basic tastes and yet we are able to detect hundreds of different taste sensations is due to a series of complex taste interactions that can range from simple 2 way interactions to complex 5 way interactions Interactions between the 4 basic tastes were previously described simplistically by the taste tetrahedron. Recognition threshold is generally higher than detection threshold. Adaptation and fatigue During exposure to a stimulus. However. recovery is usually rapid because most common organic acids are very soluble.Concentration at which the specific taste can be identified. Many foods contain naturally high levels of glutamate. • Age.Concentration of stimulus at which a subject can detect a difference between two samples in a paired test. Absolute/Detection threshold . Both absolute threshold and recognition threshold will vary between individuals. However. Most people can detect taste within 0. Recognition threshold .

sweet.0s Vision = 0.4s Sour = 0.salt. and Tapping a melon for quality. ·Different areas of the tongue respond to different sensations.0s) and the sensation lingers considerably after tasting.01s Touch = 0. ·Flavour of the food is a complex interaction of different tastes and odours. Positive aspects: • • • • Snap. bitter and umami.02s Hearing = 0. Fizz of champagne or beer.005s Reaction times also relate to retention times for example. Crispiness of lettuce or celery. The sound of food when it is being eaten is an important aspect in determining quality. COPYRIGHT 19 R L Mason and S M Nottingham .5s Bitter = 1. crackle and pop. hearing Hearing Sound is the perception by humans of vibrations in a physical medium (air). Summary • • • • • ·Five types of taste receptors .Sensory Evaluation • • • • • • • Salt = 0. ·Sensitivity to taste varies between individuals and is affected by their physiological state. THE SENSE OF HEARING (Audition) Stimuli = physical movement of sound waves in a medium (air) Receptor= ear drum Perception=sound. bitterness has the longest reaction time (1.3s Sweet = 0. ·Substances must be dissolved for taste buds to detect them. sour.

Finger feel Firmness/Softness indicates the eating quality of some food products: • • • • Ripeness level of fruit such as avocado and mango. . e. stone cells in fruit. Firmness of cheese. hard. meat.Sensory Evaluation Negative aspects: noisy environment may distract tasters or mask product sounds. Juiciness can be used as a subjective quality index (eg the “thumbnail” test for corn). coarse. crunchy. tough. firm. texture. feel. e. Kinesthetics) Stimuli = physical contact between the food and body tissue Receptors= muscles and nerves in mouth and fingers Perception=touch. milk.thin to viscous. grainy. viscosity Texture usually relates to solid food while viscosity relates to homogeneous liquid foods and consistency relates to non-homogeneous liquid foods. e.g. brittle. Textural Terminology Hardness Brittleness Chewiness Grittiness COPYRIGHT Mechanical Characteristics Soft. fruit ripeness. Mouth feel Liquids • • Solids Classification of textural characteristics . chewy. Instrumental methods only measure one aspect of "texture" and again cannot relate the complex interactions which produce the perception of food texture. Consistency .g. and Spreadability of butter or spread. Gritty. R L Mason and S M Nottingham 20 Viscosity . e. THE SENSE OF TOUCH (Texture. e. Crumbly.g.g. fruit yoghurts.g. e. muesli bars and biscuits Tender. "sand" in ice-cream.g.assessed mainly by chewing. Crumb texture of bread. cheese maturity.thin to thick. cream.

Other interactions include: • • • Odour .g. caffeine have all been shown to be lower in water than in tomato sauce.sight This is a very important aspect because vision is the first sense affected and appearance of a product will have a major influence on absolute quality.g. cracker biscuit. string/fibre in vegetables. Types of sensory interactions Taste .Sight Odour .odour Receptors for these two senses are very close so that interactions between these senses are highly likely and these may be important in classifying a particular taste. cheeses.tactile The taste threshold for sugar. french fries. e. It is not known whether interactions occur at the receptor site or the brain. e. greasy. chips. Negative . moist. pink milkshake with pineapple flavour. cellular. Oily. wet. fatty. There are two aspects of this: Positive . salt.g. This may be due to the fact that in more viscous solutions the chemicals do not react with the receptors as easily as in pure solutions. e.g. Interaction between senses This is the ability of a response from one modality to influence or affect the response from another. e. COPYRIGHT .interactions giving clues to possible identity. Taste . pink milkshake being strawberry flavoured. the second option would appear to be more likely. interactions may occur. Dry. water melon.g.Sensory Evaluation Fibrousness Moistness Oiliness/Greasiness SENSORY INTERACTION As has been indicated previously when eating or tasting food there is a continuous relationship between the senses and unless steps are taken to separate the individual senses or stimuli.Tactile Taste – Hearing 21 R L Mason and S M Nottingham Fibrous. Taste . Bright colours indicate strong flavours whereas dull colours indicate mild flavours. However. e.If clues are not correct this may lead to confusion and a wrong judgement.

g. Example: Tasting food pureed. neutralisation of one flavour by another. However. fruit in cheese.g.Hearing Multiple interactions Multiple interactions between more than two modalities are also possible. Summary Interaction must be considered when designing sensory panels. if interactions are required then ensure this can be achieved by means of sample preparation. e. This is the basis of ensuring brix/acid ratio for fruit juices are constant. Similar situations may exist for all other stimuli. e. COPYRIGHT 22 R L Mason and S M Nottingham . no effect.Sensory Evaluation • Odour . Interactions between stimuli These interactions are more difficult to define and measure but are just as important as interactions between the senses. partial blending producing a new flavour and the original flavours.g. blindfolded and with nose clips gives a different response than when interactions are allowed. e. original flavours are distinct and separate. blending to produce a totally different flavour. salt and MSG on food improves the natural flavours.g. intensification resulting in enhancement of flavours. If only one sense or stimulus is to be evaluated then all others must be masked. garlic flavoured cheeses. e. Some examples include: • • • • • • suppression of one flavour by another. sweetness is suppressed by acidity.

e. Sample preparation and serving COPYRIGHT 23 R L Mason and S M Nottingham It must be maintainable. Serving samples. Give feedback on results whenever possible. Make tasting interesting and desirable. Serving temperature . This prevents panellists from influencing one another. This helps to eliminate the numerous errors or biases that can be caused by psychological and physiological factors.g. Preparing the testing environment. Designing the experiment. Make sure the environment gives optimum opportunity for concentration. e. sample size and temperature). Eating utensils. when running taste panels. Avoid giving any unnecessary information to panellists that may influence their scores. Motivated tasters are more efficient.standardise for all samples. These relate to: • • • • • Selection of panellists. as stringently as circumstances allow. There are therefore a number of basic rules which should always be applied. and the atmosphere of the testing environment all influence their judgements. Plan your experiment in advance. i.g.Sensory Evaluation OPERATIONAL PRINCIPLES OF SENSORY TESTING When evaluating properties of foods using people as measuring instruments it is important to control the methods and conditions of testing as rigidly as possibly.adequate but not excessive. Make sure that the "correct" panellists are selected (see section on panel selection and training) and that they know in advance when they will be required. Which will be the best test to use? Consider all aspects including how you will get the information required from your results (statistics). Serving vessels. General principles that should always be followed are: Never ask anyone to taste food they do not like. Preparing samples. acceptable temperature for the food. and be an . Tasters usually find what they expect to find. in a storage test they expect to find samples deteriorating. vary these and choose foods that contrast with those being tasted. Keep a strict control over all variables except those being tested (e. Train panellists to be silent while tasting. practise and choose the best method for: Sample size . Tasting properly is a difficult job. The mental attitude and physical condition of a taster. Run preliminary tests. Use rewards to motivate taters.

coloured lighting). give courteous friendly service. Record temperatures and size of samples served and any special conditions (e. Run a taste panel as you would expect a good restaurant to be run.e.g. It is important that panellists do not see the samples being prepared as this may indicate quality difference. i. be efficient.Sensory Evaluation Serve tasters promptly and make sure they have everything they need. Sample preparation should be uniform: • • • • Temperature Cooking Thawing Size and shape (provided this is not a variable) Sample should be randomly allocated to: • • Avoid bias Overcome any non—uniformity Sample size should be adequate: • • 30g solids 30mL liquids Samples should be served immediately after preparation to reduce: • • • Flavour loss Discoloration Textural changes Sufficient samples should be prepared to allow for seconds COPYRIGHT 24 R L Mason and S M Nottingham . and serve good food. Keep accurate records of any cooking or preparation methods used.

Carriers are substances that are added to assist tasting of certain products. etc. some products such as spices.Sensory Evaluation Containers for presentation Containers for presentation and tasting should be: • • • • • Clean Identical for all samples and sessions Disposable containers or re—usable Coloured to mask product appearance (if required) Relevant to product Serving temperature • • • • Serve at room temperature where possible Preference tests use normal temperature Difference tests may alter temperature to accentuate flavours/odours Do not overheat: too hot to taste drying out off flavours browning Dilutions and Carriers Most foods should be served in the way they are normally eaten. If dilutions are used they must be uniform in terms of diluent and concentration. may require dilution before testing. Carriers are a problem because they can be: • • • • Expensive Time consuming Variable quality Difficult to control product/carriers ratio uniformity. However. For example: developing a cake icing individually may not allow for interaction with flavour or it may be incompatible with the cake (affects texture or falls off).strong flavour —> less samples Type of test Rating scale may require fewer samples Test dictates sample number eg: triangle test = 3 samples 25 R L Mason and S M Nottingham COPYRIGHT . onions. Number of samples Samples / Sessions The number of samples presented at any testing session will depend on: • • • • Type of product . chillies. alcohol.

It also gives you practice at preparing and serving the quantity of samples needed. In calculating the number of sessions consider the following: • • • • • Total number of samples for tasting Statistical design Taster fatigue Motivation Type of panel (trained/untrained) Phsiological factors in taste testing Time of Tests • • • • • Monday and Friday are recognised as being bad days for tasting Normally taste 1 hour before meals and 1 . smoking affects sensitivity to flavours —therefore should either: • • • Illness Sensitivity of people suffering from illness is reduced -particularly those with colds or flu (physical and psychological) Likes / Dislikes In preference testing a series of treatments within a specific product type. it is legitimate to eliminate people who dislike the product (or those who are not discriminatory).2 hours before tasting Chewing gum. Sessions / Trials Before starting your scheduled tasting sessions run two preliminary sessions. These will familiarise your panel with the scoresheet.2 hours after Sometimes this becomes difficult in practice due to: Unavailability of tasters Number of sessions Smoking / Taste Affecting Substances As indicated earlier. Palate Clearing COPYRIGHT 26 R L Mason and S M Nottingham Not use smokers Ensure they do not smoke for at least 1 .Sensory Evaluation • • Type of panel — trained / experienced -> more Experimental design As a general rule usually not more than 6 samples/sessions. and a last chance to iron out any unforeseen problems. the products to be tested and the procedures you wish them to follow. mints and spices etc may also influence taste .

Motivating panellists by can reduce this problem by: • • • • • • • • • Stressing importance of work Stimulating company expansion Greater profits More pay Ensuring panellists know what is involved with the trial ie: sessions. • • • Single digit numbers Consecutive letters Same codes at consecutive sessions Randomly or statistically generated three digit number codes are best. apples may be used as a palate cleaning agent. Tasting becomes a chore when there are large numbers of samples/sessions involved. biscuits. it is necessary to allow for any psychological factors that may influence results and possibly lead to errors. bread. Motivation Good results can only be obtained from a co-operative. The time between samples should also be kept constant if possible Perfumes / Spices Ask panellists to refrain from wearing strong perfumes or breathing spicy odours wherever possible.Sensory Evaluation It is a good idea to get panellists to cleanse their palate: • • • Before tasting to remove any lingering tastes Between samples to reduce adaptation of taste buds. responsive panel. products. Do not use. when and where tasting will be conducted Having adequate facilities Using effective methods and designs Publicising results obtained from work Rewarding panellists Sample Coding Remove possible bias or influence from samples codes. Order of Presentation Always use either a random order of presentation or a statistically balanced design to avoid: COPYRIGHT 27 R L Mason and S M Nottingham . Palate clearing can be optional but whatever is done must be constant. Psychological factors Because sensory evaluation is a subjective system. Warm water.

Sensory Evaluation • • • Donkey vote (first is best; last is worst) Position bias - in triangle tests middle one is different Contrast effect — good after bad appears better, or bad after good appears worse.

Devise your own system for remembering orders, e.g. 3 digit numbers - put in sequence of one of digits. Keep it a secret! Always work systematically in coding, labelling, setting up, e.g. as in reading a page (1) (2) Left to Right Top to Bottom

This provides an automatic check if something goes wrong. Balance presentation of samples whenever possible. This avoids contrast effect. ie. 2 samples A, B. Half panel taste A first, other half taste B first. Half panel receive A on the left, other half receive B on the left. 6 different orders in which they are tasted. Use every order the same number of times. Number of tasters is a multiple of six. Position of samples on plate must also be balanced. 24 different orders: use them all if possible (see table on next page). Generate random order. Write out set of cards and shuffle them.

3 samples 4 samples 4 samples

When you cannot use balance to eliminate bias, use randomisation.



R L Mason and S M Nottingham

Sensory Evaluation Four sample balanced orders 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Expectation Error Any information a panellist receives before a test will influence the results. This is called expectation error. To overcome this: • • • Do not give detailed information about treatments Do not use people on panel who know what the treatments are Sample coding and design can prevent expectation error A A A A A A B B B B B B C C C C C C D D D D D D B B C C D D A A C C D D A A B B D D A A B B C C C D B D B C C D A D A C B D A D A B B C A C A B D C D B C B D C D A C A D B D A B A C B C A B A

Logical / Stimulus Error Tasters look for clues to get the “right” answer eg: a difference in sweetness may be associated with sample differences such as size, shape and colour. This error can be overcome by ensuring sample preparation is uniform or use masking. Halo Effect When more than one factor in a sample is evaluated at one time the result obtained may be different than if factors evaluated separately. This can be overcome by tasting each aspect separately. However, this is time consuming and would only be done if extremely accurate results were required. Testing one aspect at a time in preference does not simulate the “real COPYRIGHT R L Mason and S M Nottingham 29

Sensory Evaluation situation” ie: consumers do not taste every aspect separately. Suggestion Influence of other panellist may bias or influence results. This can be prevented by: • • • Using booths Not allowing talking in tasting area Reducing outside distractions

Questionnaire design Questionnaire design should be simple and easy to follow in terms of design and language and make sure tasters know how to use it. You may need to include some instructions on the scoresheet itself, but it is usually better to give instructions verbally to your panel first. The questionnaire should generally not be more than one page and include: • • • • • • • Name Date Time Product Sample codes Instructions Comments section



R L Mason and S M Nottingham

and Toilets.constant. Use odourless cleaning agents. Office. General testing area • • • • • • • Easily accessible but in quiet position. Preparation area/kitchen. Use odourless materials in construction and decoration. Total area should include: • • • • • • Testing area with individual booths and a group area.close proximity to preparation area. and with complete "close-off" capability.Sensory Evaluation DESIGN OF A SENSORY TESTING AREA The main considerations to keep in mind when preparing an area for sensory testing concern the requirements for an atmosphere conducive to concentration. Odours . where conditions can be controlled. The conditions should be controlled in order to : • Reduce bias • Improve accuracy • Improve sensitivity • (compare to the conditions used in an analytical laboratory) International standard (ISO 8589-1988) The standard looks at the design of the testing area for both new and existing buildings. Product characteristics can be markedly affected by temperature and humidity. and comfortable.keep to a minimum. Location . It also specifies which recommendations are considered essential and which are only desirable.keep area free from odours (air conditioner with carbon filters. these should be seriously considered. but separate entrance. and appearance is affected by lighting intensity. controllable. Noise . Rest room. slight positive pressure). Sensory panellists need somewhere comfortable and free from distractions if they are to be able to "tune in" to the sensations triggered by the stimuli in the food products they are tasting. Important points summarised from the standard are listed below. Cloakroom. If designing an area that is to be dedicated solely to taste panel work. Temperature and relative humidity . soundproof area as much as possible. 31 R L Mason and S M Nottingham COPYRIGHT .

Include comfortable seats. light colours for walls and furniture (e. Storage space for crockery etc. bookcase. Include large table and several chairs. but they are not always feasible. utensils. filing cabinet. rinsing agents. Temporary acceptable.allow sufficient space for movement of tasters and for serving samples. off-white.e. Office area General Separate but close to testing area. cooking equipment. computerised equipment. dishwasher. Set-up . either from the point of view of financial COPYRIGHT 32 R L Mason and S M Nottingham . sinks.permanent booths recommended. normally five to ten . Space . shadow-free and controllable. refrigerator. scoresheets and pens. Group work area General Necessary for discussion and training purposes. electricity).ambient lighting must be uniform.Sensory Evaluation • • Booths Number . glassware etc for serving samples. Additional areas Useful to include rest room.minimum three. Lighting . controllable.use neutral. Practical alternatives The requirements specified in the International Standard (ISO 8589) will obviously provide a suitable area. gas. Decoration . Preparation area General Located close to assessment areas but no access to tasters.uniform. adequate intensity for assessing appearance.g. If adjacent to preparation area include openings in the wall to pass samples through. etc. desk. Size and style specified. with coloured lighting options like booths. For consumer testing . Flexible services (i. shadow-free. cloak room and toilets. Lighting . Crockery. etc. Include board for discussion notes. Devices to mask appearance ( close to home conditions as possible.six is a useful number since it fits in well with balanced ordering of 3 samples. Include working surfaces. freezer. dimmers. coloured lights or filters). Consider space for samples. Lighting As for general area. Equipment Depending on testing required. "Lazy Susan" useful. spittoons. Well ventilated. Design for efficient work-flow. light grey). reasonable size. plumbing.g. computer. Photocopying service needed.

All equipment likely to be needed while a panellist is tasting. Hand washing facilities. spittoons. Washing up facilities . If possible position these at opposite ends of the room to avoid messy paperwork! Testing area with entrance separate from preparation area. toothpicks.minimum double sink with hot and cold running water. efficient lighting.g. Waiting are with noticeboard . Refrigerator . pencils.large with liner bags.Sensory Evaluation resources or physical space available. They can be made specifically to fit any available benches or tables and folded and stored when not in use. tissues/serviettes. Minimum of 2 areas: Preparation area and office area.depending on sample requirements. Minimum space . I therefore would like to abbreviate the list proposed in the standard to one which I consider includes the bare essentials. e. or "corflute". Well placed. Rubbish bin . A system using collapsible booths can work quite well if it is not possible to keep an area solely for sensory work. heavy duty cardboard. Preparation area requires • • • • • • • • Adequate storage for utensils and equipment. Very few industries are able to start from scratch. Table which can be easily divided into booths if required.for tasters to wait for booths to become free and to collect rewards after tasting. These may be made of painted wood. Adequate working surfaces to set out samples.minimum 2 door with separate freezer. Source of boiling water. preferably at least auto-defrost. The type of facility will depend on: • • • • Finance Available space Frequency of use Tests conducted COPYRIGHT 33 R L Mason and S M Nottingham .4 panellists. Cooking equipment . Testing area requires • • • • • • Comfortable chairs for panellists. designing new premises solely dedicated to sensory analysis work.

How can we describe our data? Lets say we have collected some data from an experiment and we have 20 scores of flavour acceptability in a mango sample rated on a 9 point hedonic scale. eg adaptation differences among samples. COPYRIGHT 34 R L Mason and S M Nottingham . An example is shown below. Why do we need statistics in sensory evaluation? When we measure something (eg salt level in cheese) we find there is variation in what we are measuring. Some of these include. There are many sources of error in sensory data. If we plot a bar graph (histogram) using the score along the horizontal axis and the count for a particular score on the vertical axis then we have a frequency distribution. • • • • • differences between people.Sensory Evaluation STATISTICAL PRINCIPLES This section looks at the role of statistics in sensory evaluation and introduces some terms and concepts required to correctly apply statistical methods in evaluating sensory type data. differences in interpretations of scales and many more. Using statistics we have rules to estimate and minimise the risk and enable us to extrapolate our results from an experiment to a more general situation. (likes and dislikes) differences within a person from time to time. This is a fact of life and we have limited control over this sort of error. What is an experiment? It is any process that generates raw data. This variation is called natural variation or experimental error and implies that there is some true measurement but because of our limitations we cannot reproduce the correct readings every time. Because of this variation there is some risk in making decisions about changing formulations or introducing new products onto the market.

Sensory Evaluation 6 5 Frequency 4 3 2 1 0 3 4 5 6 7 8 9 Flavour Acceptability Looking at the graph or distribution we ask what is the best single estimate of the panels score and what is a good measure of their variability? The best or most likely single estimates are called measures of central tendency.or average (sum of all data values divided by number of observations) median . The range is simply the difference between the smallest and the largest. s= ∑ ( X − M )2 ( N − 1) where M is the mean or average of X scores and N is the number of scores.50th percentile or middle value mode . COPYRIGHT 35 R L Mason and S M Nottingham . The three most commonly used are: mean .most frequent value. The standard deviation is probably the most common and is calculated by using the formula below. This formula calculates the deviation of each score from the mean and squares it to take into account positive and negative values and the square root is then taken to bring it back to the original units. good for categorical data Measures of variability include the range. The variance is simply the square of the standard deviation and is used in a number of statistical formulas. standard deviation and variance.

0 and standard deviation of 1.89.05) than that for artificially ripened mango.0.00 then 66% of the values would be between 5. By making this generalisation we often express our results in terms of probability or p.01 is used for greater precision. Z = X-µ/σ Since z-scores are related to percentages under the normal curve they can predict how far a score is from the mean and how likely or unlikely it is. Replication is the assessment of each treatment more than once. Since we cannot take all possible results from the population we infer from our sample results what should happen in the rest of the population. X can be described in terms of a z-value.scores to p – values. With replication we COPYRIGHT 36 R L Mason and S M Nottingham .values. In addition any score. This is our safety margin or level of confidence about our result. If the standard deviation had been 1. An important concept When we do an experiment we are using results from a sample taken from a larger population of possible results.0 and 7. So for a normal distribution about 66% of our data will be within one standard deviation of the mean and about 95% will be within two standard deviations.11 and 7. But what does this mean? We are at least 95% certain that based on our experimental conditions the naturally ripened mango will have more flavour than artificially ripened mangoes. A treatment can be the addition of sweetener to a product or the storage temperature of a fruit. a smaller range indicating less variability. For our mango flavour data with a mean of 6. How does all this help? We need to identify some more concepts before we can be confident in using statistics. This p . Sometimes a 1 % value or 0. which describes how far the score is from the mean in standard deviation units.value is found from the area under the curve outside the z score and is the chance with which we would see a score of that size or greater. How does the standard deviation relate to the normal distribution? Standard deviations describe discrete percentages of observations at certain degrees of difference from the mean. This conclusion will be wrong about five times out of 100.Sensory Evaluation The normal distribution Many things we measure about a group of people will be normally distributed.89 then 66% of our data lie between 4. Tables are often used to convert z . This means they will form a bell shaped curve described by an equation usually attributed to Gauss. Experiments need to be planned and carried out correctly before we can use statistics and two important principles are replication and randomisation. So the z-score can be converted to a probability value or p – value. It is often quoted like this the flavour score for naturally ripened mango was significantly higher (P<0.

Is our signal greater than the background noise (natural variation)? Random allocation of treatments to samples or products ensures each sample has an equal opportunity of receiving any treatment. and that this chance is unaffected by the treatments assigned to other samples. COPYRIGHT 37 R L Mason and S M Nottingham .Sensory Evaluation can assess the natural variability and separate this from our variability due to treatment differences. Subjective allocation of treatments in a haphazard way is not a satisfactory alternative to randomisation. For example if two products are tasted by 24 tasters and they all taste product A first then this may well bias the results. This is like a signal to noise ratio. as the first product tasted may tend to be preferred regardless of which it is.

100 = consumer panel COPYRIGHT 38 R L Mason and S M Nottingham . panellists should be Panel size 1. instructed/briefed in terms of: • • • • Method Questionnaire Length of trial Number of samples However. either overall or in relation to a particular parameter. 20 = pilot consumer panel 3. in terms of technique or ability. Acceptance infers actual utilisation/purchase of the product.Sensory Evaluation SENSORY EVALUATION METHODS There are two main types of sensory methods: Affective :tests which involve consumer preference or acceptance Analytical : tests which are involved with analyzing specific product attributes in terms of: • • discrimination/difference description AFFECTIVE TESTS Preference infers a preference for one product over another. 20 to 100 people 2. Panel selection Select panel on basis of end use: • Age • Race • Religion • Sex or • Random selection for overall Panel training No need for training.

Principle: a pair of samples (one may be a control) is presented to each assessor.Sensory Evaluation SPECIFIC TEST METHODS PAIRED PREFERENCE TEST (Reference: AS 2542.TIME………ASSESSOR…… ……………. The assessors are asked to choose the sample they prefer.) Application: to establish whether there is a preference between two samples. 1982.2.DATE…………. Please tick the appropriate box. Which sample do you prefer? Please examine code 349 first. Conclude that this sample is significantly preferred to the other if the number obtained is greater than or equal to that shown in Table 4. The test supervisor accepts a 5% level COPYRIGHT 39 R L Mason and S M Nottingham . Code Place tick 922 349 YOU MUST MAKE A CHOICE Conclusions • • • no preference A preferred to B B preferred to A Question — which of the two samples do you prefer? Count the number of replies citing one of the two samples the more frequently. Statistically based on null hypothesis that there is no preference between the expectation of preferences Specimen Answer form for bilateral paired preference test PRODUCT…………………. ie:PA = PB = 50%= 0 5 Bilateral Test . Example: Two drinks ‘A’ and ‘B’. are offered to a panel of 30 assessors. This test is a ‘forced choice’ ie: the assessors must select one sample as being more preferable... Responses indicating no preference are not permitted.1. The two samples are presented under random number eg: ‘789’ and ‘379’.

Do you prefer sample ‘A’ to sample ‘B’? Replies: 23 Yes and 7 No.Sensory Evaluation of significance (ie: P < 0. Specimen Answer form for unilateral paired preference test PRODUCT…………………. The two samples are presented under a random number eg: ‘789’ and ‘379’. The test supervisor accepts a 1% level of significance (ie: P < 0..DATE…………. Question . Example: Two drinks. Unilateral Test . From Table 3 it can be concluded that there is preference for drink ‘A’ over drink ‘B’.Which sample do you prefer? Replies: 22 prefer ‘A’ 8 prefer ‘B’ From Table 4 it can be concluded that Drink ‘A’ is preferred to Drink ‘B’.expect one sample to be preferred.05%). Do you prefer sample 186 to sample 592? Please examine code 592 first.TIME………ASSESSOR…… …………….01%). It is known that drink ‘A’ contains more sugar than drink ‘B’. COPYRIGHT 40 R L Mason and S M Nottingham .. ‘A’ and ‘B’. It is not known which of the two samples contains more sugar. are offered to a panel of 30 assessors. Please tick the appropriate box. YES YOU MUST MAKE A CHOICE NO Conclusion • • no preference the declared sample is preferred Question — Do you prefer sample ‘A’ to sample ‘B’? Conclude sample A is preferred if number of positive replies is greater or equal to the number shown in Table 3. Question .

B. Please taste the samples in the order presented. Give the sample that you most prefer the a rank of 1 and the sample you prefer next a rank of 2 etc. You must give each sample a different rank. Specimen Answer form for ranking for preference. STP 434.2. 1968) No magnitude of preference is given ie they both may be disliked but one can still be preferred. Advantages • • • Simple test to conduct Suitable for children and consumer panels Easy to analyse (for > 100 assessors use t test or CHI squared) Disadvantages • Only suitable for 2 products (note – multiple Comparisons can be used but other preferences tests are more commonly used.TIME………ASSESSOR…… ……………. You may retaste the samples to check the ranking.. • Applications • • • Product Development Product Matching Process Change RANKING FOR PREFERENCE (Australian Standard 2542. Ranking is a forced choice procedure ie no ties are allowed.6) Principle: Judges are asked to rank two or more samples in order or preference ie: most preferred sample is ranked first. If uncertain always use the bilateral test.. See ASTM manual on sensory testing method.DATE…………. Samples Rank COPYRIGHT 41 R L Mason and S M Nottingham . moving from left to right and rank them in order of preference. Equal ranks are not allowed. PRODUCT………………….Sensory Evaluation N.

They were asked to use the samples as directed and to rank them in order of preference. which have been used in the past to analyses differences between rank sums. Example Twelve households were presented with four samples of meat seasoning to be used in cooking. When there is no expectation of a specific rank order being made (eg when ranking preference of new product prototypes) the Friedman Test should be used (see statistical method s section for details).Sensory Evaluation Statistical analysis Kramer’s tables. The results are shown below: COPYRIGHT 42 R L Mason and S M Nottingham . should not be used due to questions of accuracy and statistical validity.

Two samples will be significantly different if the absolute value of the difference between the rank sums is greater than or equal to the following critical value: 12 × 4(4 + 1) = 12.960 B 36b C 26ab D 38b Rank sums that do not have a common superscript are significantly different (P<0.05) difference between the rank sums.8-180 =10.81.8 is greater than 7. since 10.05) COPYRIGHT 43 R L Mason and S M Nottingham .81 for 3 df).396 6 Sample A Rank Sum 20a 1.Sensory Evaluation Rankings for the preference of four meat seasonings HOUSHOLD 1 2 3 4 5 6 7 8 9 10 11 12 Rank sums Seasoning A 1 2 1 1 2 3 3 3 1 1 1 1 20 B 3 1 4 4 3 4 4 4 2 2 2 3 36 C 2 3 2 2 1 2 2 1 3 3 3 2 26 D 4 4 3 3 4 1 1 2 4 4 4 4 38 The F value is calculated as follows: 12 (20 2 + 36 2 + 26 2 + 38 2 ) − 3 × 12(4 + 1) F= 12 × 4(4 + 1) =190. the experimenter can conclude that there is a significant (p<0.8 the calculated value is compared to the critical f value in table 7 (7.

the more chance there is of obtaining a significant result. Personal preferences in foods are being measured which are purely subjective. Pilot consumer panel = 20-25 Consumer panel = 100 Types of response scale Category scale/structured scale The response scale is divided into categories or boxes.Sensory Evaluation RATING FOR PREFERENCE (Australian Standard 2542. Hedonic category rating AROMA Like extremely Like very much Like moderately Like slightly Neither like nor dislike Dislike slightly Dislike moderately Dislike very much Dislike extremely FLAVOUR TEXTURE COPYRIGHT 44 R L Mason and S M Nottingham . The response scale is usually divided into an arbitrary number of categories . This makes it more difficult to obtain statistically significant results.2. Verbal descriptors or facial expressions may be used to identify the levels of acceptance. so the variance in the data is large.usually between 7 and 13 Category scales must be bipolar.30 Principle Assessors are asked to evaluate one or more samples and indicate the degree of liking for the product or some characteristic of the product. Only untrained panellists are used and should be selected at random or from a targeted group related to the product. The larger the panel. When performing preference testing it is important to include as many panellists as possible.

Sensory Evaluation Facial hedonic scale 7 point facial hedonic scale Appearance Aroma Flavour Mouth-feel Graphic rating scale • • • • • The response is recorded by marking a position on a line Also called visual-analogue scale. The arithmetic mean and standard deviation. the analysis of variance technique is appropriate (or a t-test in the case of one or two samples). and Can measure > one parameter at a time. and Results may be biased by type of assessors used. Numbers and/or descriptors are usually attached to a rating scale. Recording and interpretation of results Ratings must be converted to numerical scores for analysis and interpretation. successive integers are assigned to successive categories and these are used in analysis. Disadvantages • • Statistical analysis is required. Correlation or regression analysis may be used for subjective/objective correlations. For statistical analysis. Advantages • • • • • Test is relatively simple and easily understood. in mm. Suitable for different age groups. e.g. serve as measures of central tendency and variability. Can be used to infer acceptance. This scale may also use facial expressions for measurement. For category scales. COPYRIGHT 45 R L Mason and S M Nottingham . respectively.g. with a 9-point scale. between the response mark and one end of the scale serves as the response score. the distance. the integers 1-9 would be used. Indicates the degree of preference. when obtained for each sample. For graphic scales. e. line mark scale or unstructured scale -Physical lengths 100-150 mm.

It is important to note for example in paired preference testing that although one product may be preferred to another. or specific characteristics of a product is collectively grouped under what we call consumer testing.4 b Scores within each row that do not have a suffix in common are significantly different. texture and general acceptability of the products using a 13 point The following results were obtained: CHICKEN CASEROLE Appearance Flavour Texture General Acceptability A 10. However.5 d 9.Sensory Evaluation Applications • • • • • • storage trials product development consumer testing quality control subjective/objective correlations research Example: Three samples of frozen chicken casserole were presented to a 24 member panel who assessed the appearance.3 a B 8.9a 10. a product concept.2 b 9.2 b C 10. Unfortunately ‘preference’ is widely used as a generic term to describe both acceptance and preference judgements.6 b 9.4 a 10. it is important to define the terms acceptance and preference often associated with consumer testing.4 b 9. The term ‘hedonic’ is an adjective associated with degrees of pleasure or displeasure and is applied to both acceptance and preference testing.1 b 9.8 a 9.3 b 9. flavour. neither product may be liked to any degree. COPYRIGHT 46 R L Mason and S M Nottingham . Acceptance refers to the degree of liking or disliking for a particular product or the ability of the product to meet expectations of consumers while preference refers to a choice made by panellists among several products on the basis of liking or disliking. SENSORY EVALUATION IN CONSUMER TESTING Introduction The personal response by current or potential customers of a product.

process and formulation changes and packaging modifications without affecting the product characteristics and overall acceptance. production site. COPYRIGHT 47 R L Mason and S M Nottingham . condition. tested by a trained panel to verify that the desired attribute differences are perceptible. For product optimisation. Usually difference tests would be used to determine whether a difference was perceived or not but it is necessary to take the product out to the consumer to determine if the reformulated product will achieve at least parity with the current product. Product improvement/optimisation The intense competition among consumer products drives companies to constantly improve and optimise products so that they can deliver what the consumer is really looking for and therefore increase market share. substitution of ingredients. consumer testing should be used throughout in conjunction with trained panel assessment. In product improvement.Sensory Evaluation Applications of Consumer Testing The reasons for conducting consumer tests usually fall into one of the following categories: • • • • Product maintenance Product improvement/optimization Development of new products Assessment of market potential Product maintenance Research and development projects may involve cost reduction. Development of new products During the new product development from concept to a range of trial samples to a modified sample range and finally a choice to launch. ingredients or process variables are manipulated and a trained panel identifies the key sensory attributes affected and consumer tests are conducted to determine if consumers perceive the change in attributes and if such modifications improve the overall acceptability. Product maintenance is also a key issue with quality control/quality assurance and shelf-life/ storage projects. Feedback on consumer response gives important information on those sensory characteristics that are most important to consumer choice and which should therefore be rigorously controlled. raw material sources etc can be used in conjunction with consumer testing to determine how large a difference is sufficient to change the acceptance rating. A combination of in-house profile testing on the magnitude and type of change over time. and then tested with consumers to determine the degree of perceived product improvement and its effect on overall acceptance or preference scores. prototypes are made.

Some typical designs used include: • • • • Monadic test where only one product is assessed which makes it fast and the least expensive but is relatively insensitive and requires large numbers of consumers (at least 200). Essentially small groups are used to uncover as much specific information from as many participants as possible. COPYRIGHT 48 R L Mason and S M Nottingham . one is qualitative measuring subjective responses while the other is quantitative determining the responses of a large group to a set of questions regarding preference. Sequential monadic where one product is assessed. current purchase habits. Usually the nine-point hedonic scale is used to determine consumers liking of a product and if required the relative ratings for liking can be used as a measure of preference.Sensory Evaluation Assessment of market potential In addition to the use of sensory evaluation to gather information about key attributes of a product. Acceptability testing. Qualitative Tests include focus groups. effects of packaging. focus panels and one-on-one interviews. Quantitative Tests Essentially all the good practice principles used in sensory evaluation as described in the difference and descriptive testing should be followed here such as 3 digit random codes for product and presentation in a balanced order. liking. It is often convenient for these marketing type questions to be included in a questionnaire presented to consumers when assessing the sensory characteristics of the product. advertising and convenience are critical for the acceptance of branded products. Paired preference testing where two products are assessed simultaneously and a direct comparison is made making it quite sensitive. Conducting Consumer Tests There are a number of factors to consider when conducting consumer tests and these are: • • • • Test design Test subjects Test location Test questionnaire Test Design There are two main types of design. typical marketing questions such as intent to purchase. purchase price. Each of these has their use in a particular situation depending on what is required and how sensitive the topic is. removed and then replaced by a second product in a balanced design giving it greater sensitivity. consumer food habits. sensory properties etc. It is frequently recorded either by video and or audio and a summary is made.

it should always be remembered that this is a compromise. so a compromise can be made by using large numbers staff who assess fairly infrequently. line mark scale or unstructured scale) .Sensory Evaluation Example of nine-point hedonic category rating AROMA Like extremely Like very much Like moderately Like slightly Neither like nor dislike Dislike slightly Dislike moderately Dislike very much Dislike extremely Example of seven point facial hedonic scale often used for children Appearance Aroma Flavour Mouth-feel FLAVOUR TEXTURE Graphic rating scale . These can be hedonic type attributes or sensory attributes in the form of just right scales as shown below. this is not always practical in preliminary testing of products. not sweet enough just right too sweet Test Subjects If information on the acceptance of the product by consumers is required. • Attribute testing can be used to gain information on the reasons underlying overall preferences and usually category or line scales are used. However. This scale may also use facial expressions for measurement.the response is recorded by marking a position on a line (also called visual-analogue scale. then it is they who should do the tasting. However.physical lengths 100-150 mm. and results are best COPYRIGHT 49 R L Mason and S M Nottingham .

terms. are clinical and atypical of a real COPYRIGHT R L Mason and S M Nottingham 50 • . Personal preferences in foods are being measured which are purely subjective. Locations include: Company laboratory facilities. Recruitment The number of consumers to be tested depends on the purpose of the test. However. geographic location. Products targeted towards a specific part of the community or for export ideally should be tested in that environment. If a product has broad age appeal then consumers should be selected by age in proportion to their representation in the user population. religion. Test Location It is possible to conduct consumer testing in a number of locations depending on the resources and the results can vary greatly. representative sample of the target population. The larger the panel. Gender. This can be based on income or occupation although sometimes it is difficult to get consumers to reveal such information. education level. It is important to determine if you are looking for low. This makes it more difficult to obtain statistically significant results. telephone survey. Others including race. For speciality products or niche markets. so the variance in the data is large. colleges or door to door. the more chance there is of obtaining a significant result. In general we require 60 to 120 for most consumer testing. Nationality. However. employment. Social class. medium or high users of the product.Sensory Evaluation interpreted only in relative. Staff cannot be considered to be a When performing consumer testing it is important to include as many panellists as possible. which give good control of the environment and rapid feedback of results but the sensory booths. it is possible to use foreign nationals resident here but it depends on how long they have been residing in their adopted country as they can develop the likes and dislikes characteristic of the adopted country. Researchers should use current market information. • • • • • Source of Consumers As mentioned it is important to sample properly from the consuming population but because of cost restraints employees and local residents may be used for things such as product maintenance. shopping centres. the cost of consumer testing increases as more people must be contacted before the required number are found. not absolute. embassies. for new products or product optimisation or improvement the correct audience should be selected. leaflet drop. It is not always necessary to get equal numbers as purchasing or usage habits vary between products. These can come from a database of consumers willing to assess products. the test design and the precision with which the target population can be identified. If any of these are important in defining the target audience then the researcher should consider them. Age. Recruitment and selection of consumers rely on several criteria or demographics such as: • Product usage. etc.

However the conditions are artificial compared to normal use at home and the number of questions that can be asked may be limited. funding etc.Sensory Evaluation domestic environment. Home use tests represent the ultimate in consumer testing as the product is tested under its normal conditions of use. Agree strongly Agree Neither agree nor disagree Disagree Disagree strongly What did you like about the product? This open-ended question allows for the consumer to add something you may have forgotten but it is sometimes hard to read the answer (handwriting) and some people do not bother with answering. In addition to the product itself. Central location such as school or church halls or shopping centres are convenient as large numbers can be tested at one time and on a number of products. easy to read and understand. COPYRIGHT 51 R L Mason and S M Nottingham . In essence you need to be: • Brief • Use simple plain English (provide translation for studies involving foreigners) • Be specific • Multiple choice questions should be mutually exclusive • Avoid ambiguity • Watch the effects of wording • Don’t ask what they don’t know • Try and pre-test the questionnaire For example How satisfied or dissatisfied were you with the product? Very satisfied Slightly satisfied Neither satisfied nor dissatisfied Slightly dissatisfied Very dissatisfied The product looks like how it is shown on the package. • • Test Questionnaire It is very important that the test questionnaire format is simple. You need to consider the objective of the test and any constraints such as time. Generally more information can be gathered as the consumer gets more time and can perform repeated assessments. a check on the packaging can also be determined. However it is time consuming and uses a smaller number of people and the possibility of nonresponse is great unless consumers are continually reminded. unambiguous.

Sensory Evaluation The question order should go from the more general to the more specific and ask overall acceptability first before biasing the consumer with more specific issues. partial least squares and preference mapping can also be used. COPYRIGHT 52 R L Mason and S M Nottingham . Data Analysis All quantitative data should be subjected to some form of statistical analysis from simple summary statistics and graphical representation to t-tests and analysis of variance with pairwise comparisons. Further advanced multivariate methods such as principal components analysis and cluster analysis along with regression methods to relate consumer data to other data such as linear regression. Ask the more sensitive demographic questions last.

The triangle test is an effective method to determine whether a change in ingredient.1983) Scope and Application Used to determine whether a perceptible difference exists between two samples. A triangle test can also be used for COPYRIGHT 53 R L Mason and S M Nottingham . processing. packaging or storage has resulted in product differences. product matching. Examples of directional difference tests are: Paired comparison test Ranking Rating SIMPLE DIFFERENCE TEST TRIANGLE TEST (Australian Standard 2542.Sensory Evaluation ANALYTICAL SENSORY TESTS: In general. An untrained panel would require 20-100 panellists while a trained panel would require 5-20. These situations may arise in product and process development. The difference can involve one or several sensory attributes. in quality control or as a preliminary test prior to quantitative descriptive testing.2. analytical panels are used as “measuring instruments and therefore need to be: • • Valid (able to measure appropriate parameters) Reproducible Panellists can be trained or untrained depending on the degree of difference expected. DIFFERENCE TESTING Difference tests may be sub-divided into 2 classes: • Simple difference tests are those that have no direction or characteristic associated with the difference between the products. Examples of simple difference tests are: Triangle test Duo Trio test Two-out-of-five test A not A Difference from control • Directional difference tests are those that have a direction or characteristic associated with the difference between the products. Consumers will not detect the small differences that a trained panel would. but no direction or magnitude of the difference is measured.2 .

two of which are identical. Count the number of correct responses (those that select the odd sample) and compare the result with those presented in Table 2. The panellist is required to identify the different sample. The triangle test is a forced choice test. Instruct each panellist to examine in the specified order (e. 293 Analysis of results COPYRIGHT 594 You must make a choice 54 R L Mason and S M Nottingham 862 .g.Sensory Evaluation the selection and monitoring of panellists. two for each product. The panellist is told that two samples are identical and one is different (odd). Select four 3-digit random number codes. Preparation and Procedure The samples should be representative of the product and all prepared in exactly the same way. Questionnaire Specimen answer form for the triangle test Product Date Time Assessor One of the three samples presented is different from the other two. Please examine in the order requested and place a circle around the code of the sample which is different. carryover effect or adaptation effects. are presented simultaneously to each panellist for testing in a predetermined order. Principle Three samples. left to right) and remind them that they must make a decision. the triangle test has limited application. With products that produce sensory fatigue. Make up sets in multiples of the six arrangements as required for the number of panellists.). If total number of panellists or quantity of products available is insufficient to provide equal numbers of the 6 orders. you still need to make sure there is a balance between sets with 2 ‘A’s and 2 ‘B’s. Prepare scoresheets to provide equal numbers of the following orders: AAB ABA BAA BBA BAB ABB Make up sets of 3 samples to match the score sheets so that half contain 2 samples of product A and half contain 2 samples of product B (Total number of sets should be a multiple of 6. The triangle tests should be presented at random to the panellists.

A duo-trio test can be used when one of the products is an existing standard or reference. A duo-trio test can be applied to determine whether changes in ingredients. The product development section has two different thickening agents available to them. Two batches (A. Table 2 indicates that for 17 assessors at P<0. quality control and as a preliminary test prior to analytical descriptive testing. This is based on the probability that if there is no real difference the odd sample will be chosen a third of the time. and arrange them to provide three of each of the six possible arrangements as indicated above. 10 correct responses are required for significance. Example A company wishes to put a new dessert topping on the market.2. ie the number of panelists who correctly selected the odd sample from the 3 samples presented. The Duo-Trio test is therefore only used when it is COPYRIGHT 55 R L Mason and S M Nottingham . one which is considerably cheaper. The number of correct responses is 10. but no direction or magnitude of the difference is measured.05). It can be concluded that the product from the two thickening agents are significantly different (P<0. packaging or storage have resulted in differences between products.1988) Scope and Application Used to determine whether a difference exists between two samples. One set is discarded and the remaining 17 sets are randomly distributed between the assessors. The duo-trio test finds application in the selection of panellists. product and process development.4 . that the test will reveal a difference when there is none. processing. it will be necessary to prepare 27 samples of A and 27 samples of B. product matching. The test organizer will accept a risk of error of 5% (P<0.05.Sensory Evaluation The total number of correct responses is counted as well as the total number of responses and compared to the statistical tables (Table 2). As each assessor will only make one assessment. B) are prepared using the two different thickening agents and samples are presented to 17 assessors. Statistically the duo-trio test is less powerful than the triangle test because the chance of guessing a correct result is one in two. What should the test organizer do next??? DUO-TRIO TEST (Australian Standard 2542.05). The difference can involve one or several sensory attributes. They wish to know if there is any difference in the products made using the 2 different thickeners.

This is the case when tasting a product with a lingering aftertaste such as bitterness. One sample is identified as the reference sample and panellists are instructed to assess the reference sample first and then identify which of the two samples is the same as the reference. one for each product. sequentially. samples are presented simultaneously or if required. one of the samples is a familiar product or designated standard. left to right). Remind them that they must make a decision. then you will still need to check that there is a balance between sets with 2 ‘A’s and 2 ‘B’s. The sample sets are allocated at random to the panellists. Select two 3-digit random codes. are presented to each panellist. (Total sets should be a multiple of 2). one for each product and prepare the scoresheets so that equal numbers of the two orders are presented. Principle Three samples. spicy or chilli. Constant reference mode The constant reference duo-trio test is useful when you have trained panellists. It is therefore the only one used as a reference sample.g. followed by the two other samples in order (e. It is a forced choice test. (Total number of sets should be a multiple of 4). The number of possible presentation orders is thus restricted to: RA A B RA B A Select two 3-digit random codes. Instruct each panellist to assess the reference sample first. Balanced reference mode This is used when both the samples are unfamiliar and so both the samples are used as the reference sample. If total number of panellists or quantity of products available is insufficient to provide equal numbers of the 4 orders. In this test. Prepare scoresheets to provide equal numbers of the following orders: RA A B RA B A RB B A RB A B Make up sets of 3 samples (reference plus two samples) to match the scoresheets so that half contain 2 samples of product ‘A’ and half contain 2 samples of product `B'. There are two forms of this test: balanced reference mode and constant reference mode. If possible.Sensory Evaluation required to form a judgement. COPYRIGHT 56 R L Mason and S M Nottingham . two of which are identical. Preparation and Procedure The samples prepared should be representative of the product and prepared in exactly the same way.

Remind them that they must make a decision. Taste the samples in the order left to right and circle the number of the sample which is the same as the reference. . one of the other two is the same as the reference. . followed by the two other samples in order (e. You are provided with three samples. . This fact is important when tasting a substance with a lingering aftertaste. . The left-hand one is a reference. Each tray had a control sample marked R and two coded samples. .g. . . one with methional added and one with no methional. YOU MUST MAKE A CHOICE Comments: Example A duo-trio test was used to determine if methional could be detected when added to cheddar cheese in amounts of 0. . Analysis of results Count the number of correct responses as well as the total number of responses and use the statistical Tables 3.250 ppm. . The duo-trio test was used in preference to the triangle test because less tasting is required to form a judgment. Reference Sample code:.125 ppm methional and two control samples and the other contained a sample with 0. . Questionaire DUO-TRIO TEST Name: Date: Time: Product: . Sample code: . . .250 ppm methional and two control samples. such as methional. . Each day the judges were presented with two trays. . One tray contained a sample with 0.125 ppm and 0. The test was performed on two successive days using eight judges. left to right). A total of 16 judgments were made at each level. . .Sensory Evaluation Randomly allocate sample sets to panellists and instruct each panellist to assess the reference sample first. COPYRIGHT 57 R L Mason and S M Nottingham . The results are shown in the following table.

Sensory Evaluation Duo trio test on cheddar cheese containing methional.250 ppm = 14 out of 16 correct judgments Consult Table 3 for 16 judges in a two sample test.125 ppm = 10 out of 16 correct judgments 0.125 X R X R R X R R 5 0.250 ppm.125 and 0. 0.250 R R R R R X R R 7 X = wrong R = right 0. Day1 JUDGES 1 2 3 4 5 6 7 8 TOTAL 0.250 R R R X R R R R 7 Day 2 0.125 0 R R X X R X R R 5 . This chart shows that 12 correct judgments are significant at the 5% level.125 ppm level. What would you do next?? Advantages used where a reference standard is available less tasting required than triangle test can be used with trained or untrained assessors Disadvantages No indication of character or degree of any difference Statistically less powerful than triangle test Applications Quality control — use normal product as control Product matching Product or process improvement Panel selection or training COPYRIGHT 58 R L Mason and S M Nottingham .250 ppm level but not at the 0. The conclusion is that methional added to cheddar cheese can be detected at the 0.

some of which are the reference sample “A” and some “not-A”.2.5 . The analysis of the data is quite complex. the panellist does not have access to the reference “A” while evaluating the test samples. It is an especially useful test where triangle and duo-trio tests cannot be used. Initially. The samples are presented randomly with 3-digit codes and one at a time (an assessment is made and recorded before proceeding to the next sample). sensory fatigue and memory effects may affect the test. As with the triangle and duo-trio tests. 1985. COPYRIGHT 59 R L Mason and S M Nottingham . assign 3-digit random codes to the samples and then make up the scoresheets. “A” – “NOT A” TEST (Australian Standard 2542. Panellists are then presented with a series of samples. Statistical tables exist to determine the significance of the result. There will be 20 possible combinations. Generally. Scope and Application This test is used to determine whether or not a perceptible overall difference exists between one or more samples and a control sample and also to give an indication as to the size of any difference perceived. In quality control situations. taking care to prepare the samples in an identical fashion. The panellist must determine whether the sample is the same (“A”) or different (“not-A”) so it is a forced-choice test. panellists require familiarisation with the reference or “A” sample.1991) Used to determine whether test samples in a series are the same as or different from the reference sample. This may be the case where comparisons are required between products that have a strong or lingering flavour/aftertaste when you will need to control the time between sample presentation or if there are differences in appearance. However. two or up to 10 samples in series (depending on fatigue factors). It is also useful to determine assessor sensitivity to a stimulus. It is statistically very efficient as the probability of guessing correctly the different two samples from the five samples presented is low. Panellists may test one. trained panellists may also be able to rate the degree of difference for individual attributes. Panellists are instructed to assess each product from left to right and select the two samples that are different from the other three.Sensory Evaluation TWO-OUT-OF-FIVE TEST Used to determine whether there is a sensory difference between two samples and to select and monitor panellists. There is no Australian Standard this test however further information can be obtained in Meilgaard. Only one type of “not-A” sample exists per test series. All samples are prepared in an identical way and are representative of the product. DIFFERENCE-FROM-CONTROL TEST (DFC) Also called the degree of difference (DOD) test. Civille and Carr and Aust et al. It can be useful when only a small number of panellists are available.

Each panellist evaluates the identified control sample first. For some applications such as in a quality control. Principle Each panellist is presented with an identified control sample plus one or more test samples.Sensory Evaluation A difference from control test is a useful test to use when other difference tests. Panellists may be trained or untrained but not a mixture of the two. The blind control sample is included as a measure of the placebo effect as it is very rare that the blind control will actually be rated as absolutely identical to the identified control. All panellists should be familiar with the test format. In this situation the panellists must be familiar with the range of differences expected and will require some training with reference samples and the use of the scale. Preparation and Procedure All samples should be representative of the product and all prepared in exactly the same way. how to use the scale and also be aware that some of the samples will be blind controls. In these cases the relative size of the difference is important for deciding whether the product is an accept or reject. Identified control vs blind control While the other half assess the samples as: COPYRIGHT 60 R L Mason and S M Nottingham 2. baked goods and horticultural products it can be difficult to obtain a homogeneous sample which is necessary for a triangle or duo trio test. For example. When used in conjunction with consumer acceptability testing and descriptive testing using a trained panel. Label additional blind control samples as well as the test sample(s) with 3 digit blinding codes. Label an identified control sample for each panellist. For example with products such as meats. Panellists are informed that some of the test samples may be the same as the control sample. Panellists Generally 20-50 people are required. It can be used to check production samples for the degree of difference from a recognised control or standard product. The panellists then rate the degree of difference for each test sample of which some samples will be the blind control. The order of presentation of the test and blind control samples should be balanced. the panellists would require some training. The mean difference from control for each test sample is compared with the mean difference from control obtained from the blind presentation of the control sample. the DFC test is useful for quality control and shelf life testing. Identified control vs test sample . such as triangle or duo-trio are not suitable because of the normal heterogeneity of the products to be tested. The test can also be useful in product development situations to determine which sample is closest to a target product. The panellists are asked to rate the size of difference between each test sample and the identified control sample. half the panellists assess the samples in the order: 1. Where possible the control sample and samples for assessment should be presented simultaneously.

Identified control vs blind control Examples of scales that may be used for the difference from control test: Verbal Category No difference Very slight difference Slight/moderate difference Moderate difference Moderate/large difference Large difference Very large difference Numerical category Scale 0 = No difference 1 2 3 4 5 6 7 8 9 = Very large difference COPYRIGHT 61 R L Mason and S M Nottingham .1. Identified control vs test sample Sensory Evaluation 2.

. very large difference DIFFERENCE FROM CONTROL TEST Name: Date: Time: Product:...... Due to the natural degree of batch to batch variability with the product.........................Sensory Evaluation Line scale no difference Analysis of Results Calculate the mean difference from the identified control for each of the test samples and the blind control samples........ They want to know if this batch of soup is perceived to be different or not from a control batch of soup.. not different very different REMEMBER THAT A DUPLICATE CONTROL IS THE SAMPLE SOME OF THE TIME............ COPYRIGHT 64 R L Mason and S M Nottingham .. use a randomised block analysis of variance using the panellists as blocks.... a triangle test or other forced choice difference would be unsuitable due to the risk of yielding false positives or false negatives... Example A company suspects a flavouring ingredient may have been left out of a batch of chunky vegetable soup.................... If only one test sample has been evaluated use a paired t-test to analyse the results....... If several samples have been evaluated.... Assess the sample marked “control” first..... Assess sample 386 and score the overall sensory difference between the two samples using the scale below..

Panellists are instructed to COPYRIGHT 65 R L Mason and S M Nottingham . However. The wording used on bilateral and unilateral score sheets is different.Sensory Evaluation DIRECTIONAL DIFFERENCE TESTS PAIRED COMPARISON TEST (Australian Standard 2542. one of each product. numbers can be reduced to 7 for a trained panel.g. Equal numbers of AB and BA are randomly allocated to the panellists. Test principle Two coded samples are presented. The paired comparison test can be used for multiple comparisons. The panellists complete the scoresheet questions that have been previously determined by the test objective. then much larger numbers (100+) are needed. which sample is sweeter). Twenty is a reasonable number when the panellists have been screened. but when using completely untrained tasters such as consumers. trained panellists may be selected if appropriate. The sample presented is representative of the product and all samples are prepared identically. Conclusions to be drawn include that there is no evidence of a difference or that the previously declared sample is greater in the attribute intensity or is preferred. It can be applied to determine a directional difference (e. Preparation and Procedure Two 3-digit randomly coded samples. eg substitution of a new low-calorie sweetener. it is necessary to decide whether the results will be treated as a unilateral or bilateral test. The most common paired comparison tests are twosided (bilateral) where there is no prior expectation of the result.1 . in quality assurance as well as in storage tests and in product matching. In this situation it is better to use a rating test. Panellists The test is fairly simple requiring minimal training but the panellists must understand the attribute that is being tested. but this results in a large number of pairs to assess which uses a lot of sample and can cause sensory fatigue. Statistically.1982) Scope and Application Used to determine how a specific sensory property differs between two samples. One sided tests (unilateral) also exist when there is prior expectation of the direction of difference. Conclusions that can be drawn are that there is no evidence of a difference or that one sample has a greater intensity of the chosen attribute or is preferred.2. A paired comparison test can also be used to determine if a more advanced sensory test should be applied. Before the sensory testing commences. are presented. A paired comparison test has numerous applications in product or process development.

UNILATERAL PAIRED COMPARISON TEST Name………………………. Analysis of results Use standard statistical tables for unilateral tests (Table 3) and bilateral tests (Table 4). Count the number of replies identifying a particular sample most frequently.Time………… In front of you are two coded samples of orange juice. The test is a forced choice test and ‘no difference’ responses are not allowed. Please cleanse your palate between samples. Sample code 016 Sample code 983 YOU MUST MAKE A CHOICE Comments………………………………………………………………………. YES Circle the response below.Date…………………………. Please assess them in the order shown below from left to right and indicate if sample 016 is sweeter than sample 983. Questionaires BILATERAL PAIRED COMPARISON TEST Name……………………….Sensory Evaluation assess the samples in a specific order (left to right) and identify which has the higher level of a particular attribute or is preferred. NO YOU MUST MAKE A CHOICE Comments…………………………………………………………………… COPYRIGHT 66 R L Mason and S M Nottingham . Compare this value with the number shown in the statistical table for the number of panellists used.Time………… In front of you are two coded samples of orange juice.Date…………………………. Please assess them in the order shown below from left to right and indicate which sample is sweetest by circling the appropriate code. Please cleanse your palate between samples.

01%). He does not know which of the two samples contains more sugar. Unilateral test Two drinks. Advantages/Disadvantages See paired preference.05%). it can be concluded that drink ‘A’ is significantly sweeter than drink’B’. It is used to place a series of three or more samples in a rank order to determine whether differences exist between samples. The test supervisor accepts a 1% level of significance (ie: P<0. an attribute (bitterness. From Table 3. Question: Is sample ‘A’ sweeter than sample ‘B’? Replies 22 yes and 8 No.2. ‘A’ and ‘B’. Applications Product Development Quality Control Shelf Life Measurement RANKING TEST (Australian Standard 2542. Question: Which sample is sweeter? Replies 18 opt for sample ‘A’ 12 opt for sample ‘B’ From Table 4 it can be concluded that there is no significant difference in the sweetness of the two drinks. hardness) or a preference. The two samples are presented under a random number eg: ‘789’ and ‘379’. The test supervisor accepts a 5% level of significance (ie: P< 0. The data obtained is ordinal and therefore provides directional differences between samples but does not provide information about the degree of difference. The criterion needs to be understood by the panellists. are offered to a panel of 30 assessors. The two samples are presented under a random number. are offered to a panel of 30 assessors.g. crunchiness. The ranking test is a simple way to compare samples and is useful for reducing the number of COPYRIGHT 67 R L Mason and S M Nottingham . Samples are ranked for a specified criterion. eg: ‘789’ and ‘379’.Sensory Evaluation Examples Bilateral test Two drinks ‘A’ and ‘B’. He knows that drink ‘A’ contains more sugar than drink ‘B’.6) Scope and Application The ranking test can be considered an extension of the paired comparison test. e.

A separate scoresheet is used and completed separately if the rank order is required for more than one criterion. As a panellist. They are instructed to arrange the samples in rank order according to the level of the specified criterion. The manufacturer wants to know if cordials are made at the same flavour intensity. As samples are evaluated only in relation to each other. it is often easier to perform this test by arranging the samples in a provisional order first and then to re-evaluate them before assigning final ranks. Analysis of results Rank totals are calculated for each sample and used to generate test statistics which are compared to statistical tables. and are instructed whether to assign rank 1 for the lowest or highest level. a ranking test can be used as a quick method of indicating the effects of different raw materials. It is a forced choice test and tied rankings are not permitted. Example A cordial manufacturer has been provided with two new samples of lemon flavour that are cheaper than the existing flavour. results from one test cannot be compared to those from another unless they both tested the same samples.Sensory Evaluation test samples prior to performing another test and to evaluate panellist ability. Samples are prepared at the same concentration but in order to test this from a sensory perspective the 3 samples are presented to 30 assessors who are asked to rank them in order of flavour intensity. Panellists Minimum of 8 but a larger number of panellists is better. would it be cheaper to use either of the two new flavours. In product development. Preparation and Presentation Three or more 3-digit random coded samples are presented to panellists simultaneously for assessment in a balanced or random (if more than 4 samples) order. processing. The maximum number of samples will depend on the type of product. or packaging and storage treatments. All the samples are prepared and presented identically. The results are presented below: COPYRIGHT 68 R L Mason and S M Nottingham . Test principle Samples are presented to the panellists simultaneously and are placed by the panellists into a rank order relative to one another according to the specified criterion.

2 From Table 7 the critical value for F with 2 degrees of freedom (df = number of samples –1) is 5. The technician must retain the null hypothesis that there is no difference between the flavour strength of the three products. RATING TEST COPYRIGHT 69 R L Mason and S M Nottingham .Sensory Evaluation Assessor 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 Rank Sums A 1 1 2 1 3 1 1 3 3 3 2 3 1 3 3 1 2 1 1 2 3 2 1 3 1 1 3 3 1 2 58 Cordial Samples B 2 3 1 3 2 2 3 2 2 2 1 2 2 1 2 2 3 3 2 1 1 1 3 2 3 3 2 2 2 1 61 C 3 2 3 2 1 3 2 1 1 1 3 1 3 2 1 3 1 2 3 3 2 3 2 1 2 2 1 1 3 3 61 F= 12 (58 2 + 612 + 612 ) − 3 × 30(3 + 1) 30 × 3(3 + 1) = 360.2-360 = 0.99.

more if less trained. panellists respond by marking a position on a horizontal line labelled with “anchors” at each end. A category scale is a series of 7 – 15 boxes labelled to identify levels of intensity.3 1988) (AS2542. For this type of test the basic principles of sensory evaluation should be followed eg coded samples. With a line scale.7 Selection of assessors for descriptive analysis) The rating test can be used to measure the perceived intensity of sensory characteristics eg degree of strawberry flavour in a strawberry milkshake. Panellists should be selected based on their ability to give consistent ratings to the same sample and to discriminate between samples checked by statistical analysis. The response scale used for rating may be in the form of a category scale or a line scale. controlled test environment. An advantage of this type of scale is that panellists responses are not limited to a number of categories on the scale and therefore it may be possible to identify more differences between samples. Selection and training of panellists will be discussed later in a separate section.2.1.6 Selection of assessors for rating methods and 7. The number of panellists used depends on the degree of training but generally a minimum of eight highly trained. COPYRIGHT 70 R L Mason and S M Nottingham .Sensory Evaluation (Australian Standard 2542.3 1995: 7. number of samples tested.

g. Strawberry Flavour None Analysis of results Ratings must be converted to numerical scores for analysis and interpretation. Advantages More than one sensory attribute can be examined. between the response mark and one end of the scale serves as the response score. For statistical analysis. the analysis of variance technique is appropriate (or a t-test in the case of one or two samples). respectively. e. successive integers are assigned to successive categories and these are used in analysis.Sensory Evaluation Example of a category scale. e. Size and direction of differences can be identified. For graphic scales. the distance. the integers 1-9 would be used.g. when obtained for each sample. Correlation or regression analysis may be used for subjective/objective correlations. Strawberry flavour Sample number Extremely strong Very strong Moderate Slight Absent An example of a line scale. with a 9-point scale. in mm. The arithmetic mean and standard deviation. Disadvantages Selecting realistic terminology Agreement and understanding between assessors in descriptive terms Scales are not linear ie: 13 = extremely sweet is not twice as sweet as 7 = moderately sweet Very strong 495 128 COPYRIGHT 71 R L Mason and S M Nottingham . For category scales. serve as measures of central tendency and variability.

42a B 0 2 1 1 1 1 2 0 1 2 1 2 14 1.05) in bitterness between samples C and B. The results are shown in the following table.Sensory Evaluation Applications Product Development Quality Control Storage Trials Research Example The scoring method was used to determine if there was a difference in bitterness in cheddar cheese made using three different milk-coagulating enzymes. Samples of cheese from each treatment were coded and presented to 12 judges for evaluation. ranging from 0 points for ‘not bitter’ to 5 points for ‘extremely bitter’. Judges 1 2 3 4 5 6 7 8 9 10 11 12 Total Mean A 3 2 3 1 3 2 3 2 3 4 1 2 29 2. CBA.05). ACB. BAC. CAB. Any two values not followed by the same letter are significantly different (P<0. BOA.05) more bitter than sample C and B. The ratings assigned by the judges were given numerical values.17b Samples C 1 2 2 0 3 1 2 1 2 3 0 2 19 1. COPYRIGHT 72 R L Mason and S M Nottingham .58b Total 4 6 6 2 7 4 7 3 6 9 2 6 62 The results were submitted to analysis of variance. The order in which the three samples were tasted was balanced so that each possible order was used twice: ABC. There is no significant difference (P>.0. Sample A is significantly (P.

For these data we use a special distribution called the binomial distribution and is useful for tests based on proportions.778 N + N / 3 Now for a range of N values (ie number who sit the test) we can get a range of X values (ie the minimum number who must get the test right). Since we cannot have 0. For example if we have N = 30 panellists we must have at least or 0.pexp z is obtained from tables and for a one tailed risk of 5% is equal to 1. Some examples of how these tests are used in sensory tests is given below. Panellists are asked to pick the odd one out. Purely by luck the panellist has a one in three chance of getting it right.26 of a person so we round up to 15. Three samples are presented where two are the same and one is different. In other situations when we categorise performance into right or wrong answers and count numbers of people who get tests correct or incorrect or those who make one choice over the other we call this discrete categorical data. This forms the basis of the normal approximation to the binomial test. Lets accept that p − pexp z = obs where pq / N pobs is the proportion who answered correctly ie X/N pexp is the proportion of people who we expect by chance ie 1/3 q = 1 . 2). These values have conveniently been calculated and are already tabulated for use (see tables 1.778√30 + 30/3 14. Therefore 15 out of 30 people must get the triangle test right in order to reject the null hypothesis and conclude there is a difference among the samples. as in rating scales then the t-test or other “parametric” statistics such as analysis of variance are applied.65 = and 12 N 33 X = 0. These are discussed in a different section. Triangle Test The triangle test is used when we want to know if there is a detectable difference between two samples or products.Sensory Evaluation STATISTICS FOR SENSORY: DIFFERENCE TESTING What kind of data do we have? When data are graded. COPYRIGHT 73 R L Mason and S M Nottingham .26 correct to achieve significance. By substituting into the equation and solving for X we get X 1 − N 3 1.65.

Sensory Evaluation Duo Trio This is similar to the triangle test except that a standard is presented and two other samples. When there is no preconceived idea of which sample may be sweeter then two-tailed test is appropriate (see tables 3. Panellist 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Sum (column total) Tp RANK A 1 2 1 1 3 2 3 1 3 3 2 2 3 2 2 3 3 2 39 B 3 3 3 2 1 3 2 3 1 1 3 3 2 3 3 2 2 3 43 C 2 1 2 3 2 1 1 2 2 2 1 1 1 1 1 1 1 1 26 COPYRIGHT 74 R L Mason and S M Nottingham . Paired Comparison Test Only two samples are given and panellists are asked to pick which sample is. are also given. 4). Tables for this are also available but as it is less efficient than the triangle test it is not usually preferred over the triangle test (see table 3). What we want to know is. Suppose 18 panellists are asked to rank three orange juices in order of preference. The panellist has to pick which of the two is the same as the standard so has a one in two chance of being correct. one of which is the same as the standard. Are the ranked values for all panellists the same? The results are as follows. Freidman Test This is best demonstrated by example. sweeter than the other. The same tables as for the duo trio test can be used and a one-tailed test is used when you expect one sample to be sweeter (for example) than the other. for example.

Since our calculated F is greater than this we can reject our question and conclude that there are significant differences between the samples. Two samples are different if the difference between their rank sums is greater than or equal to 11. the data we collect is not discreet data so the analysis follows the ‘parametric’ tests that we use on rating scales.T2.T3 .are the rank totals for each sample For our example we get a F of 8.99. 1.76.960 JP ( P + 1) 6 at the 5% level. Pairwise comparisons can be made using the formula below. F= 12 ∑ Tp2 − 3J ( P + 1) JP ( P + 1) where J . So looking up the chi-squared table (table 7) gives a critical value of 5. COPYRIGHT 75 R L Mason and S M Nottingham . These are discussed in the Statistics for Sensory: Descriptive Testing the number of samples (products) T1. Now when the number of panellists is large or the number of samples exceeds 5 then F follows the chi-squared distribution with P-1 degrees of freedom. Difference from Control Test Although the difference from control test is a form of difference testing.number of judges P .Sensory Evaluation The Freidman value F needs to be calculated as follows.8.

interests. This method is therefore a qualitative descriptive test. Aroma. a consensus decision is reached for each sample. Little in the late 1940’s early 1950’s. which is the balance or blending of the flavour. The list is then reviewed and refined and reference standards and definitions applied to each term. Little) This method was developed by Arthur. The panellists examine the products and the results are reported to the panel leader. flavour and texture can all be assessed in this way and the characteristics can be quantified using various techniques and scales as outlined in this section. Appearance. COPYRIGHT 76 R L Mason and S M Nottingham . Panellists are selected by screening for sensory acuity. attitude and availability. is assessed in this way. D. crunchiest etc. With sensory profiling more than two samples can be assessed simultaneously. The main disadvantage with this type of test is that a dominant panel member or the panel leader could easily influence the panels decision. The scales used with this technique involve the use of numbers and symbols and therefore cannot be analysed statistically. It uses a panel of 4-6 trained panellists. flavour and amplitude. Applications: • • • • • Tracking changes in the sensory characteristics of a product over time for shelf-life evaluation Examining the sensory properties of a target product for new product development Examining sensory characteristics of different varieties of a product eg to look at several varieties of apples in order to identify which varieties are sweetest. This type of test has the advantage of not only being able to tell you whether or not there is a difference between samples but also the nature and magnitude of these differences. odour. Sensory diagnostics of ingredient. A vocabulary is developed by exposure to a wide range of products from the product category to be assessed. process or packaging changes Correlations with instrumental methods The Flavour Profile Method® (Arthur D.Sensory Evaluation DESCRIPTIVE TESTING Descriptive testing is used to identify and provide a picture or “profile” of the important sensory characteristics of a product. Through discussion in an open session lead by the panel leader.

Sensory Evaluation Questionnaire for flavour profile of beer Product AROMA Characteristic Hoppy Fruity Sour Yeasty Malty Amplitude(overall aroma) FLAVOUR Characteristic Tingly Sweet Fruity Bitter Malty Yeasty Metallic Astringent Amplitude(overall flavour) Comments Intensity Intensity Name Date COPYRIGHT 77 R L Mason and S M Nottingham .

Other characteristics: relating to the perception of the moisture and fat contents of the food The order in which the characteristics are assessed is also very important. chewiness and adhesiveness 2. this runs the risk of a result being skewed by a dominant personality in the group. Geometrical characteristics: relating to the size. During this process the panel leader only acts to facilitate the discussion and provide references but does not influence or lead the panel. hardness. 1. It was based on the principles of the Flavour Profile method to assess the textural characteristics of a product. The panel leader evaluates the results from these trial sessions and once confident the results are reliable and COPYRIGHT 78 R L Mason and S M Nottingham . Each panellist individually lists as many descriptive words possible that describe differences between the products. The first step is to expose the panellists to a wide range of products from the product category to be assessed. However consensus methods are still employed by some people. Panellists each make their own individual judgement and then depending on the type of scale used. good. grainy. Again. Ten to twelve panellists are selected by screening for ability to discriminate between products. The Texture Profile Method® This method was developed at General Foods in the 1960’s. Trial evaluations are then carried out using the agreed vocabulary and refinements may be made until the panel is happy with the terms used. mechanical. however more recently category and line scales have been used. This standardised vocabulary then needs to be defined with verbal definitions or reference standards and anchor points for the scale agreed upon. the list of descriptive words is narrowed down to remove duplications and redundant terms until a standardised vocabulary is reached. The panel also decides the order in which the terms are to be assessed. Hedonic terms such as nice. fibrous and aerated 3. Initially the technique used an expanded version of the Flavour Profile scale. their ability to verbalise their perceptions and to work as a group. The order of assessment is first bite. “chewing” or masticatory second phase and residual or third phase. etc are not allowed. Mechanical: relating to the reaction of food to stress eg. Quantitative Descriptive Analysis (QDA®) This method of descriptive analysis was developed in the 1970’s. Panellists are selected on their ability to discriminate between textural differences in the product area to be trained. Through a group discussion. shape and orientation of the particles within the food eg. bad. geometrical and ‘other’ characteristics. Textural characteristics are categorised into three groups. Mean scores could then be calculated and the data statistically analysed. Standardised terminology and rating scales are used for the assessments and each scale point is anchored with a specific food.Sensory Evaluation Profile Attribute Analysis® The Flavour Profile method was renamed the Profile Attribute Analysis with the introduction of numerical scales. a consensus decision is reached or statistical analysis is performed on the data. Six to ten panellists are suggested.

9 6.6 9. The assessment and trial sessions are completed in sensory booths following the basic principles of sensory evaluation.6 8.6 4.9 4.64 0. The results are often displayed visually on a spider web or star diagram.50 0.62 0. Data is then analysed using an analysis of variance.8 9.48 0.6 6.60 Probability 0.494 0.2 7.325 0.001 0. Several replicates (3+) are required to validate the data. Results of ANOVA of orange jelly using QDA Attribute Orange colour Orange aroma Firmness Tartness Orange flavour Foreign flavour Sweetness Rate of breakdown Brand A 10.1 SEM 0.9 6.9 6.001 0.1 Brand B 7.Sensory Evaluation repeatable the actual assessment can take place.072 0.464 <0.72 0.66 0.6 6.1 5.6 7.242 COPYRIGHT 79 R L Mason and S M Nottingham . An unstructured 6-inch or 15cm line scale is used to measure the intensities of the agreed characteristics.3 7.011 0.42 0.

Each panellist generates their own list of terms and scales. The data from this type of assessment is then analysed using Generalised Procrustes analysis. flavour. Free Choice Profiling Unlike other descriptive testing techniques this method does not use an agreed vocabulary to assess the samples. For example you might rate the intensity of mint flavour perceived in chewing gum over a 3 minute period.0 14. The main advantage of this technique is the time saved on training a panel. aroma. Scale value 3.0 11. however interpretation of individual attributes can be subjective as the terms are not defined as with other descriptive testing methods. This can be measured using pencil and paper or using one of the sensory software packages with time intensity facilities. although they must use these consistently for all samples.0 Reference Aerosol whipped cream Miracle whip Cheese whiz Peanut butter Cream cheese Sample size Redi whip Kraft Kraft CPC Best Foods Kraft/Philadelphia 1oz 1oz 1oz 1oz 1oz Time Intensity This is used to track the changes in perception of a particular attribute of a product over time.Sensory Evaluation Other Methods Other methods which you may come across in literature but which will not be discussed in detail in this workshop are: Spectrum Method This is a descriptive analysis technique developed by Civille to cover any or all of appearance.0 5. COPYRIGHT 80 R L Mason and S M Nottingham . Data is analysed in a similar way to QDA.0 8. texture or sound characteristics. Example of intensity scale values (0 to 15) for firmness. This method requires extensive training of the panel to use standardised scales anchored with multiple reference points and panellists are trained to use the scale identically. Panellists use a standardised lexicon of terms to evaluate the products.

25 2. usually a probability of 0. We calculate a t value from the formula below and compare it to some tabled values for probabilities less than our accepted risk. d. The paired t test A common question we have in sensory evaluation is when we are comparing two products or samples and we want to know if they are the same or different. We will also introduce some advanced methods for separating data into logical groups using Principal Component Analysis. mean of difference scores. the paired t-test and Analysis of Variance (one way and two way also known as repeated measures analysis of variance). In this section we will look at the most common form of statistical analysis for rating or preference type data.05. t= mean of difference scores standard deviation of difference scores/ N Here is an example taken from O’Mahony. Intensity scores for two products are measured by 10 panellists on a 25 point scale.Sensory Evaluation STATISTICS FOR SENSORY: DESCRIPTIVE TESTING We mentioned earlier about different types of data and how they are analysed using different statistical methods. = 1 the standard deviation of d is computationally = 1 = 1.538 COPYRIGHT 81 R L Mason and S M Nottingham . We can use statistics and in particular the paired t test to determine statistical difference.538 10 The degrees of freedom term so t = d2 4 1 4 16 16 9 0 16 1 1 ∑d 2 = 68 ∑d 2 − (( ∑ d ) 2 / N ) N −1 = 58 / 9 = 2. Panellist 1 2 3 4 5 6 7 8 9 10 N = 10 Score Product A 20 18 19 22 17 20 19 16 21 19 Product B 22 19 17 18 21 23 19 20 22 20 Difference .d 2 1 -2 -4 4 3 0 4 1 1 ∑ d = 10 d =1 Now for some calculations.

The distribution is known as the F distribution and we calculate a F ratio or F test. By then doing a ratio of these variances (ie signal to noise) we can then compare this to tabulated values. Suppose we have 10 panellists rating three samples of mango for mango flavour intensity on a nine point scale.05 is 2. If we simply wish to test whether a mean is different from the population then we use a two-tailed test. A word on twotailed and one-tailed test. A two-way analysis of variance is used when the same judges or panellists rate the same samples (sometimes called repeated measures). greater than or less than then we need a one-tail test.25 is less than this so we do not reject our notion that product A is the same as product B. Most computer packages now do analysis of variance. In general. minus the number of parameters we are estimating. If we have four samples then we could do six paired t-tests and cover all possible pairings of the four treatments.Sensory Evaluation If we had four flavour scores. p=0. We need this value when looking up statistical tables. sometimes described as one-way analysis of variance and two-way analysis of variance. Analysis of variance looks at the amount of variance attributed to the samples or treatments and also estimates the error variance or natural variation. An alternative to this is to use a technique known as analysis of variance to compare several samples at the same time. However. COPYRIGHT 82 R L Mason and S M Nottingham . if it is directional ie. Analysis of Variance If we have only two samples we want to compare then we can use the paired t-test as described earlier and establish a difference if it is there. and knew three of them plus the variance or standard deviation then we could calculate the value of the fourth unknown data point. Our data indicates that the difference scores are not significantly different from 0. Their results are entered into a computer that then completes a two-way analysis of variance and gives the following table. This however becomes very inefficient and unreliable as the number of samples increase. The tabulated t value for df = 9. Lets look at an example. two-tailed. degrees of freedom are equal to the sample size.262 (see table 6). Our value of 1.

panellists tend to agree on the use of the scales. COPYRIGHT 83 R L Mason and S M Nottingham . * for P=0. If you have more than two samples then you can use analysis of variance techniques and pairwise comparisons to determine differences. Forty-two judges are asked to measure the perceived overall sensory difference between two prototypes (samples F and N) and the regular analgesic cream (control). so any two means with a difference greater than this lsd are significantly different. indicating the average score given by any one judge is not that different to another judge’s score.456 3. In this example the lsd (P=0.80 which is greater than the tabulated value of 6. indicating that they are using different parts of the scale.01) is 1.01 with 2 and 18 degrees of freedom (often ** indicates significance at P=0. A category scale is used where 0 is no difference and 10 is extreme difference. If you have a blind control and one test sample then you can perform a paired t-test.01. Formulas for these tests can be found in most statistical textbooks or in some cases the computer package may do the test for you. Now to further test which pairs are significantly different we have a number of options.667 1. Quite often the panellists F ratio is significant. The most common test would be the least significant difference (lsd) test which is based on the ttest.80 ** 1. Difference from Control Test The analysis of the data from this test can take a number of forms but I will outline the most common and simplest to use. An extension to the two-way analysis of variance is the three-way analysis of variance where we add replicates to the AOV table to provide a complete analysis of the experimental data.Sensory Evaluation Source of variation Samples Panellists Error d.84 ns The F ratio for samples is 11.05). An example taken from Meilgaard et al is given below.01 at P=0.987 11.f 2 9 18 Mean square F 23. The lsd tells us what the minimum difference between two means must be for there to  2 be a significant difference. The formula is t α . You will also note that the panellist’s F ratio is not significant (ns). Other pairwise comparison tests are Duncan’s multiple range and Tukeys honestly significant difference (HSD). With highly trained panels.df ems  where ems is the error mean  n square and n is the number of observations per sample and t come from tables of calculated values.32. This means that the variability due to samples is greater than that occurring naturally so there are differences between the mango flavour intensity of the three mango samples.

COPYRIGHT 84 R L Mason and S M Nottingham .8b Product N 5.Sensory Evaluation Difference from control test – Analgesic cream Judge 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Mean Blind control 1 4 1 4 2 1 3 0 6 7 0 1 4 1 4 2 2 4 0 5 2 Product F 4 6 4 8 4 4 3 2 8 7 1 5 5 6 7 2 6 5 3 4 3 d.44 Sample Blind control Mean response 2. Descriptive testing is recommended to determine the nature of these differences.4b Within a row.8 Product N 7 6 6 3 1 5 4 6 7 7 5 5 4 5 6 6 4 7 6 5 4 5.183 105. means not followed by a similar letter are significantly different at the 95% confidence level.05 = 0.4a Sample Blind control Mean response 2.365 1.4a Product F 4. It is concluded that both samples F and N were found to be significantly different from the control.024 6.4 Product F 6 5 6 3 5 5 6 5 4 6 4 5 4 6 3 4 4 8 5 5 4 4.93 ** Sample means with Fishers LSD0.f 41 2 82 Product N 5 6 6 7 3 5 6 4 9 9 2 6 7 5 6 5 7 7 4 5 3 Judge 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 Blind control 3 3 4 0 2 2 2 3 1 4 1 3 1 4 2 3 0 4 0 1 3 2.4 Source of variation Judge Product Error Mean square F 6.04 ** 102.

It is useful for classifying a number of products by grouping them according to the variables that describe them. Fragrant Sweet Oil Cereal Wetwood Roast Vittoria Australian Blend NSW SL 34 Douwe Egbert premium Andronicus Mocha Kenya Harris Gold Label Robert Timms Mocha Kenya QLD SE 8 Melita Colombian Premium Style Robert Timms New Guinea Gold COPYRIGHT 85 R L Mason and S M Nottingham . The aim of the technique is to reduce the number of variables that describe a sample to that of fewer dimensions. Below is an example of the separation of nine coffees using principal components.Sensory Evaluation Advanced analysis using principal components When we have lots of data and have asked our panel a number of questions we need a technique for reducing the data down to a manageable size. If we reduce to two then we can plot the results onto a graph. These two dimensions are called principal components and are a linear combination of the original variables.

In a company situation. detailing the above COPYRIGHT 86 R L Mason and S M Nottingham . sex.1.3 .1995) Recruitment Panel members are usually recruited from staff in laboratories. TRAINING AND MOTIVATION OF A PANEL When developing a sensory panel. Talks. QA/QC) Organisation and management support and commitment (time and money) Resources required o Sensory staff o Interest and availability of potential panellists o Samples and references for screening and training o Availability of a panel room and booths o Facilities for data collection and statistical analysis Establishment of a trained sensory panel can be divided into 2 steps: Selection Training Selection for Descriptive Testing (Australian Standard 2542. nationality. Pre-screening questionnaire Potential panellists need to complete a pre-screening questionnaire to obtain background information on their: interest in participating in the screening and training program as well as ongoing work availability general good health (note any illnesses or allergies and permanent impairment to the senses) any food idiosyncrasies (strong food dislikes or reactions to foods) other information that might be relevant (age.Sensory Evaluation SELECTION. noticeboards or personal invitations may be used to recruit potential panellists. Information should be provided to the prospective panellists concerning the application of sensory evaluation. offices and the plant of a company. cultural and religious background. previous sensory experience. External panellists may also be recruited from the community nearby if the sensory panel work is going to be very time consuming. Some companies test their products at a different company facility. smoking habits) Panellists should not be asked to assess a food that they dislike. distribute questionnaires for employees to fill in. there are several areas that need to be addressed that include: The need for a panel in the organisation (R&D. circulars. what will be involved for the panellists and the envisaged work program.

Do not talk or distract other panellists while testing. Someone who distracts other panellists by talking or making comments.Sensory Evaluation criteria. a preliminary session is a good idea to set the rules that may need to be enforced politely but firmly. An interview is also used to confirm interest and availability. it is important to make note of both attendance and personalities of panellists. Do not discuss samples with other panellists until after they have evaluated the samples. Record all the information you receive in some form of database. Interview Individual interviews are required to determine whether prospective panellists will work well in a group situation as well as for the analytical approach required in descriptive testing. Read any instructions on the scoresheet before starting to evaluate samples. is a liability. For a descriptive sensory panel. Sensory screening tests also give the prospective panellists an indication of the methods used in sensory analysis. Make sure you evaluate the samples in the required order. However. not an asset. drinking. It is best to complete a thorough screening process rather than training unsuitable subjects. there is a large investment involved in terms of both time and money. decide which prospective panellists are to proceed in the screening process. A panellist who is repeatedly late or unavailable can be more trouble than they are worth. Based on the above criteria. Sensory screening tests Screening is completed to obtain information on prospective panellists who need to be able to: Detect differences in attributes present and their intensities Describe the attributes using verbal descriptors and scaling methods for the different intensities Be able to recall and apply attribute references when required Prior to the first screening test. If you make all the questions optional you will find that the majority of people respond truthfully. Ignore your personal likes and dislikes. Instructions for panellists Avoid eating. smoking or chewing gum for 30 minutes before testing. Samples of COPYRIGHT 87 R L Mason and S M Nottingham . The screening tests used should be chosen with the envisaged sensory program in mind. Have confidence in your own judgement. Basic tastes and odours are commonly used for screening tests as well as materials that illustrate the attributes that may be included in the sensory program. During the selection process. despite repeated requests to remain silent while testing. Pre-screening questionnaires can also be used to select individuals who can describe sensory concepts. Don't forget to fill in your name and the date. it is recognised that the best panellists available may need to be used although they may not necessarily meet all the requirements.

The potential panellists may be screened for their ability to rank or rate products for selected attributes using the same technique as the final panel will use. potential panellists should respond correctly 100% of the time.. The products used should be related to those that will be used in the envisaged sensory testing. 1 point for a wrong association and 0 points for no response. In order to evaluate the ability of the panellists to describe sensory responses.g... .. a series of products can be presented and potential panellists asked to describe the sensory impression. Mushroom. Rancid butter. 4 points for absolutely correct. Panellists are chosen if a satisfactory level is attained which will depend on the intensities of the samples used... acid drops.. Floral... Peppermint. . Vanilla. Panellists are given these samples to assess one at a time and asked to describe the odour using his/her own words... Matching tests may be used to evaluate the ability of a prospective panellist to distinguish between different sensory stimuli. .. Fruit. .. .. .. Aniseed. Musty/mouldy... .Sensory Evaluation the actual food products may also be used.. Vinegar. .. ... a range of odours may be presented: Chemical name Benzaldehyde Octene-3-ol Phenyl-2 ethyl acetate Diallyl sulfide Camphor Menthol Eugenol Anethol Vanillin Geosmin Beta-ionone Butyric acid Acetic acid Isoamyl acetate Dimethylthiophene Name most commonly associated with the odour Bitter almonds. . .. A system of marking can be devised e. Similar techniques can be applied for taste and texture. Camphor. raspberries. All potential panellists are presented with the samples in the same order. A satisfactory level for selection of panellists needs to be specified in relation to the materials used. Clove. COPYRIGHT 88 R L Mason and S M Nottingham . A series of triangle or duo-trio tests may be completed to assess the ability of the potential panellists to detect small differences between stimuli at supra-threshold levels.. .. . Also check that they have used most of the scale.. Violets. Preferably. .. . 2 points for a vague association..... For example. Garlic. Grilled onions. 3 points for correct in general terms.

If panellists are motivated and interested they will perform well. the panellists each list the attributes used to describe each sample. Panellists must be taught the correct procedures for evaluating samples and ways to reduce or eliminate sensory adaptation. The panellists are presented with coded samples in triplicate and asked to rate them using the scoresheets and attribute scales they have trained with. Generally the panel leader decides on the type of scale used. Like any instrument. The references can be used to help the panellists to identify and remember a sensory attribute found in the sample. Reliability is checked by completing test replications and the descriptive data obtained is analysed statistically using an analysis of variance. A trained panel usually consists of 10-20 panellists. it is important that the panellists develop confidence as well as the skills for product assessment. Once the panellists have become familiar with the samples. Motivation of panellists is one of the most important factors in maintaining an efficient trained sensory panel. Towards the end of training. a scoresheet is created by the panellists. the panel leader will be able to determine if further training is required or if the evaluation phase can begin. references and definitions. For panellists. They should be made to feel that attendance at sensory evaluation sessions is COPYRIGHT 89 R L Mason and S M Nottingham . The panellists then assess the samples alongside the references until a consensus is reached regarding the sensory attributes. When appropriate on completion of a project. They must also learn to disregard their personal preferences. reference standards and definitions. On completion of this task. feedback should be given to the panel as to the project objectives and outcomes and the contribution of the sensory results. By statistically analysing the data. The initial stage of training involves vocabulary development. the number of attributes as well as the validity and reliability required. It is then the role of the panel leader to provide reference standards for the attributes that have been previously selected by general panel consensus.Sensory Evaluation Training In this phase. ingredients or products. Between 40 and 120h of training are required for a descriptive sensory panel which will depend on the product. panel evaluation sessions are completed that should be similar to the final testing situation. it is very important that the panel leader does not lead or judge the descriptive words generated by the panellist although they can ask for clarification. Individual panellist feedback is also important. The references may be chemicals. This process should continue until the panellists are all happy and understand the terms used. a sense of completing meaningful work is an important source of motivation. At this time. The entire range of products is presented to the panellists. The panellists themselves will usually start to move towards a general consensus once the total attribute list has been generated. The panellists decide on the order in which the attributes are to be assessed. They are instructed to individually assess the sensory differences between the samples and record any differences as descriptive words. the performance of individual panellists as well as the panel as a whole needs to be monitored to check they are producing reliable results. although the panel decides on the verbal anchors to be used.

as in butter and cheese grading. coffee. Make sure you look after them. Throughout training as well as during ongoing sensory evaluation sessions. your trained panel will be one of the most valuable resources in the company. These panels usually include only 2 or 3 highly trained tasters. if the job is done correctly right from the start. Ongoing records of panellists' training and experience are invaluable. They are usually responsible for arranging the tasting conditions and samples themselves. Commodities that utilise expert tasters include the tea. in addition to actually tasting and making a final report. These tasters are particularly sensitive to the nuances of a specific product. wine and dairy industries. it is important to keep the channels of communications open through panel discussion at the completion of a training session or a sensory testing session. COPYRIGHT 90 R L Mason and S M Nottingham . In some instances training can occupy more time than the actual experimental testing sessions. This can be reinforced by running sessions strictly and efficiently to keep their time input to a minimum. However. especially when you first start.Sensory Evaluation important. It takes a great deal of practice to develop the skill and requires continued tasting to stay "tuned". This type of panel is most frequently used to assign a quality grade to a finished product. They also have the ability to carry the characteristics of standard samples in their sensory memory. In the wine and coffee industries one expert may use these skills to blend individual components to produce a final product with the desired characteristics. An aside: Expert panels Panellists who have a great deal of experience in assessing a particular product are often referred to as "Expert tasters".

Recommendations may also need to be included depending on the nature of the work. Remember that it is much easier to write the report if you keep a record as you go along! COPYRIGHT 91 R L Mason and S M Nottingham . The Australian standards for each test type details what should be included in the report.Sensory Evaluation REPORTING As with any other scientific experiment your sensory testing needs to be reported in a clear and concise manner. The results obtained should be interpreted and conclusions drawn using all the information gathered in the experiment.

1980. “Statistical Methods in Food and Consumer Research. 1984. California. Aust. Journal of Food Science. 1985 Bartoshuk. M A. Gacula M C. Pangborn. 1992. USA. London. UK. New York: Academic Press. Lyon. “A Psychology of Food”. Makr. McBride. 1993. “Flavor Measurement”. 1990. Lawless. “Design and analysis of Sensory Optimization”. 1993. D H. New York. Inc. “Sensory Evaluation of Food: Horwood. H. “Sensory Evaluation of Food: Principles and Practices”. Food & Nutrition Press. “Manual on Sensory Testing Methods”. Test. Hasdell. R W. C H. 57-58. J. G. Chi-Tang Ho. Gacula. Marcel Dekker.. 1965. “Separate worlds of taste” Psychology Today 14 (9): 48-57. Australia. 1993. L M. H T & Heymann. Inc. Pennsylvania. Soc. H T. New York: Academic Press. New York. Academic Press Inc. 1998. R L. “Degree of Difference Test Method in Sensory Evaluation of Heterogeneous Product Types.. “Food Texture and Viscosity: Concept and Measurement”.. USA. ASTM. Lyman. “Pepper potency and the forgotten flavour sense” Food Technology 43 (11): 52. Francombe.. Van Nostrand Reinhold Co. Gacula. 1989. K. M A. AMS. Amerine.S A and Washam. “Guidelines for Cookery and Sensory Evaluation of Meat”. Theory and Practice”. Chichester: Ellis Lawless. Bartoshuk. T A and Lawson. L. Sun Books. Philadelphia. COPYRIGHT 92 R L Mason and S M Nottingham . E B. “Principles of Sensory Evaluation of Food”. Beard. M C. “The Bliss Point Factor”. 1989. M C and Singh. 1982. Chapman & Hall. STP 434. Am. 50: 511 – 513. 1978. Chapmann and Hall. “The biological basis of food perception and acceptance” Food Quality & Preference 4: 21-32. 1985. B. R M and Roessler.Sensory Evaluation SELECTED BIBLIOGRAPHY American Meat Science Association. L B. M C. Bourne. Jellinek. Manley. (editors) “Guidelines for Sensory Analysis in Food Product Development and Quality Control”. 1968.

“Sensory Evaluation Methods with Statistical Analysis (for Research Product Development and Quality Control)”. 1988. G and Larmond. London: Elsevier Applied Science. M. B T. M J (eds) “Encyclopedia of Food Science. “Food Acceptability”. Food Technology 44 (12): 78-84. H and Sidel. Moskowitz. London: Elsevier Applied Science. UK. J R. Meilgaard. “Applied Sensory Analysis of Food”. Robinson. Moskowitz. D A. Academic Press. K P and Hudson. M & Ishii. 1990. Piggott. I “Do you have an umami tooth?” Nutrtion Today May/June. G V and Carr. J L. Volumes 1 and 2. Butter. 1999. O’Mahoney. (3rd Edition) Miflora Minoza-Gatchalian. R K & Sadler. 1988. “Laboratory Methods for Sensory Analysis of Food”. “Statistical Procedures in Food Research”. “Sensory Evaluation of Food: Statistical Methods and Procedures”. “Sensory Evaluation: Method for Establishing and Training a Descriptive Flavour Panel. Piggott. 1985. Mackie. New York: Marcel Dekker. 1993. D M H. 1992. Blackie Academic & Poste. “Sensory Analysis of Food”. 2nd edition. Stone. COPYRIGHT 93 R L Mason and S M Nottingham . H R. O’Mahony. UK. M. 1988 (2nd edition now available). 1986. 1994. Civille. A “Understanding Natural Flavors”.Sensory Evaluation McBride. Volume 6. J R. R L. R. 1990. “Sensory Evaluation Practices”. Agriculture Canada Publication 1864/E. 1991. London. McRae. Inc. Elsevier Applied Science. Elsevier Applied Science. Professional. Paterson. J R. Inc. Food & Nutrition Press. Thomson. H. L M. E. “Psychological Basis of Sensory Evaluation”. 1981. Rutledge. (editor). CRC Press. 1986. London. Fla: CRC Press. London. New York: Academic Press. Piggott. J M. USA. Food Technology and Nutrition”. 1985. “Sensory Evaluation Techniques: Boca Raton. “New Directions for Product Testing and Sensory Analysis of Foods”. Florida.

M C. “Food Quality and Preference”. H J. Food & Nutrition Press. MacFie. “Journal of Sensory Studies”..Sensory Evaluation JOURNALS Gacula. H L. Elsevier Applied Scien COPYRIGHT 94 R L Mason and S M Nottingham .. Inc. Meiselman.

STATISTICAL TABLES Table 1: Probability of X or More Correct Judgments in n Trials (one-tailed. p = 1/3)a n\x 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 47 38 39 40 41 42 43 44 45 46 47 48 49 50 0 1 868 912 941 961 974 983 988 992 995 997 998 998 999 999 2 539 649 737 805 857 896 925 946 961 973 981 986 990 993 995 997 998 998 999 999 999 3 210 320 429 532 623 701 766 819 861 895 921 941 956 967 976 982 987 991 993 995 996 997 998 999 999 999 4 045 100 173 259 350 441 527 607 678 739 791 834 890 898 921 940 954 965 974 980 985 989 992 994 996 997 998 998 999 999 999 5 004 018 045 088 145 213 289 368 448 524 596 661 719 769 812 848 879 904 924 941 954 964 972 979 984 988 991 993 995 996 997 998 998 999 999 999 6 001 007 020 042 077 122 178 241 310 382 453 522 588 648 703 751 794 831 862 888 910 928 943 955 965 972 978 983 987 990 992 994 996 997 997 998 999 999 999 999 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 003 008 020 039 066 104 149 203 263 326 391 457 521 581 638 690 737 778 815 847 874 897 916 932 946 957 965 973 978 963 987 990 992 994 995 996 997 998 998 999 999 999 999 001 003 009 019 035 058 088 126 172 223 279 339 399 460 519 576 630 679 725 765 801 833 861 885 905 922 937 949 959 967 973 979 983 987 990 992 994 995 996 997 998 998 001 004 009 017 031 050 075 108 146 191 240 293 349 406 462 518 572 623 670 714 754 789 821 849 873 895 913 928 941 952 961 968 974 980 984 987 990 992 994 995 001 002 004 008 016 027 043 065 092 125 163 206 254 304 357 411 464 517 568 617 662 705 744 779 810 838 863 885 903 920 933 945 955 963 970 976 980 984 987 001 002 004 008 014 024 038 056 079 107 140 178 220 266 314 364 415 466 516 565 612 656 697 735 769 800 829 854 876 895 912 926 938 949 958 965 972 001 002 004 007 013 021 033 048 068 092 121 154 191 232 276 322 370 419 468 516 562 607 650 689 726 761 791 820 845 867 887 904 919 932 943 001 002 004 007 012 019 028 042 058 079 104 133 166 203 243 285 330 376 422 469 515 560 603 644 683 719 753 783 811 836 859 879 896 001 002 003 006 010 016 025 036 050 068 090 115 144 177 213 252 293 336 381 425 470 515 558 600 639 677 713 745 776 803 829 001 002 003 006 009 014 022 031 043 059 078 100 126 155 187 223 261 301 32 385 428 471 514 556 596 635 672 706 739 001 002 003 055 008 013 019 027 038 051 067 087 109 135 164 196 231 268 307 347 389 430 472 514 554 593 631 001 002 003 005 007 011 016 023 033 044 058 075 095 118 144 173 205 239 275 313 352 392 433 473 513 001 001 002 004 066 010 014 020 028 038 051 066 083 104 127 153 182 213 246 282 318 356 395 001 001 002 004 006 009 012 018 025 033 044 057 073 091 111 135 161 189 220 253 287 001 001 002 003 005 007 011 016 021 029 038 050 063 079 098 119 142 168 196 001 001 002 003 004 007 010 014 019 025 033 043 055 070 086 105 126 001 001 002 003 004 006 008 012 016 022 029 038 048 061 076 001 001 001 002 003 005 007 010 014 019 025 033 042 001 001 002 003 004 006 009 012 017 022 001 001 002 003 004 006 008 011 001 001 002 002 003 005 001 001 001 002 001 COPYRIGHT R L Mason and S M Nottingham 95 .

04 5 5 6 6 7 7 7 8 8 9 9 10 10 11 11 11 12 12 13 13 14 14 14 15 15 16 16 16 17 17 18 18 18 19 19 20 20 20 21 21 22 22 22 23 23 24 27 31 35 39 0. p = 1/3) Probability Levels 0. of trials (n) 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 60 70 80 90 COPYRIGHT 96 R L Mason and S M Nottingham .01 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 14 15 15 15 16 16 17 17 18 18 18 19 19 20 20 21 21 21 22 22 23 23 24 24 24 25 25 26 30 34 38 42 0.02 5 5 6 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 13 14 14 15 15 16 16 16 17 17 18 18 19 19 19 20 20 21 21 21 22 22 23 23 23 24 24 25 29 33 36 40 0.001 No.Table 2: Minimum Numbers of Correct Judgments to Establish Significance at Various Probability Levels for the Triangle tests (one tailed.005 5 6 7 7 8 8 9 9 10 10 11 11 12 12 13 13 14 14 15 15 16 16 17 17 17 18 18 19 19 20 20 20 21 21 22 22 23 23 24 24 24 25 25 26 26 26 31 35 39 43 7 8 8 9 10 10 11 11 12 12 13 13 14 14 15 15 16 16 17 17 18 18 19 19 20 20 21 21 22 22 22 23 23 24 24 25 25 26 26 27 27 27 28 28 33 37 41 45 0.05 4 5 5 6 6 7 7 8 8 9 9 9 10 10 11 11 12 12 12 13 13 14 14 15 15 15 16 16 17 17 17 18 18 19 19 19 20 20 20 21 21 22 22 22 23 23 27 31 35 38 0.03 5 5 6 6 7 7 8 8 9 9 10 10 10 11 11 12 12 13 13 13 14 14 15 15 16 16 16 17 17 18 18 18 19 19 20 20 20 21 21 22 22 22 23 23 24 24 28 32 36 40 0.

03 7 8 8 9 10 10 11 11 12 13 13 14 15 15 16 16 17 18 18 19 19 20 21 21 22 22 23 23 24 25 25 26 26 27 27 28 29 29 30 30 31 31 32 33 38 44 49 55 60 0..05 7 7 8 9 9 10 10 11 12 12 13 13 14 15 15 16 16 17 18 18 19 19 20 20 21 22 22 23 23 24 24 25 26 26 27 27 28 28 29 30 30 31 31 32 37 43 48 54 59 0.B . 1978.E.Table 3: Minimum Numbers of Correct Judgments to Establish Significance at Various Probability Levels for Paired . 43. p=1/2) Probability levels 0.001 No of trials (N) 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 60 70 80 90 100 Source : .005 0.Comparison and Duo-Trio Tests (one-tailed.01 7 8 9 10 10 11 12 12 13 14 14 15 15 16 17 17 18 19 19 20 20 21 22 22 23 24 24 25 25 26 26 27 28 28 29 29 30 31 31 32 32 33 34 34 40 46 51 57 63 8 9 10 11 11 12 13 13 14 15 15 16 17 17 15 19 19 20 20 21 22 22 23 24 24 25 25 26 27 27 28 28 29 30 30 31 31 32 33 33 34 34 35 41 47 52 58 64 10 11 12 13 13 14 15 16 16 17 18 18 19 20 20 21 22 22 23 24 24 25 26 26 27 27 28 29 29 30 30 31 32 32 33 34 34 35 36 36 37 43 49 55 61 66 0.04 7 7 8 9 9 10 11 11 12 12 13 14 14 15 15 16 17 17 15 18 19 20 20 21 21 22 23 23 24 24 25 25 26 27 27 28 28 29 29 30 30 31 32 32 38 43 49 54 60 0. 940-947 COPYRIGHT 97 R L Mason and S M Nottingham . Journal of Food Science.02 7 8 8 9 10 10 11 12 12 13 14 14 15 16 16 17 17 18 19 19 20 20 21 22 22 23 23 24 25 25 26 26 27 27 28 29 29 30 30 31 31 32 33 33 39 45 50 56 61 0.Roessler et al.

Table 4: Minimum Numbers of Agreeing Judgements Necessary to Establish Significance at Various Probability Levels for the Paired – Preference Tests and Difference (two tailed.01 8 9 10 11 11 12 13 13 14 15 15 16 17 17 18 19 19 20 20 21 22 22 23 24 24 25 25 26 27 27 28 28 29 30 30 31 31 32 33 33 34 34 35 41 47 52 58 64 0.04 7 8 8 9 10 10 11 12 12 13 14 14 15 16 16 17 17 18 19 19 20 20 21 22 22 23 23 24 25 25 26 26 27 27 28 29 29 30 30 31 31 32 33 33 39 45 50 56 61 0.02 7 8 9 10 10 11 12 12 13 14 14 15 15 16 17 17 18 19 19 20 20 21 22 22 23 23 24 25 25 26 26 27 28 28 29 29 30 30 31 32 32 33 34 34 40 46 51 57 63 0. Probability Levels No.005 0.05 7 8 8 9 10 10 11 12 12 13 13 14 15 15 16 17 17 18 18 19 20 20 21 21 22 23 23 24 24 25 25 26 27 27 28 28 29 29 30 31 31 32 32 33 39 44 50 55 61 0. p=1/2).03 7 8 9 9 10 11 11 12 13 13 14 15 15 16 16 17 18 18 19 19 20 21 21 22 22 23 24 24 25 25 26 27 27 28 28 29 30 30 31 31 32 32 33 34 39 45 51 56 62 0.001 9 10 11 12 12 13 14 14 15 16 16 17 18 18 19 20 20 21 22 22 23 24 24 25 25 26 27 27 28 29 29 30 30 31 32 32 33 33 34 35 35 36 42 48 53 59 65 11 12 13 14 14 15 16 17 17 18 19 19 20 21 21 22 23 23 24 25 25 26 27 27 28 29 29 30 31 31 32 32 33 34 34 35 36 36 37 37 44 50 56 61 67 COPYRIGHT 98 R L Mason and S M Nottingham . of trials (n) 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 60 70 80 90 100 0.

50 19.37 3.80 2.09 2.22 4.91 2.50 8.67 3.68 3.61 2.14 4.30 19.95 4.67 1.84 2 199.58 3.41 4.70 19.52 8 254.35 4.20 3.40 2.23 3.74 2.02 2.84 2.07 3.12 4.62 1.51 10.42 2.44 3.59 3. COPYRIGHT 99 R L Mason and S M Nottingham .74 4.34 2.59 5.75 4.23 2.96 4.53 4.30 9.00 1.90 2.88 1.15 3.21 2. Agricultural and Medical Research published by Longman Group UK Ltd.84 3.96 1.92 1.26 3.41 4.08 4.12 3.05 3.95 1.70 1.68 4.92 1.40 3.42 3.07 3.79 5.10 2.66 2.84 2.79 2.74 5.61 2.83 1.28 4.37 5 230.15 2.53 2.75 24 249.93 2.69 2.53 5.74 2.07 2.36 3.03 3.98 3.94 5.90 1.42 2.38 4.28 2.46 4.38 3.28 6.29 2.60 4 224.30 2.77 4.63 4.29 3.96 2.00 9.94 6.65 1.45 4.97 3.24 2.24 4.81 2.82 4.16 4.87 2.44 2.74 3.13 7.45 8.01 1.68 2.02 1.64 2.42 2.45 2.81 1.48 3.53 2.28 3.90 19.52 2.29 2.90 19.59 5.99 5.41 3.12 6.11 3.53 2.30 2.11 2.66 2.57 2.06 2.00 3.50 2.59 2.00 2.99 2.52 3.88 3.51 1.93 1.62 2.70 2.07 2.31 2.20 3.26 4.77 2.84 6.32 5.35 2.33 3.15 3.18 2.49 3.06 3.60 2.05 2.10 3.84 4.Table 5a n2\n1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 40 60 120 8 5 % Points for the Distribution of F 1 161.70 2.64 2.56 2. London (previously published by Oliver and Boyd Ltd.82 2.21 6 234.00 3.93 2.33 8.12 2.85 2.39 5.98 2.89 1.79 1.01 2.91 4.00 19.48 3.73 1.69 2.09 3.55 2.01 6.55 6.25 2.95 2.60 19.47 2.41 3.00 1.38 2.95 2.74 2.80 3.30 4.94 12 243.59 3.91 1.49 3.49 2.74 2.60 2.47 3.34 3.40 18.64 5.84 3. Edinburgh) and by permission of the authors and publishers.76 2.08 2.86 3.76 1.92 2.96 1.07 2.27 2.07 2.04 4.96 2.73 3.53 3.78 1.32 2.46 2.98 1.36 2.29 2.71 6.13 2.17 4.68 2.48 2.69 1.63 3.90 2.17 2.42 2.60 4.55 2.28 2.35 4.39 1.71 2.18 2.76 2.57 3.44 3.00 Source : Table 9 is taken from Table V of Fisher and Yates : 1974 Statistical Tables for Biological.36 3.45 2.76 4.92 2.34 2.20 2.67 4.51 2.70 2.37 3.20 19.48 2.60 2.55 3.25 2.40 2.15 2.64 1.63 3.61 5.10 3.78 2.87 3.12 2.71 2.28 3.61 1.05 4.59 2.18 3.49 4.22 3.69 3.45 2.09 8 238.79 2.93 2.01 2.51 2.38 2.85 2.54 2.20 4.23 3.19 2.63 2.92 3.35 3.41 8.43 2.24 3.77 2.84 1.90 2.54 4.34 3.26 5.13 3.34 2.16 2.71 2.85 2.23 2.16 3.54 2.13 2.00 19.71 3.73 2.18 4.37 2.40 2.26 4.11 3.21 4.99 3 215.33 3.71 1.16 9.39 3.37 8.25 1.25 9.32 4.32 3.96 2.57 2.19 4.

68 8.60 13.51 12 6106 99.36 3.07 3. Edinburgh) and by permission of the authors and publishers.03 4.55 8.56 3.63 3.80 8 5981 99.06 4.94 4.17 2.57 5.30 28.92 9.79 3.15 7.85 5.03 2.62 2.76 4.88 7.72 5.66 2.56 7.82 52.18 3.01 6.40 4.99 3.67 5.53 3.96 2.18 24 6234 99.90 3.08 3.56 3.31 7.99 2.82 7.23 3.74 5.72 4.24 15.67 3.33 4.16 3.06 9.66 5.02 3.02 3.25 28.86 8.96 2.11 4.90 2.44 4.45 5.12 3.71 15.64 7.98 11.18 4.42 2.56 7.18 4.43 3.47 7.07 4.45 3.60 3 5403 99.57 2.36 2.00 2.31 4.66 2.02 7.65 9.78 3.99 2.75 2.96 3.32 3.04 4.93 2.16 3.67 8.67 3.66 2.06 4.11 6.12 13.39 5.26 2.17 3.45 7.23 6.02 7.63 6.20 3.97 8.86 4.49 14.94 3.18 5.59 3.50 2.37 5.74 12.36 27.48 3.87 2.26 3.25 11.03 4.18 8.70 3.39 9.40 8.65 2.12 21.23 3.00 13.14 4.46 26.65 4.81 18.86 4.47 5.64 4.36 6.34 4.41 5.46 4.81 3.65 3.78 5.46 6.00 2.60 3.46 9.46 13.51 3.00 30.80 2.71 3.13 3.87 2.02 6.34 3.69 4.45 3.80 10.06 5.29 8.42 5.21 10.89 3.51 4.07 5.59 3.50 26.51 6.42 5.55 6.30 4.75 7.79 4.77 4.04 9.27 10.80 5.47 3.70 6.84 2.32 4.68 7.00 3.52 2.71 4.37 9.01 3.72 7.41 3.93 9.84 6.56 5.57 4.94 7.56 4.89 7.30 3. Agricultural and Medical Research published by Longman Group UK Ltd.01 5.59 6. COPYRIGHT 100 R L Mason and S M Nottingham .25 4.29 3.47 7.88 5.22 5.69 12.61 5.38 1.49 5.95 3.64 2 4999 99.80 1.53 5.49 34.11 4.54 4.10 4.82 3.07 8.32 5.77 7.98 4.75 3.60 4.94 3.17 29.60 1.91 3.21 2.95 5. London (previously published by Oliver and Boyd Ltd.87 3.86 3.50 4.52 10.83 3.13 2.09 5.86 2.51 3.73 4.26 4.12 2.74 4.89 4.72 6.39 5.07 4.28 4.12 1.06 2.26 10.31 6.68 4.46 16.58 2.20 5.76 3.22 4.85 7.28 8.85 6.42 27.65 8.82 4.55 3.33 27.99 6.31 3.93 5.75 2.93 6.62 4.32 5 5764 99.70 2.43 4.78 8.29 3.01 1.42 5.37 3.29 3.49 2.63 3.20 6.06 2.78 4 5625 99.04 3.67 5.31 4.10 8.87 4.49 2.64 5.19 6.91 15.99 5.29 2.73 3.10 6.20 4.17 4.10 2.37 4.56 10.60 7.31 2.58 4.53 8.78 3.02 6 5859 99.67 4.50 4.47 5.00 n2\n1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 40 60 120 8 Source : Table 9 is taken from Table V of Fisher and Yates : 1974 Statistical Tables for Biological.33 9.80 3.34 2.71 3.36 3.55 2.01 4.17 3.Table 5b 1 % Points for the Distribution of F 1 4052 98.17 2.10 4.05 14.92 2.79 8 6366 99.20 16.08 6.47 2.29 5.14 4.82 4.50 3.95 1.

313 1.345 1.717 2.08 1.704 2.467 2.069 1.841 4.323 1.85 3.878 2.05 3.14 4.624 2.708 2.025 Level of significance for two-tailed test 0. Edinburgh) and by permission of the authors and publishers.318 4.499 3.528 2.041 4.289 1.492 2.372 1.681 2.358 2.353 3.228 1.032 3.619 31.11 1.776 1.321 1.734 2.221 4.303 1.316 1.697 2.485 2. London (previously published by Oliver and Boyd Ltd.551 3.169 3.373 3.423 2.328 1.66 2.787 2.048 1.671 2 1.143 2.042 1.405 5.764 2.63 2.25 3.01 63.355 3.39 2.319 1.415 1.056 1.941 8.182 1.201 1.303 1.898 2.721 2.314 12.921 2.699 2. published by Longman Group Ltd.947 2.896 2.064 1.763 2.965 3.98 1.571 1.714 2. Source: Table III of Fisher and Yates. If the observed t is greater than or equal to the tables value.021 1.552 2.598 12.747 3.01 0. Agricultural and Medical Research.587 4.753 2.701 2.646 3.06 1.674 3.33 1.2 0.356 1.746 2.711 2.333 1.833 2.106 3.959 5.325 1.46 3.296 1.602 2.69 3.1 0.771 2.282 1.05 0.005 0.12 1.604 4. Statistical Tables for Biological.756 2.318 1.44 1.541 3.1 0.745 3.315 1.645 1.457 2.132 2.311 1.508 2.767 3.131 1.703 2.337 1.977 2.807 2.576 0.539 2.706 2. reject Ho.397 1.617 2.447 1.718 2. COPYRIGHT 101 R L Mason and S M Nottingham .859 5.861 2.314 1.812 2.965 4.5 2.998 2.922 3.659 3.943 2.725 2.101 1.179 1.045 1.052 1.761 2.845 2.779 2.437 4.078 6.479 2.145 1.819 3.725 3.657 9.925 5.771 2.96 0.61 6.796 2.883 3.706 1.797 2.341 1.886 2.684 2.729 2.462 2.383 1.86 2.073 4.012 2.895 2.055 3.086 1.or two-tailed test at the significance level indicated at the top of each column.093 1.782 2.583 2.533 2.363 1.792 3.476 2.819 2.365 1.31 1.35 1.74 2.658 1.567 2.0005 0.2 df 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 40 60 120 00 a The value listed in the table is the critical value of t for the number of degrees of freedom listed in the left column for a one.707 3.831 2.16 1.75 2.821 2.821 6.326 0.473 2.638 2.707 3.365 3.92 4.306 1.Table 6: Critical Value of ta Level of significance for one-tailed test 0.518 2.02 31.015 3.015 2.001 636.781 4.074 1.262 1.

68 78.57 42.78 9.92 13.89 58.6 5.88 32.26 43.24 11.73 51.64 9.41 26.69 68.14 45.25 7. published by Longman Group Ltd.49 65.62 18.51 88.66 69.77 38.09 16.48 20.27 18.03 16.51 53.14 30.29 9.09 42.75 84.16 22.14 39.41 31.2 36.62 32.71 3.69 29.72 26.81 18.99 4.33 65.Table 7: Critical Values of Chi-Squarea Level of significance for one-tailed test 0.74 40.36 15.2 30.31 22.76 25.31 43.64 6.7 39.99 60.21 24.02 14.41 34.31 25 20.2 75.55 33.88 29.17 29.62 54.67 13.4 78.22 4.59 50. If the observed value of xobs2 is greater than or equal to the tabled value. reject Ho.31 45.26 32.36 18.46 20.62 61.16 63.0005 0.99 18.01 6.05 55.97 74.77 27.59 31.9 48. Statistical Tables for Biological.69 35.65 31.99 70.19 40.41 7.84 3.26 29.06 23.54 26.69 47.38 47.59 46.68 44.02 79.59 22.64 2.99 28.64 12.2 0.56 38.03 37.27 49.38 37.17 29.03 16.025 Level of significance for two-tailed test 0.66 38.68 19.1 0.82 16.56 10.49 53 55.001 10.88 47.28 19.8 36.58 0.10 0.04 28.37 60.04 89.6 42.81 18. Edinburgh) and by permission of the authors and publishers.68 34.98 44.27 51.01 35.3 30.53 36.55 21.99 7.81 55.27 41.25 40.08 0.84 11.47 26.96 48.92 28.91 65.18 62.32 46.86 44.4 79.21 51 45.06 58.67 27.76 51.21 11.8 12.83 13.05 1.14 25.02 36.05 0.58 32 33.34 13.63 17.93 40.02 5.42 46.28 15.24 14.005 0.48 55.98 19.71 73.19 37.82 84.87 28. Source: Table IV of Fisher and Yates.49 57.62 24.32 26.03 13.09 21.25 67.62 24.34 37.59 9.68 21.15 21.3 59.42 30.43 32.62 83.82 45.49 7.05 25.92 41.48 56.69 36.7 73.81 33.42 69.29 40.07 8.51 12.29 41.92 74.67 23.34 70. London (previously published by Oliver and Boyd Ltd.68 15.91 36.34 35.57 38.01 0.64 46.81 22.97 40.61 df 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 32 34 36 38 40 44 48 52 56 60 a The table lists the critical values of chi square for the degrees of freedom shown at the left for tests corresponding to those significance levels heading each column.17 60.44 15.3 21.39 15.28 49.82 5.89 53. COPYRIGHT 102 R L Mason and S M Nottingham .9 27.68 16.17 19.46 23.75 42.64 56.38 0.63 31 32.44 65. Agricultural and Medical Research.49 56.64 42.08 49.97 60.31 14.82 9.52 22.11 34.07 11.12 27.8 35.12 37.46 99.27 94.8 48.47 42.83 64.35 33.22 27.7 62.47 68.46 24.96 50.87 23.91 34.

Sign up to vote on this title
UsefulNot useful