Professional Documents
Culture Documents
Development of A Standard Test To Assess Negative Reinforcement Learning in Horses
Development of A Standard Test To Assess Negative Reinforcement Learning in Horses
a r t i c l e i n f o a b s t r a c t
Article history: Most horses are trained by negative reinforcement. Currently, however, no standardised test for evalu-
Received 14 March 2015 ating horses’ negative reinforcement learning ability is available. The aim of this study was to develop an
Received in revised form 15 May 2015 objective test to investigate negative reinforcement learning in horses. Twenty-four Icelandic horses (3
Accepted 17 May 2015
years old) were included in this study. The horses were tested in a pressure-release task on three separate
Available online 28 May 2015
days with 10, 7 and 5 trials on each side, respectively. Each trial consisted of pressure being applied on the
hindquarter with an algometer. The force of the pressure was increased until the horse moved laterally
Keywords:
away from the point of pressure. There was a significant decrease in required force over trials on the first
Algometry
Horse behaviour
test day (P < 0.001), but not the second and third day. The intercepts on days 2 and 3 differed significantly
Learning performance from day 1 (P < 0.001), but not each other. Significantly stronger force was required on the right side com-
Operant conditioning pared to the left (P < 0.001), but there was no difference between first and second side tested (P = 0.56).
Pressure-release Individual performance was evaluated by median-force and the change in force over trials on the first
Horse training test day. These two measures may explain different characteristics of negative reinforcement learning. In
conclusion, this study presents a novel, standardised test for evaluating negative reinforcement learning
ability in horses.
© 2015 Elsevier B.V. All rights reserved.
http://dx.doi.org/10.1016/j.applanim.2015.05.005
0168-1591/© 2015 Elsevier B.V. All rights reserved.
L.P. Ahrendt et al. / Applied Animal Behaviour Science 169 (2015) 38–42 39
2.1. Animals
2.3. Methods
The handler held the horse in halter and lead rope, while the
experimenter applied the pressure on the horse’s hindquarter. The
handler and experimenter were always the same two persons. Prior
to testing, the horses were habituated to the surroundings and
stood calmly when the experimenter approached and stroked the
hindquarter of the horse. Fig. 1. Point of pressure. The yellow dot indicates the spot on the horse’s hindquarter
The horses were tested on three separate days with four and five where the pressure was applied (For interpretation of the color information in this
days between. Each test day consisted of 10 (Day 1), 7 (Day 2) and figure legend, the reader is referred to the web version of the article.).
5 (Day 3) trials on each side of the horse. The number of trials was
reduced on the second and third day because these young horses trial (Fig. 1). In each trial, the horse was placed so that all four legs
appeared to become impatient and less attentive with increasing were weight bearing. The experimenter then placed the algome-
number of trials within days, i.e. they were increasingly difficult to ter on the marked spot and increased the force according to the
get to stand still between trials, were nibbling the handler more set rate. When the horse responded correctly the algometer was
and pawed more. In each trial pressure was applied until the horse immediately removed to release the pressure and thereby reward
responded correctly by moving at least one hind leg laterally away the horse. When the actuator limit of 30 N was reached without
from the point of pressure, i.e. if pressure was applied on the left correct response, the experimenter used the hand, not operating
side the horse should move to the right. The side tested first was the algometer, to apply the extra force necessary to obtain the cor-
randomly chosen for each individual, but balanced according to rect response. This was done to prevent a connection between not
sex. Sides of an individual horse were tested in the same order on responding and the reward of removing the pressure which would
all three test days. The first side was always finished with all trials have disrupted the learning process. In trials where the actuator
before continuing with the trials on the second side. A mark was limit was exceeded a “max” was recorded for the force. Regard-
drawn on the approximate middle of both sides of the hindquarters less of the side of the horse the experimenter faced the point of
to ensure that the pressure was applied on the same spot in each pressure and used the right hand to operate the algometer. Between
40 L.P. Ahrendt et al. / Applied Animal Behaviour Science 169 (2015) 38–42
each trial, the handler repositioned the horse for the next trial. The Finally, we investigated if the proposed performance values
inter-trial interval was approximately 15–60 s, depending on how ranked the horses similarly. All correlations were calculated by
long it took the handler to get the horse in the right position for the Spearman’s rank correlation rho.
next trial. All analyses were done in R version 3.1.2 (R Core Team, 2014)
with significance evaluated at P = 0.05 and tendencies between
2.4. Statistics 0.05 < P < 0.1.
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 1 2 3 4 5
Fig. 2. Development in force over trials within test days. Dots indicate estimated means for force per trial for the left side on non-trained control horses. The right side
required 1.4 times stronger force (P < 0.001) and the trained horses required 0.6 times lighter force (P = 0.09). The lines show the regression curve for the development over
trials. Only the slope on Day 1 is significantly different from 0 (P < 0.001). The intercepts on Day 2 and Day 3 is significantly different from Day 1 (P < 0.001), but not each other
(P = 0.24).
L.P. Ahrendt et al. / Applied Animal Behaviour Science 169 (2015) 38–42 41
suggests that some transfer of behaviour between exercises on the For future studies using this method, we first recommend to use
same side of the horse may occur. However, the tendency towards an algometer with an actuator calibrated to measure higher force.
trained horses requiring lighter force could also relate to trained Secondly, we recommend using a combination of median-force and
horses having a more well-developed coordination necessary to slope to evaluate individual learning performance, as these meas-
perform the sideways movement compared to the non-trained con- ures may explain different parts of learning this task.
trol horses, thereby making the exercise in the present study easier
to perform. Acknowledgements
4.1. Evaluation of individual performance We would like to thank the horse owners for kindly allowing
us to use their horses; Anna Feldberg Marsbøll and Maria Vilain
Currently, there exists no gold standard to evaluate equine neg- Rørvang for help with the experiments. This study was funded by
ative reinforcement learning ability. The way learning performance The Independent Danish Research Council|Technology and Produc-
is measured in this test, or any learning test, may influence the eval- tion (grant no. 11-107010) and The Graduate School of Science and
uation of the individual horse’s learning ability and which factors Technology, Aarhus University.
influence the performance. The reason for evaluating performance
through different measures is that it potentially explains different References
characteristics of learning. In this study we proposed three poten-
tial individual performance values, based on the initial analysis of Baragli, P., Vitale, V., Paoletti, E., Sighieri, C., Reddon, A.R., 2011. Detour behaviour in
horses (Equus caballus). J. Ethol. 29, 227–234.
the applied force. Median-force and frequency of max-trials (i.e. Chiou, S.H., Kang, S., Kim, J., Yan, J., 2014a. Marginal semiparametric multivariate
where the force exceeded the 30 N limit of the algometer) on the accelerated failure time model with generalized estimating equations. Lifetime
first day correlated significantly, i.e. a horse with a high median Data Anal. 20, 599–618.
Chiou, S.H., Kang, S., Yan, J., 2014b. Fitting accelerated failure time models in routine
also had many trials exceeding 30 N. This was expected but not
survival analysis with R package aftgee. J. Stat. Softw. 61, 1–23.
theoretically necessary, because the median could take any value Christensen, J.W., Ahrendt, L.P., Lintrup, R., Gaillard, C., Palme, R., Malmkvist, J.,
between 0 and 30 N, when the frequency of max-trials was below 2012. Does learning performance in horses relate to fearfulness, baseline stress
5. In contrast to the median and frequency of max-trials, which hormone, and social rank? Appl. Anim. Behav. Sci. 140, 44–52.
Domjan, M., 2010. The Principles of Learning and Behaviour, sixth ed. Wadsworth
may express individual tactile sensitivity, the slope on Day 1 may Cengage Learning TM, USA, pp. 155.
express the ‘speed of learning’. The slopes on the left and right side, Dougherty, D.M., Lewis, P., 1993. Generalization of a tactile stimulus in horses. J. Exp.
however, did not correlate and only the slope on the left side corre- Anal. Behav. 59, 521–528.
Gabor, V., Gerken, M., 2010. Horses use procedural learning rather than con-
lated with the median. This may be explained by the slopes being ceptual learning to solve matching to sample. Appl. Anim. Behav. Sci. 126,
rather unstable. The slopes were calculated from only 10 trials and 119–124.
may, thus, be easily influenced by a few trials where a high force Gibson, E., 1939. Sensory generalization with voluntary reactions. J. Exp. Psychol.
24, 237–253.
was required. This is further supported by the result that the sides Gooijers, J., Swinnen, S.P., 2014. Interactions between brain structure and behavior:
did not correlate on the second and third day either, both of which the corpus callosum and bimanual coordination. Neurosci. Biobehav. R 43, 1–19.
were calculated from yet fewer repetitions. Hanggi, E.B., 1999. Interocular transfer of learning in horses (Equus caballus). J.
Equine Vet. Sci. 19, 518–524.
An often used, easy to compare, performance value is number Jin, Z., Lin, D.Y., Ying, Z., 2006. On least-squares regression with censored data.
of sessions to accomplish a set learning criterion. In our study this Biometrika 93, 147–161.
was not possible for two reasons; firstly we did not know at which Kalbfleisch, J.D., Prentice, R.L., 2011. The Statistical Analysis of Failure Time Data,
13., second ed. Wiley-Blackwell, New Jersey, USA, pp. 195.
force we should set this criterion, since this approach had not been
Kirsch, W., Hoffmann, J., 2010. Asymmetrical intermanual transfer of learning in a
tried before. The other, perhaps more important reason, was that sensorimotor task. Exp. Brain Res. 202, 927–934.
horses may have different threshold for when the pressure became Marsbøll, A.F., Christensen, J.W., 2015. Effects of handling on fear reactions in young
sufficiently motivating for them to offer a response (McGreevy and Icelandic horses. Equine Vet. J., DOI: 10.1111/evj.12338.
McCall, C.A., Salters, M.A., Simpson, S.M., 1993. Relationship between number of
McLean, 2010). In earlier studies using avoidance learning tasks, the conditioning trials per training session and avoidance learning in horses. Appl.
horses’ reaction threshold to electric shock was established before Anim. Behav. Sci. 36, 291–299.
initiating the task, to ensure the power of the shock to be sufficient McGreevy, P.D., Rogers, L.J., 2005. Motor and sensory laterality in thoroughbred
horses. Appl. Anim. Behav. Sci. 92, 337–352.
to get the horses to react (McCall et al., 1993). However, the aim McGreevy, P.D., McLean, A.N., 2010. Equitation Science. Wiley-Blackwell, Oxford,
in these studies was not to investigate if the power of the shock UK. pp. 91, 89, 105.
could be decreased by learning but rather to use it as a motivator Murphy, J., Sutherland, A., Arkins, S., 2005. Idiosyncratic motor laterality in the horse.
Appl. Anim. Behav. Sci. 91, 297–310.
to teach the horses to react to a conditioned stimulus and thereby Nicol, C.J., 2002. Equine learning: progress and suggestions for future research. Appl.
avoid the electric shock. This study is the first step in the devel- Anim. Behav. Sci. 78, 193–208.
opment of a standardised test for negative reinforcement learning Olivares, R., Montiel, J., Aboitiz, F., 2001. Species differences and similarities in the
fine structure of the mammalian corpus callosum. Brain Behav. Evol. 57, 98–105.
in horses. Further studies are required to validate the test and it R Core Team, 2014. R: A Language and Environment for Statistical Computing. R
should be investigated if the asymptotic level could be used as a Foundation for Statistical Computing, Vienna.
learning criterion, which would ease the analysis of individual per- Valenchon, M., Lévy, F., Prunier, A., Moussu, C., Calandreau, L., Lansade, L., 2013.
Stress modulates instrumental learning performances in horses (Equus caballus)
formance. Further studies are also required to relate performance
in interaction with temperament. PloS One 8, 1–10.
in this test to tactile sensitivity. The performance in this test should Visser, E.K., van Reenen, C.G., Schilder, M.B.H., Barneveld, A., Blokhuis, H.J., 2003.
also be related to other tests of equine learning ability and also to Learning performances in young horses using two different learning tests. Appl.
practical training of horses. Anim. Behav. Sci. 80, 311–326.
Wolff, A., Hausberger, M., 1996. Learning and memorisation of two different tasks
In conclusion, the presented test can be used to evaluate neg- in horses: the effects of age, sex and sire. Appl. Anim. Behav. Sci. 46, 137–143.
ative reinforcement learning in horses in a standardised manner.