You are on page 1of 51

Brain Computations and Connectivity

Edmund T. Rolls
Visit to download the full and correct content document:
https://ebookmass.com/product/brain-computations-and-connectivity-edmund-t-rolls/
Brain Computations and Connectivity
Brain Computations and Connectivity

Edmund T. Rolls
Oxford Centre for Computational Neuroscience
Oxford
England
Great Clarendon Street, Oxford, OX2 6DP,
United Kingdom
Oxford University Press is a department of the University of Oxford.
It furthers the University’s objective of excellence in research, scholarship,
and education by publishing worldwide. Oxford is a registered trade mark of
Oxford University Press in the UK and in certain other countries
c Edmund T. Rolls 2023

The moral rights of the author have been asserted
Some rights reserved. No part of this publication may be reproduced, stored in
a retrieval system, or transmitted, in any form or by any means, for commercial purposes,
without the prior permission in writing of Oxford University Press, or as expressly
permitted by law, by licence or under terms agreed with the appropriate
reprographics rights organization.

This is an open access publication, available online and distributed under the terms of a
Creative Commons Attribution – Non Commercial – No Derivatives 4.0
International licence (CC BY-NC-ND 4.0), a copy of which is available at
http://creativecommons.org/licenses/by-nc-nd/4.0/.
Enquiries concerning reproduction outside the scope of this licence
should be sent to the Rights Department, Oxford University Press, at the address above
Published in the United States of America by Oxford University Press
198 Madison Avenue, New York, NY 10016, United States of America
British Library Cataloguing in Publication Data
Data available
Library of Congress Control Number: 2023940370
ISBN 978–0–19–888791–1
DOI: 10.1093/oso/9780198887911.001.0001
Printed and bound by
CPI Group (UK) Ltd, Croydon, CR0 4YY
Links to third party websites are provided by Oxford in good faith and
for information only. Oxford disclaims any responsibility for the materials
contained in any third party website referenced in this work.
Preface

This book, Brain Computations and Connectivity is an update to Rolls Brain Computations:
What and How published by Oxford University Press in 2021. A key part of this updated
version of Brain Computations: What and How is the addition of much new evidence that
has emerged since 2021 on the connectivity of the human brain, which by analysing the
effective or causal connectivity between 360 cortical regions, enriches our understanding
of information processing streams in the human cerebral cortex. Knowing how one cortical
region connects to other cortical regions helps to provide a deeper understanding of how
each stage in a processing stream contributes to the computations being performed, as each
processing stream is now becoming better defined.
The addition in Brain Computations and Connectivity of much new evidence on the con-
nectivity of the human brain is leading to better understanding of what is computed in diff-
erent human brain regions, which in turn has implications for how the computations are per-
formed. The added emphasis in this book on the organisation, connectivity, and functions of
360 cortical regions in the human brain increases the application of new approaches taken to
understanding how the brain works that are described in this book, by making the evidence
very relevant to the human brain, and thereby to those who are interested in disorders of the
human brain, including neurologists and psychiatrists, as well as to all those interested in
understanding how the human brain works in health and disease. The enables the compu-
tational approaches to understanding brain function described in this book to be combined
with and made highly relevant to understanding human brain function. It is hoped that these
developments in understanding human brain function by including analysis of the connec-
tivity of the 360 cortical regions in the Human Connectome Project Multimodal Parcellation
(HCP-MMP) atlas (Glasser et al., 2016a; Huang et al., 2022) as described here will provide a
foundation for future research that by using the HCP-MMP can be related to the connectivity
and functions of these 360 cortical regions in the human brain that are described in this book.
The title of this book, Brain Computations and Connectivity, refers to the importance of this
type of connectivity in understanding what and how the brain computes.
Another feature of this book is that it adopts what was described as a ‘grand unifying
approach’ in a review in the journal Brain (Manohar, 2022) of Brain Computations: What
and How (Rolls, 2021b) in that it takes a biologically plausible approach to how the brain
computes, rather than invoking the artificial intelligence approach of deep learning to try
to account for brain computation. The approach to how the brain computes in this book is
biologically plausible in that it utilises primarily local synaptic modification rules to set up
the synaptic connections between neurons, and unsupervised learning, yet considers how far
this can take us in understanding how the brain operates. This approach is quite different to the
deep learning approach used now in much machine learning and AI, which backpropagates an
error backwards through the network to correct every synaptic weight in all preceding layers
of the network, but there is no connectivity in the brain for that. Moreover, the learning in the
brain is largely unsupervised, whereas deep learning approaches typically require a supervisor
for each output neuron, which in turn requires what must be computed for each output neuron
to be specified by the designer of the deep network. The point is developed (in Chapter 19)
that machine learning might benefit from understanding some of the unsupervised approaches
used by the brain to solve complex problems, including navigation, attention, semantics, and
vi | Preface

even syntax. The title of this book, Brain Computations and Connectivity, also refers to this
concept that the actual connectivity at the local level of neurons and synapses that is found in
the brain is important in understanding what and how it computes.
My aim in making this book Open Access is to make the scholarship and ideas in this
book easily available to colleagues, in the interests of better understanding the human brain
in health and disease. This book can be cited as Rolls, E. T. (2023) Brain Computations
and Connectivity, Oxford University Press, Oxford. The book can be downloaded from
https://www.oxcns.org or from the Oxford University Press website. An advantage of pub-
lication of this book as a .pdf is that it contains extensive hyperlinks, enabling readers to
quickly cross-refer to other parts of the book or to citations. With the .pdf, figures can also be
expanded to show details.
Many scientists, and many others, are interested in how the brain works. In order to un-
derstand this, we need to know what computations are performed by different brain systems;
and how they are computed by each of these systems.
The aim of this book is to elucidate what is computed in different brain systems; and to
describe current computational approaches and models of how each of these brain systems
computes.
To understand how our brains work, it is essential to know what is computed in each
part of the brain. That can be addressed by utilising evidence relevant to computation from
many areas of neuroscience. Knowledge of the connections between different brain regions
is important, for this shows that the brain is organised as systems, with whole series of brain
regions devoted for example to visual processing. That provides a foundation for examin-
ing the computation performed by each brain region, by comparing what is represented in a
brain regions with what is represented in the preceding and following brain regions, using
techniques of for example neurophysiology and functional neuroimaging. The ways in which
effective connectivity is now available for 360 cortical regions are introduced in Chapter 1,
with the results of these new analyses found throughout this book. Neurophysiology at the
single neuron level is needed because this is the level at which information is transmitted
between the computing elements of the brain, the neurons. Evidence from the effects of brain
damage, including that available from neuropsychology, is needed to help understand what
different parts of the system do, and indeed what each part is necessary for. Functional neuro-
imaging is useful to indicate where in the human brain different processes take place, and to
show which functions can be dissociated from each other. So for each brain system, evidence
on what is computed at each stage, and what the system as a whole computes, is essential.
To understand how our brains work, it is also essential to know how each part of the
brain computes. That requires a knowledge of what is computed by each part of the brain,
but it also requires knowledge of the network properties of each brain region. This involves
knowledge of the connectivity between the neurons in each part of the brain, and knowledge
of the synaptic and biophysical properties of the neurons. It also requires knowledge of the
theory of what can be computed by networks with defined connectivity.
There are at least three key goals of the approaches described here. One is to understand
ourselves better, and how we work and think. A second is to be better able to treat the system
when it has problems, for example in mental illnesses. Medical applications are a very impor-
tant aim of the type of research described here. A third, is to be able to emulate the operation
of parts of our brains, which some in the field of artificial intelligence (AI) would like to do to
produce useful computers that incorporate some of the principles of computation utilised by
the brain. All of these goals require, and cannot get off the ground, without a firm foundation
in what is computed by brain systems, and theories and models of how it is computed. To
understand the operation of the whole brain, it is necessary to show how the different brain
Preface | vii

systems operate together: but a necessary foundation for this is to know what is computed in
each brain system, and the connectivity between the different regions involved.
Part of the enterprise here is to stimulate new theories and models of how parts of the brain
work. The evidence on what is computed in different brain systems had advanced rapidly in
the last 50 years, and provides a reasonable foundation for the enterprise, though there is
much that remains to be learned. Theories of how the computation is performed are less
advanced, but progress is being made, and current models are described in this book for
many brain systems, in the expectation that before further advances are made, knowledge of
the considerable current evidence on how the brain computes provides a useful starting point,
especially as current theories do take into account the limitations that are likely to be imposed
by the neural architectures present in our brains. In this book, the focus is on biologically
plausible algorithms for brain computation, rather than for example deep learning which does
not appear to be how the brain learns and computes (Sections B.12 and B.14).
The simplest way to delineate brain computation is to examine what information is
represented at each stage of processing, and how this is different from stage to stage. For
example in the primary visual cortex (V1), neurons respond to simple stimuli such as bars
or edges or gratings and have small receptive fields. Little can be read off from the firing
rates about for example whose face is represented from a small number of neurons in V1.
On the other hand, after four or five stages of processing, in the inferior temporal visual
cortex, information can be read from the firing rates of neurons about whose face is being
viewed, and indeed there is remarkable invariance with respect to the position, size, contrast
and even in some cases view of the face. That is a major computation, and indicates what can
be achieved by biological neuronal network computation.
These approaches can only be taken to understand brain function because there is consid-
erable localization of function in the brain, quite unlike a digital computer. One fundamental
reason for localization of function in the brain is that this minimizes the total length of the
connections between neurons, and thus brain size. Another is that it simplifies the genetic
information that has to be provided in order to build the brain, because the connectivity in-
structions can refer considerably to local connections. These points are developed in my book
Cerebral Cortex: Principles of Operation (Rolls, 2016b).
That brings me to what is different about the present book and Cerebral Cortex: Prin-
ciples of Operation (Rolls, 2016b). Cerebral Cortex: Principles of Operation took on the
enormous task of making progress with understanding how the major part of our brains, the
cerebral cortex, works, by understanding its principles of operation. The present book builds
on that approach, and uses it as background, but has the different aim of taking each of our
brain systems, and describing what they compute, and then what is known about how each
system computes. The issue of how they compute relies for many brain systems on how the
cortex operates, so Cerebral Cortex: Principles of Operation provides an important comple-
ment to the present book.
With its focus on what and how each brain system computes, a field that includes com-
putational neuroscience, this book is distinct from the many excellent books on neuroscience
that describe much evidence about brain structure and function, but do not aim to provide an
understanding of how the brain works at the computational level. This book aims to forge an
understanding of how some key brain systems may operate at the computational level, so that
we can understand how the brain actually performs some of its complex and necessarily com-
putational functions in memory, perception, attention, decision-making, cognitive functions,
and actions.
Indeed, as one of the key aims of this book is to describe what computations are per-
formed by different brain systems, I have chosen to include in this book some of the key actual
discoveries in neuroscience that I believe help to define what computations are performed in
viii | Preface

different brain systems. Part of the aim of citing the original pioneering discoveries is to show
how conceptual advances have been made in our understanding of what is computed in each
region of our brains, for understanding the key discoveries provides a foundation for further
advances.
That makes this book very different from many of the textbooks of neuroscience (such
as Principles of Neural Science (Kandel et al., 2021)), and some of the textbooks of theoret-
ical neuroscience that describe principles of operation of neurons or of networks of neurons
(Dayan and Abbott, 2001; Hertz et al., 1991; Gerstner et al., 2014), but not in general what
is computed in different brain systems, and how it is computed. Further, there are likely to be
great developments in our understanding of how the brain computes, and this book is intended
to set out a framework for new developments, by providing an analysis of what is computed
by different brain systems, and providing some current approaches to how these computations
may be performed. I believe that this book, and its predecessor Brain Computations: What
and How, is pioneering in what I propose is the necessary approach to understanding brain
function, understanding ‘What’ is computed by each brain region, and ‘How’ it is computed.
A test of whether one’s understanding is correct is to simulate the processing on a com-
puter, and to show whether the simulation can perform the tasks performed by the brain, and
whether the simulation has similar properties to the real brain. This approach to neural com-
putation leads to a precise definition of how the computation is performed, and to precise
and quantitative tests of the theories produced. How memory systems in the brain work is a
paradigm example of this approach, because memory-like operations which involve altered
functionality as a result of synaptic modification are at the heart of how many computations in
the brain are performed. It happens that attention and decision-making can be understood in
terms of interactions between and fundamental operations in memory systems in the cortex,
and therefore it is natural to address these areas of cognitive neuroscience in this book. The
same fundamental concepts based on the operation of neuronal circuitry can be applied to all
these functions, as is shown in this book.
One of the distinctive properties of this book is that it links the neural computation ap-
proach not only firmly to neuronal neurophysiology, which provides much of the primary data
about how the brain operates, but also to psychophysical studies (for example of attention);
to neuropsychological studies of patients with brain damage; and to functional magnetic res-
onance imaging (fMRI) (and other neuroimaging) approaches. The empirical evidence that
is brought to bear is largely from non-human primates and from humans, because of the con-
siderable similarity of their cortical systems, and the major differences in their systems-level
computational organization from that of rodents, as set out in Section 19.10.
Another feature of the book is that it aims to describe the principles of operation of the
brain, and in doing that, frequently refers to the original research in which those principles
were discovered. Another feature of the book is that I have also chosen to make each Chapter
relatively understandable on its own, which may involve some information also present in
other parts of the book, so that readers with an interest in one part of the brain can benefit by
reading the relevant chapters, rather than necessarily having to read the whole book.
The overall aims of the book are developed further, and the plan of the book is described,
in Chapter 1. Appendix B describes the fundamental operation of key networks of the type
that are likely to be the building blocks of brain function. Appendix C describes quantitative,
information theoretic, approaches to how information is represented in the brain, which is an
essential framework for understanding what is computed in a brain system, and how it is com-
puted. Appendix D describes Matlab software that has been made available with this book
to provide simple demonstrations of the operation of some key neuronal networks related to
cortical function; to show how the information represented by neurons can be measured; and
to provide a tutorial version of the VisNet program for invariant visual object recognition
Preface | ix

described in Chapter 2. The neural networks programs are also provided in Python. The pro-
grams are available at https://www.oxcns.org.
Part of the material described in the book reflects work performed in collaboration with
many colleagues, whose tremendous contributions are warmly appreciated. The contributions
of many will be evident from the references cited in the text. Especial appreciation is due
to Alessandro Treves, Gustavo Deco, Sylvia Wirth, and Simon M. Stringer, who have con-
tributed greatly in always interesting and fruitful research collaborations on computational
and related aspects of brain function, and to many neurophysiology and functional neuro-
imaging colleagues who have contributed to the empirical discoveries that provide the foun-
dation to which the computational neuroscience must always be closely linked, and whose
names are cited throughout the text. Charl Ning (University of Warwick) is thanked for help
with translating the Matlab neural network programs described in Appendix D into Python.
Dr Patrick Mills and Imogen Kruse are thanked for very helpful comments on an earlier ver-
sion of this book. Much of the work described would not have been possible without financial
support from a number of sources, particularly the Medical Research Council of the UK, the
Human Frontier Science Program, the Wellcome Trust, and the James S. McDonnell Founda-
tion. I am also grateful to many colleagues who I have consulted while writing this book. The
Gatsby Foundation is thanked for a grant towards the cost of making this book Open Access.
The book was typeset by the author using LATEXand WinEdt.

Updates to and .pdfs of many of the publications cited in this book are available at
https://www.oxcns.org. Updates and corrections to the text and notes are also available at
https://www.oxcns.org.

I dedicate this work to the overlapping group: my family, friends, and colleagues – in salutem
praesentium, in memoriam absentium.
Contents

1 Introduction 1
1.1 What and how the brain computes: introduction 1
1.2 What and how the brain computes: plan of the book 3
1.3 Neurons 5
1.4 Neurons in a network 6
1.5 Synaptic modification 8
1.6 Long-term potentiation and long-term depression 10
1.6.1 Long-Term Potentiation 10
1.6.2 Long-Term Depression 13
1.6.3 Spike Timing-Dependent Plasticity 14
1.7 Information encoding by neurons, and distributed representations 15
1.7.1 Definitions 16
1.7.2 Advantages of sparse distributed encoding 17
1.8 Neuronal network approaches versus connectionism 18
1.9 Introduction to three neuronal network architectures 19
1.10 Systems-level analysis of brain function 22
1.11 Brodmann areas 22
1.12 Human Connectome Project Multi-Modal Parcellation atlas of the human cortex 27
1.13 Connectivity of the human brain 36
1.13.1 Connections analyzed with diffusion tractography 37
1.13.2 Functional connectivity 37
1.13.3 Effective connectivity 37
1.14 The fine structure of the cerebral neocortex 39
1.14.1 The fine structure and connectivity of the neocortex 39
1.14.2 Excitatory cells and connections 40
1.14.3 Inhibitory cells and connections 41
1.14.4 Quantitative aspects of cortical architecture 44
1.14.5 Functional pathways through the cortical layers 46
1.14.6 The scale of lateral excitatory and inhibitory effects, and modules 51

2 The ventral visual system 53


2.1 Introduction and overview 53
2.1.1 Introduction 53
2.1.2 Overview of what is computed in the ventral visual system 53
2.1.3 Overview of how computations are performed in the ventral visual system 57
2.1.4 What is computed in the ventral visual system is unimodal, and is related to
other ‘what’ systems after the inferior temporal visual cortex 58
2.2 What: V1 – primary visual cortex 60
2.3 What: V2 and V4 – intermediate processing areas in the ventral visual system 61
2.4 What: Invariant representations of faces and objects in the inferior temporal visual
cortex 62
2.4.1 Reward value is not represented in the primate ventral visual system 62
2.4.2 Translation invariant representations 63
2.4.3 Reduced translation invariance in natural scenes 64
xii | Contents

2.4.4 Size and spatial frequency invariance 67


2.4.5 Combinations of features in the correct spatial configuration 67
2.4.6 A view-invariant representation 69
2.4.7 Learning in the inferior temporal cortex 71
2.4.8 A sparse distributed representation is what is computed in the ventral visual
system 74
2.4.9 Face expression, gesture, and view 80
2.4.10 Specialized regions in the temporal cortical visual areas 80
2.5 The connectivity of the ventral visual pathways in humans 85
2.5.1 A Ventrolateral Visual Cortical Stream to the inferior temporal visual cortex
for object and face representations 86
2.5.2 A Visual Cortical Stream to the cortex in the inferior bank of the superior
temporal sulcus involved in semantic representations 89
2.5.3 A Visual Cortical Stream to the cortex in the superior bank of the superior
temporal sulcus involved in multimodal semantic representations includ-
ing visual motion, auditory, somatosensory and social information 89
2.6 How the computations are performed: approaches to invariant object recognition 92
2.6.1 Feature spaces 93
2.6.2 Structural descriptions and syntactic pattern recognition 94
2.6.3 Template matching and the alignment approach 96
2.6.4 Invertible networks that can reconstruct their inputs 97
2.6.5 Deep learning 98
2.6.6 Feature hierarchies 98
2.7 Hypotheses about how the computations are performed in a feature hierarchy ap-
proach 104
2.8 VisNet: a model of how the computations are performed in the ventral visual system 107
2.8.1 The architecture of VisNet 108
2.8.2 Initial experiments with VisNet 117
2.8.3 The optimal parameters for the temporal trace used in the learning rule 124
2.8.4 Different forms of the trace learning rule, and error correction 126
2.8.5 The issue of feature binding, and a solution 134
2.8.6 Operation in a cluttered environment 147
2.8.7 Learning 3D transforms 154
2.8.8 Capacity of the architecture, and an attractor implementation 159
2.8.9 Vision in natural scenes – effects of background versus attention 162
2.8.10 The representation of multiple objects in a scene 170
2.8.11 Learning invariant representations using spatial continuity 172
2.8.12 Lighting invariance 173
2.8.13 Deformation-invariant object recognition 175
2.8.14 Learning invariant representations of scenes and places 176
2.8.15 Finding and recognising objects in natural scenes 178
2.8.16 Non-accidental properties, and transform invariant object recognition 181
2.9 Further approaches to invariant object recognition 183
2.9.1 Other types of slow learning 183
2.9.2 HMAX 183
2.9.3 Minimal recognizable configurations 188
2.9.4 Hierarchical convolutional deep neural networks 189
2.9.5 Sigma-Pi synapses 190
2.9.6 A principal dimensions approach to coding in the inferior temporal visual
cortex 190
2.10 Visuo-spatial scratchpad memory, and change blindness 192
2.11 Processes involved in object identification 194
2.12 Top-down attentional modulation is implemented by biased competition 195
2.13 Highlights on how the computations are performed in the ventral visual system 198

3 The dorsal visual system 201


3.1 Introduction, and overview of the dorsal cortical visual stream 201
Contents | xiii

3.2 Global motion in the dorsal visual system 202


3.3 Invariant object-based motion in the dorsal visual system 204
3.4 What is computed in the dorsal visual system: visual coordinate transforms 206
3.4.1 The transform from retinal to head-based coordinates 207
3.4.2 The transform from head-based to allocentric bearing coordinates 208
3.4.3 A transform from allocentric bearing coordinates to allocentric spatial view
coordinates 209
3.5 How visual coordinate transforms are computed in the dorsal visual system 211
3.5.1 Gain modulation 211
3.5.2 Mechanisms of gain modulation using a trace learning rule 211
3.5.3 Gain modulation by eye position to produce a head-centered representation
in Layer 1 of VisNetCT 213
3.5.4 Gain modulation by head direction to produce an allocentric bearing to a
landmark in Layer 2 of VisNetCT 214
3.5.5 Gain modulation by place to produce an allocentric spatial view represent-
ation in Layer 3 of VisNetCT 215
3.5.6 The utility of the coordinate transforms in the dorsal visual system 216
3.6 The human Dorsal Visual Cortical Stream 217
3.6.1 Dorsal stream visual division regions 217
3.6.2 MT+ complex regions (FST, LO1, LO2, LO3, MST, MT, PH, V3CD and V4t) 218
3.6.3 Intraparietal sulcus posterior parietal cortex, regions (AIP, LIPd, LIPv, MIP,
VIP; with IP0, IP1 and IP2) 219
3.6.4 Area 7 regions 220

4 The taste and flavor system 221


4.1 Introduction and overview 221
4.1.1 Introduction 221
4.1.2 Overview of what is computed in the taste and flavor system 221
4.1.3 Overview of how computations are performed in the taste and flavor system 223
4.2 Taste and related pathways: what is computed 223
4.2.1 Hierarchically organised anatomical pathways 223
4.2.2 Taste neuronal tuning become more selective through the taste hierarchy 226
4.2.3 The primary, insular, taste cortex represents what taste is present and its
intensity 228
4.2.4 The secondary, orbitofrontal, taste cortex, and its representation of the
reward value and pleasantness of taste 229
4.2.5 Sensory-specific satiety is computed in the orbitofrontal cortex 231
4.2.6 Oral texture is represented in the primary and secondary taste cortex: vis-
cosity and fat texture 234
4.2.7 Vision and olfaction converge using associative learning with taste to repre-
sent flavor in the secondary but not primary taste cortex 236
4.2.8 Top-down attention and cognition can modulate taste and flavor represen-
tations in the taste cortical areas 238
4.2.9 The tertiary taste cortex in the anterior cingulate cortex provides the rewards
for action-reward outcome learning 240
4.2.10 Taste, oral texture and flavor provide the rewards for eating, and the gut
provides satiety signals 241
4.3 Taste and related pathways: how the computations are performed 245
4.3.1 Increased selectivity of taste and flavor neurons through the hierarchy by
competitive learning and convergence 245
4.3.2 Pattern association learning of associations of visual and olfactory stimuli
with taste 245
4.3.3 Rule-based reversal of visual to taste associations in the orbitofrontal cortex 246
4.3.4 Sensory-specific satiety is implemented by adaptation of synapses onto
orbitofrontal cortex neurons 246
4.3.5 Top-down cognitive and attentional modulation is implemented by biased
activation 247

5 The olfactory system 251


xiv | Contents

5.1 Introduction 251


5.1.1 Overview of what is computed in the olfactory system 251
5.1.2 Overview of how the computations are performed in the olfactory system 252
5.2 What is computed in the olfactory system 253
5.2.1 1000 gene-encoded olfactory receptor types, and 1000 corresponding
glomerulus types in the olfactory bulb 253
5.2.2 The primary olfactory, pyriform, cortex: olfactory feature combinations are
what is represented 255
5.2.3 Orbitofrontal cortex: olfactory neuronal response selectivity 256
5.2.4 Orbitofrontal cortex: olfactory to taste convergence 257
5.2.5 Orbitofrontal cortex: olfactory to taste association learning and reversal 257
5.2.6 Orbitofrontal cortex: olfactory reward value is represented 259
5.2.7 Cognitive influences on olfactory representations in the orbitofrontal cortex 260
5.3 How computations are performed in the olfactory system 262
5.3.1 Olfactory receptors, and the olfactory bulb 262
5.3.2 Olfactory (pyriform) cortex 263
5.3.3 Orbitofrontal cortex 267

6 The somatosensory system 268


6.1 What is computed in the somatosensory system 268
6.1.1 The receptors and periphery 268
6.1.2 The anterior somatosensory cortex, areas 1, 2, 3a, and 3b, in the anterior
parietal cortex 268
6.1.3 The ventral somatosensory stream: areas S2 and PV, in the lateral parietal
cortex 269
6.1.4 The dorsal somatosensory stream to area 5 and then 7b, in the posterior
parietal cortex 270
6.1.5 Somatosensory representations in the insula 272
6.1.6 Somatosensory and temperature inputs to the orbitofrontal cortex, affective
value, pleasant touch, and pain 272
6.1.7 Decision-making in the somatosensory system 276
6.1.8 Somatosensory cortical regions and connectivity in humans 278
6.2 How computations are performed in the somatosensory system 284
6.2.1 Hierarchical computation in the somatosensory system 284
6.2.2 Computations for pleasant touch and pain 284
6.2.3 The mechanisms for somatosensory decision-making 285

7 The auditory system 286


7.1 Introduction, and overview of computations in the auditory system 286
7.2 Auditory Localization 287
7.3 Ventral and dorsal cortical auditory pathways 290
7.4 The ventral cortical auditory stream 291
7.5 The dorsal cortical auditory stream 293
7.6 Auditory cortical regions and connectivity in humans 293
7.6.1 Early Auditory cortical regions 294
7.6.2 Ventral auditory cortical streams 294
7.6.3 Dorsal auditory cortical streams 295
7.6.4 Other auditory system cortical connectivities 297
7.7 How the computations are performed in the auditory system 298

8 The temporal cortex 299


8.1 Introduction and overview 299
8.2 Middle temporal gyrus and face expression and gesture 299
8.3 Semantic representations in the temporal lobe neocortex 301
8.3.1 Neurophysiology of the medial temporal lobe, including concept cells 301
Contents | xv

8.3.2 Neuropsychology 303


8.3.3 Functional neuroimaging 303
8.3.4 Brain stimulation 304
8.4 Connectivity and functions of the human temporal lobe regions related to semantics 305
8.4.1 Group 1 semantic regions that include regions in the ventral bank of the
Superior Temporal Sulcus 306
8.4.2 Group 3 semantic regions that include regions in the dorsal bank of the
Superior Temporal Sulcus 309
8.5 The mechanisms for semantic learning in the human anterior temporal lobe 311

9 The hippocampus, memory, and spatial function 313


9.1 Introduction and overview 313
9.1.1 Overview of what is computed by the hippocampal system 313
9.1.2 Overview of how the computations are performed by the hippocampal sys-
tem 315
9.2 What is computed in the hippocampus 316
9.2.1 Systems-level anatomy 316
9.2.2 Evidence from the effects of damage to the hippocampus 319
9.2.3 Episodic memories need to be recalled from the hippocampus, and can be
used to help build neocortical semantic memories 321
9.2.4 Systems-level neurophysiology of the primate including human hippo-
campus 324
9.2.5 Spatial view cells in primates including humans, and foveal vision 342
9.2.6 Head direction cells in the presubiculum 344
9.2.7 Perirhinal cortex, recognition memory, and long-term familiarity memory 346
9.2.8 Connectivity of the human hippocampal system 353
9.3 How computations are performed in the hippocampal system 364
9.3.1 Historical development of the theory of the hippocampus 364
9.3.2 Hippocampal circuitry 366
9.3.3 Medial entorhinal cortex, spatial processing streams, and grid cells 369
9.3.4 Lateral entorhinal cortex, object processing streams, and the generation of
time cells in the hippocampus 372
9.3.5 CA3 as an autoassociation memory 378
9.3.6 Dentate granule cells 397
9.3.7 CA1 cells 400
9.3.8 Backprojections to the neocortex, memory recall, and consolidation 405
9.3.9 Backprojections to the neocortex – quantitative aspects 409
9.3.10 Simulations of hippocampal operation 413
9.3.11 The learning of spatial view and place cell representations 414
9.3.12 Linking the inferior temporal visual cortex to spatial view and place cells 416
9.3.13 A scientific theory of the art of memory: scientia artis memoriae 418
9.3.14 How navigation is performed 419
9.3.15 Navigational computations using neuron types found in primates including
humans 423
9.4 Tests of the theory of hippocampal cortex operation 432
9.4.1 Dentate gyrus (DG) subregion of the hippocampus 432
9.4.2 CA3 subregion of the hippocampus 435
9.4.3 CA1 subregion of the hippocampus 442
9.5 Comparison of spatial processing and computations in primates including humans vs
rodents 445
9.5.1 Similarities and differences between the spatial representations in primates
and rodents 445
9.5.2 Hippocampal computational similarities and differences between primates
and rodents 447
9.6 Synthesis: the hippocampus: memory, navigation, or both? 450
9.7 Comparison with other theories of hippocampal function 454
xvi | Contents

10 The parietal cortex, spatial functions, and navigation 459


10.1 Introduction and overview 459
10.1.1 Overview of what is computed in the parietal cortex 459
10.1.2 Overview of how the computations are performed in the parietal cortex 460
10.2 Inferior parietal cortex somatosensory stream, PF regions 460
10.3 Intraparietal sulcus posterior parietal cortex, regions AIP, LIPd, LIPv, MIP, VIP, IP0,
IP1, and IP2 464
10.4 Posterior superior parietal cortex, regions 7AL, 7Am, 7PC, 7PL, and 7Pm 466
10.5 Inferior parietal cortex, visual regions PFm, PGi, PGs and PGp 467
10.5.1 Region PGi 469
10.5.2 Region PGs 470
10.5.3 Region PFm 470
10.5.4 Region PGp 471
10.6 Navigation: What computations are performed in the parietal and related cortex 473
10.7 How the computations are performed in the parietal cortex 474

11 The orbitofrontal cortex, amygdala, reward value, emotion, and


decision-making 475
11.1 Introduction and overview 475
11.1.1 Introduction 475
11.1.2 Overview of what is computed in the orbitofrontal cortex 475
11.1.3 Overview of how the computations are performed by the orbitofrontal cortex 478
11.2 The topology and connections of the orbitofrontal cortex 479
11.2.1 Inputs to the orbitofrontal cortex 480
11.2.2 Outputs of the orbitofrontal cortex 482
11.3 What is computed in the orbitofrontal cortex 483
11.3.1 The orbitofrontal cortex represents reward value 483
11.3.2 Neuroeconomic value is represented in the orbitofrontal cortex 490
11.3.3 A representation of face and voice expression and other socially relevant
stimuli in the orbitofrontal cortex 492
11.3.4 Negative reward prediction error neurons in the orbitofrontal cortex 495
11.3.5 The human medial orbitofrontal cortex represents rewards, and the lateral
orbitofrontal cortex non-reward and punishers 500
11.3.6 Decision-making in the orbitofrontal / ventromedial prefrontal cortex 502
11.3.7 The ventromedial prefrontal cortex and memory 504
11.3.8 The orbitofrontal cortex and emotion 506
11.3.9 Emotional orbitofrontal vs rational routes to action 508
11.3.10 The connectivity of the human orbitofrontal cortex, and its relation to function 521
11.3.11 Mental problems associated with the orbitofrontal cortex 528
11.4 What is computed in the amygdala for emotion 529
11.4.1 Overview of the functions of the amygdala in emotion 529
11.4.2 The amygdala and the associative processes involved in emotion-related
learning 530
11.4.3 Connections of the amygdala 531
11.4.4 Effects of amygdala lesions 532
11.4.5 Amygdala lesions in primates 532
11.4.6 Amygdala lesions in rats 534
11.4.7 Neuronal activity in the primate amygdala to reinforcing stimuli 535
11.4.8 Neuronal responses in the amygdala to faces 536
11.4.9 Evidence from humans 538
11.4.10 Connectivity of the human amygdala 539
11.5 How the computations are performed in the orbitofrontal cortex 544
11.5.1 Decision-making in attractor networks in the brain 544
11.5.2 Analyses of reward-related decision-making mechanisms in the orbitofrontal
cortex 549
11.5.3 A model for reversal learning in the orbitofrontal cortex 554
Contents | xvii

11.5.4 A theory and model of non-reward neural mechanisms in the orbitofrontal


cortex 559
11.6 Highlights: the special computational roles of the orbitofrontal cortex 560

12 The cingulate cortex 564


12.1 Introduction to and overview of the cingulate cortex 564
12.1.1 Introduction 564
12.1.2 Overview of what is computed in the cingulate cortex 565
12.1.3 Overview of how the computations are performed by the cingulate cortex 567
12.2 Anterior cingulate cortex 568
12.2.1 Anterior cingulate cortex anatomy and connections in primates 568
12.2.2 Anterior cingulate cortex: A framework 569
12.2.3 Anterior cingulate cortex and action-outcome representations 571
12.2.4 Anterior cingulate cortex lesion effects 572
12.2.5 Anterior cingulate cortex and ventromedial prefrontal cortex connectivity and
functions in humans 572
12.2.6 Pregenual anterior cingulate representations of reward value, and supracal-
losal anterior cingulate representations of punishers and non-reward 575
12.2.7 The human supracallosal anterior cingulate cortex, dACC, and action-
outcome learning 578
12.2.8 Reward value outputs from the orbitofrontal and pregenual anterior cortex,
and vmPFC, to the hippocampal memory system 579
12.2.9 The pregenual anterior cingulate cortex has connectivity with the septal
cholinergic system that is involved in memory consolidation 579
12.2.10 Reward value outputs from the orbitofrontal and pregenual anterior cortex,
and vmPFC, to the hippocampal system to provide the goals for naviga-
tion 580
12.2.11 Subgenual cingulate cortex 581
12.3 Posterior cingulate cortex 581
12.3.1 Introduction and overview 581
12.3.2 Postero-ventral posterior cingulate and medial parietal regions 31pd, 31pv,
d23ab, v23ab and 7m, and their relation to episodic memory 582
12.3.3 Antero-dorsal Posterior Cingulate Division regions 23d, 31a, PCV; and RSC,
POS2, and POS1; and their relation to navigation and executive function 584
12.3.4 Dorsal Visual Transitional area and ProStriate region: the retrosplenial scene
area 587
12.4 Mid-cingulate cortex, the cingulate motor area, and action–outcome learning 588
12.5 How the computations are performed by the cingulate cortex 589
12.5.1 The anterior cingulate cortex and emotion 589
12.5.2 Action-outcome learning in the supracallosal anterior cingulate cortex
(dACC) 590
12.5.3 Connectivity of the posterior cingulate cortex with the hippocampal memory
system 592
12.6 Synthesis and conclusions 593

13 The prefrontal cortex 596


13.1 Introduction and overview 596
13.2 Divisions of the lateral prefrontal cortex 600
13.2.1 The dorsolateral prefrontal cortex 600
13.2.2 The caudal prefrontal cortex 603
13.2.3 The ventrolateral prefrontal cortex 603
13.3 The connectivity and computational organisation of the human prefrontal cortex 603
13.3.1 Inferior frontal gyrus 604
13.3.2 Dorsolateral prefrontal cortex division 606
13.4 The lateral prefrontal cortex and top-down attention 609
13.5 The frontal pole cortex 612
xviii | Contents

13.6 How the computations are performed in the prefrontal cortex 613
13.6.1 Cortical short-term memory systems and attractor networks 613
13.6.2 Prefrontal cortex short-term memory networks, and their relation to percep-
tual networks 615
13.6.3 Mapping from one representation to another in short-term memory 620
13.6.4 The mechanisms of top-down attention 621
13.6.5 Computational necessity for a separate, prefrontal cortex, short-term mem-
ory system 622
13.6.6 Synaptic modification is needed to set up but not to reuse short-term memory
systems 623
13.6.7 Sequence memory 623
13.6.8 Working memory, and planning 623

14 Language and syntax in the brain 624


14.1 Introduction and overview 624
14.1.1 Introduction 624
14.1.2 Overview 624
14.2 What is computed in different brain systems to implement language 626
14.2.1 The Wernicke-Lichtheim-Geschwind hypothesis 626
14.2.2 The dual-stream hypothesis of speech comprehension 627
14.2.3 Reading requires different brain systems to hearing speech 627
14.2.4 Semantic representations 629
14.2.5 Syntactic processing 630
14.2.6 The parietal cortex: supramarginal and angular gyri 631
14.3 Cortical regions for language and their connectivity in humans 631
14.3.1 A semantic system that includes the inferior bank of the superior temporal
sulcus including object representations 632
14.3.2 A semantic system that includes the superior bank of the superior temp-
oral sulcus including visual motion, auditory, somatosensory and social
information 634
14.3.3 Multimodal semantic representations 635
14.3.4 Broca’s area and related regions (TGv 44 45 47l SFL 55b) 637
14.4 Hypotheses about how semantic representations are computed 640
14.5 A neurodynamical hypothesis about how syntax is computed 641
14.5.1 Binding by synchrony? 641
14.5.2 Syntax using a place code 642
14.5.3 Temporal trajectories through a state space of attractors 642
14.5.4 Hypotheses about the implementation of language in the cerebral cortex 643
14.5.5 Tests of the hypotheses – a model 646
14.5.6 Tests of the hypotheses – findings with the model 651
14.5.7 Evaluation of the hypotheses 654
14.5.8 Further approaches 658

15 The motor cortical areas 660


15.1 Introduction and overview 660
15.2 What is computed in different cortical motor-related areas 661
15.2.1 Ventral parietal and ventral premotor cortex F4 661
15.2.2 Superior parietal areas with activity related to reaching 661
15.2.3 Inferior parietal areas with activity related to grasping, and ventral premotor
cortex F5 662
15.3 The mirror neuron system 662
15.4 How the computations are performed in motor cortical and related areas 664

16 The basal ganglia 665


16.1 Introduction and overview 665
16.2 Systems-level architecture of the basal ganglia 666
Contents | xix

16.3 What computations are performed by the basal ganglia? 669


16.3.1 Effects of striatal lesions 669
16.3.2 Neuronal activity in different parts of the striatum 670
16.4 How do the basal ganglia perform their computations? 683
16.4.1 Interaction between neurons and selection of output 683
16.4.2 Convergence within the basal ganglia, useful for stimulus-response habit
learning 686
16.4.3 Dopamine as a reward prediction error signal for reinforcement learning in
the striatum 688
16.5 Comparison of computations for selection in the basal ganglia and cerebral cortex 692

17 Cerebellar cortex 695


17.1 Introduction 695
17.2 Architecture of the cerebellum 697
17.2.1 The connections of the parallel fibres onto the Purkinje cells 697
17.2.2 The climbing fibre input to the Purkinje cell 698
17.2.3 The mossy fibre to granule cell connectivity 698
17.3 Modifiable synapses of parallel fibres onto Purkinje cell dendrites 700
17.4 The cerebellar cortex as a perceptron 701
17.5 Cognitive functions of the cerebellum 702
17.5.1 Anatomical connections from most neocortical regions 702
17.5.2 Functional connectivity of different cortical systems with different parts of
the cerebellum 703
17.5.3 Activation of different cerebellar cortical regions in different tasks 703
17.5.4 Damage to different parts of the cerebellum can produce different cognitive,
emotional, and motor impairments 703
17.5.5 Neocortical–cerebellar cortical computations for cognition 703
17.6 Highlights: differences between cerebral and cerebellar cortex microcircuitry 707

18 Cortical attractor dynamics and connectivity, stochasticity, psychi-


atric disorders, and aging 709
18.1 Introduction and overview 709
18.1.1 Introduction 709
18.1.2 Overview 709
18.2 The noisy cortex 711
18.2.1 Reasons why the brain is inherently noisy and stochastic 711
18.2.2 Attractor networks, energy landscapes, and stochastic neurodynamics 714
18.2.3 A multistable system with noise 719
18.2.4 Stochastic dynamics and the stability of short-term memory 721
18.2.5 Stochastic dynamics in decision-making, and the evolutionary utility of prob-
abilistic choice 725
18.2.6 Selection between conscious vs unconscious decision-making, and free will 726
18.2.7 Stochastic dynamics and creative thought 728
18.2.8 Stochastic dynamics and unpredictable behavior 729
18.3 Attractor dynamics and schizophrenia 729
18.3.1 Introduction 729
18.3.2 A dynamical systems hypothesis of the symptoms of schizophrenia 730
18.3.3 Reduced functional connectivity of some brain regions in schizophrenia 733
18.3.4 Beyond the disconnectivity hypothesis of schizophrenia: reduced forward
but not backward connectivity 734
18.4 Attractor dynamics and obsessive-compulsive disorder 738
18.4.1 Introduction 738
18.4.2 A hypothesis about obsessive-compulsive disorder 739
18.4.3 Glutamate and increased depth of the basins of attraction 741
18.5 Depression and attractor dynamics 742
xx | Contents

18.5.1 Introduction 742


18.5.2 A non-reward attractor theory of depression 743
18.5.3 The orbitofrontal cortex, and the theory of depression 745
18.5.4 Altered connectivity of the orbitofrontal cortex in depression 746
18.5.5 Activations of the orbitofrontal cortex related to depression 751
18.5.6 Implications, and possible treatments, and subtypes of depression 753
18.5.7 Mania and bipolar disorder 756
18.6 Attractor stochastic dynamics, aging, and memory 758
18.6.1 NMDA receptor hypofunction 758
18.6.2 Dopamine and norepinephrine 760
18.6.3 Impaired synaptic modification 760
18.6.4 Cholinergic function and memory 761
18.7 High blood pressure, reduced hippocampal functional connectivity, and impaired
memory 766
18.8 Brain development, and structural differences in the brain 766

19 Computations by different types of brain, and by artificial neural


systems 768
19.1 Introduction and overview 768
19.2 Computations that combine different computational systems in the brain to produce
behavior 769
19.3 Brain computation compared to computation on a digital computer 769
19.4 Brain computation compared with artificial deep learning networks 775
19.5 Reinforcement Learning 778
19.6 Levels of explanation, and the mind-brain problem 780
19.6.1 A levels of explanation theory of causality, and the relation between the mind
and the brain 780
19.6.2 Downward or Upward Causality? 782
19.6.3 Consciousness - a Higher Order Syntactic Thought theory 784
19.6.4 Levels of explanation, and levels of investigation 785
19.7 Biologically plausible computation in the brain: a grand unifying theory? 787
19.8 Brain-Inspired Intelligence 791
19.9 Brain-Inspired Medicine 792
19.9.1 Computational psychiatry and neurology 792
19.9.2 Reward systems in the brain, and their application to understanding food
intake control and obesity 793
19.9.3 Multiple Routes to Action 796
19.10 Primates including humans have different brain organisation than rodents 796
19.10.1 The visual system 796
19.10.2 The taste system 797
19.10.3 The olfactory system 798
19.10.4 The somatosensory system 798
19.10.5 The auditory system 799
19.10.6 The hippocampal system, memory, and navigation 799
19.10.7 The orbitofrontal cortex and amygdala 800
19.10.8 The cingulate cortex 801
19.10.9 The motor system 802
19.10.10 Language 802

A Introduction to linear algebra for neural networks 803


A.1 Vectors 803
A.1.1 The inner or dot product of two vectors 803
A.1.2 The length of a vector 805
A.1.3 Normalizing the length of a vector 805
Contents | xxi

A.1.4 The angle between two vectors: the normalized dot product 805
A.1.5 The outer product of two vectors 806
A.1.6 Linear and non-linear systems 807
A.1.7 Linear combinations, linear independence, and linear separability 808
A.2 Application to understanding simple neural networks 809
A.2.1 Capability and limitations of single-layer networks 810
A.2.2 Non-linear networks: neurons with non-linear activation functions 812
A.2.3 Non-linear networks: neurons with non-linear activations 813

B Neuronal network models 815


B.1 Introduction 815
B.2 Pattern association memory 815
B.2.1 Architecture and operation 816
B.2.2 A simple model 818
B.2.3 The vector interpretation 821
B.2.4 Properties 822
B.2.5 Prototype extraction, extraction of central tendency, and noise reduction 825
B.2.6 Speed 825
B.2.7 Local learning rule 826
B.2.8 Implications of different types of coding for storage in pattern associators 831
B.3 Autoassociation or attractor memory 832
B.3.1 Architecture and operation 832
B.3.2 Introduction to the analysis of the operation of autoassociation networks 834
B.3.3 Properties 836
B.3.4 Diluted connectivity and the storage capacity of attractor networks 843
B.3.5 Use of autoassociation networks in the brain 854
B.4 Competitive networks, including self-organizing maps 855
B.4.1 Function 855
B.4.2 Architecture and algorithm 856
B.4.3 Properties 857
B.4.4 Utility of competitive networks in information processing by the brain 862
B.4.5 Guidance of competitive learning 864
B.4.6 Topographic map formation 866
B.4.7 Invariance learning by competitive networks 870
B.4.8 Radial Basis Function networks 871
B.4.9 Further details of the algorithms used in competitive networks 873
B.5 Continuous attractor networks 876
B.5.1 Introduction 876
B.5.2 The generic model of a continuous attractor network 878
B.5.3 Learning the synaptic strengths in a continuous attractor network 879
B.5.4 The capacity of a continuous attractor network: multiple charts 881
B.5.5 Continuous attractor models: path integration 882
B.5.6 Stabilization of the activity packet within a continuous attractor network 884
B.5.7 Continuous attractor networks in two or more dimensions 886
B.5.8 Mixed continuous and discrete attractor networks 887
B.6 Network dynamics: the integrate-and-fire approach 888
B.6.1 From discrete to continuous time 888
B.6.2 Continuous dynamics with discontinuities 889
B.6.3 An integrate-and-fire implementation 893
B.6.4 The speed of processing of attractor networks 894
B.6.5 The speed of processing of a four-layer hierarchical network 897
B.6.6 Spike response model 900
B.7 Network dynamics: introduction to the mean-field approach 901
B.8 Mean-field based neurodynamics 902
B.8.1 Population activity 903
B.8.2 The mean-field approach used in a model of decision-making 905
B.8.3 The model parameters used in the mean-field analyses of decision-making 907
B.8.4 A basic computational module based on biased competition 908
xxii | Contents

B.8.5 Multimodular neurodynamical architectures 909


B.9 Interacting attractor networks 911
B.10 Sequence memory implemented by adaptation in an attractor network 915
B.11 Error correction networks 915
B.11.1 Architecture and general description 916
B.11.2 Generic algorithm for a one-layer error correction network 916
B.11.3 Capability and limitations of single-layer error-correcting networks 917
B.11.4 Properties 920
B.12 Error backpropagation multilayer networks 922
B.12.1 Introduction 922
B.12.2 Architecture and algorithm 923
B.12.3 Properties of multilayer networks trained by error backpropagation 927
B.13 Deep learning using stochastic gradient descent 928
B.14 Deep convolutional networks 928
B.15 Contrastive Hebbian learning: the Boltzmann machine 930
B.16 Deep Belief Networks 931
B.17 Reinforcement learning 932
B.17.1 Associative reward–penalty algorithm of Barto and Sutton 933
B.17.2 Reward prediction error or delta rule learning, and classical conditioning 934
B.17.3 Temporal Difference (TD) learning 935
B.18 Learning in the neocortex 938
B.19 Forgetting in cortical associative neural networks, and memory reconsolidation 940
B.20 Genes and self-organization build neural networks in the cortex 945
B.20.1 Introduction 945
B.20.2 Hypotheses about the genes that build cortical neural networks 946
B.20.3 Genetic selection of neuronal network parameters 950
B.20.4 Simulation of the evolution of neural networks using a genetic algorithm 952
B.20.5 Evaluation of the gene-based evolution of single-layer networks 961
B.20.6 The gene-based evolution of multi-layer cortical systems 964
B.20.7 Summary 965
B.21 Highlights 965

C Neuronal encoding, and information theory 967


C.1 Information theory 968
C.1.1 The information conveyed by definite statements 968
C.1.2 Information conveyed by probabilistic statements 969
C.1.3 Information sources, information channels, and information measures 970
C.1.4 The information carried by a neuronal response and its averages 971
C.1.5 The information conveyed by continuous variables 974
C.2 The information carried by neuronal responses 976
C.2.1 The limited sampling problem 976
C.2.2 Correction procedures for limited sampling 977
C.2.3 The information from multiple cells: decoding procedures 978
C.2.4 Information in the correlations between cells: a decoding approach 982
C.2.5 Information in the correlations between cells: second derivative approach 987
C.3 Neuronal encoding results 990
C.3.1 The sparseness of the distributed encoding used by the brain 991
C.3.2 The information from single neurons 1002
C.3.3 The information from single neurons: temporal codes versus rate codes 1005
C.3.4 The information from single neurons: the speed of information transfer 1008
C.3.5 The information from multiple cells: independence versus redundancy 1019
C.3.6 Should one neuron be as discriminative as the whole organism? 1023
C.3.7 The information from multiple cells: the effects of cross-correlations 1025
C.3.8 Conclusions on cortical neuronal encoding 1029
Contents | xxiii

C.4 Information theory terms – a short glossary 1033


C.5 Highlights 1034

D Simulation software for neuronal networks, and information analysis


of neuronal encoding 1035
D.1 Introduction 1035
D.2 Autoassociation or attractor networks 1036
D.2.1 Running the simulation 1036
D.2.2 Exercises 1038
D.3 Pattern association networks 1038
D.3.1 Running the simulation 1038
D.3.2 Exercises 1040
D.4 Competitive networks and Self-Organizing Maps 1041
D.4.1 Running the simulation 1041
D.4.2 Exercises 1042
D.5 Further developments 1043
D.6 Matlab code for a tutorial version of VisNet 1043
D.7 Matlab code for information analysis of neuronal encoding 1044
D.8 Matlab code to illustrate the use of spatial view cells in navigation 1044
D.9 The Automated Anatomical Labelling Atlas 3, AAL3 1044
D.10 The extended Human Connectome Project extended atlas, HCPex 1044
D.11 Highlights 1044

Bibliography 1045

Index 1139
1 Introduction

1.1 What and how the brain computes: introduction


The subject of this book is how the brain works. In order to understand this, it is essential to
know what is computed by different brain systems; and how the computations are performed.
The aim of this book is to elucidate what is computed in different brain systems; and to
describe current computational approaches and models of how each of these brain systems
computes. Understanding the brain in this way has enormous potential for understanding
ourselves better in health and in disease. Potential applications of this understanding are to
the treatment of the brain in disease; and to artificial intelligence which will benefit from
knowledge of how the brain performs many of its extraordinarily impressive functions. This
book is pioneering in taking this approach to brain function: to consider what is computed
by many of our brain systems; and how it is computed.
To understand how our brains work, it is essential to know what is computed in each
part of the brain. That can be addressed by utilising evidence relevant to computation from
many areas of neuroscience. Knowledge of the connections between different brain areas is
important, for this shows that the brain is organised as systems, with whole series of brain
areas devoted for example to visual processing. That provides a foundation for examining
the computation performed by each brain area, by comparing what is represented in a brain
area with what is represented in the preceding and following brain areas, using techniques
of for example neurophysiology and functional neuroimaging. Neurophysiology at the single
neuron level is needed because this is the level at which information is transmitted between
the computing elements of the brain, the neurons. Evidence from the effects of brain damage,
including that available from neuropsychology, is needed to help understand what different
parts of the system do, and indeed what each part is necessary for. Functional neuroimaging
is useful to indicate where in the human brain different processes take place, and to show
which functions can be dissociated from each other. So for each brain system, evidence on
what is computed at each stage, and what the system as a whole computes, is essential.
To understand how our brains work, it is also essential to know how each part of the brain
computes. That requires a knowledge of what is represented and computed by the neurons
in each part of the brain, but it also requires knowledge of the network properties of each
brain region. This involves knowledge of the connectivity between the neurons in each part
of the brain, and knowledge of the synaptic and biophysical properties of the neurons. It
also requires knowledge of the theory of what can be computed by networks with defined
connectivity.
There are at least three key goals of the approaches described here. One is to understand
ourselves better, and how we work and think. A second is to be better able to treat the sys-
tem when it has problems, for example in mental illnesses. Medical applications are a very
important aim of the type of research described here. A third goal, is to be able to emulate
and learn from the operation of parts of our brains, which some in the field of artificial intel-
ligence (AI) would like to do to produce more useful computers and machines. All of these
goals require, and cannot get off the ground, without a firm foundation in what is computed
by brain systems, and theories and models of how it is computed. To understand the operation

Brain Computations and Connectivity. Edmund T. Rolls, Oxford University Press. © Edmund T. Rolls 2023.
DOI: 10.1093/oso/9780198887911.003.0001
2 | Introduction

of the whole brain, it is necessary to show how the different brain systems operate together:
but a necessary foundation for this is to know what is computed in each brain system.
Part of the enterprise here is to stimulate new theories and models of how parts of the brain
work. The evidence on what is computed in different brain systems has advanced rapidly in
the last 50 years, and provides a reasonable foundation for the enterprise, though there is
much that remains to be learned. Theories of how the computation is performed are less ad-
vanced, but progress is being made, and current models are described in this book for many
brain systems. Before further advances are made, knowledge of the considerable current evi-
dence on how the brain computes provides a useful stating point, especially as current theories
do take into account the limitations that are likely to be imposed by the neural architectures
present in our brains.
The simplest way to define brain computation is to examine what information is repres-
ented at each stage of processing, and how this is different from stage to stage. For example
in the primary visual cortex (V1), neurons respond to simple stimuli such as bars or edges or
gratings and have small receptive fields. Little can be read off about for example whose face
is represented from the firing rates of a small number of neurons in V1. On the other hand,
after four or five stages of processing, in the inferior temporal visual cortex, information can
be read from the firing rates of neurons about whose face is being viewed, and indeed there is
remarkable invariance with respect to the position, size, contrast and even in some cases view
of the face. That is a major computation, and indicates what can be achieved by biological
neuronal network computation.
These approaches can only be taken to understand brain function because there is consid-
erable localization of function in the brain, quite unlike a digital computer. One fundamental
reason for localization of function in the brain is that this minimizes the total length of the
connections between neurons, and thus brain size. Another is that it simplifies the genetic
information that has to be provided in order to build the brain, because the connectivity in-
structions can refer considerably to local connections. These points are developed in my book
Cerebral Cortex: Principles of Operation (Rolls, 2016b).
That brings me to what is different about the present book and Cerebral Cortex: Princi-
ples of Operation (Rolls, 2016b). The previous book took on the enormous task of making
progress with understanding how the major part of our brains, the cerebral cortex, works, by
understanding its principles of operation. The present book builds on that approach, and uses
it as background, but has the different aim of taking each of our brain systems, and describing
what they compute, and then what is known about how each system computes. The issue of
how they compute relies for many brain systems on how the cortex operates, so Cerebral
Cortex: Principles of Operation provides an important complement to the present book.
One of the distinctive properties of this book is that it links the neural computation ap-
proach not only firmly to neuronal neurophysiology, which provides much of the primary data
about how the brain operates, but also to psychophysical studies (for example of attention);
to neuropsychological studies of patients with brain damage; and to functional magnetic res-
onance imaging (fMRI) (and other neuroimaging) approaches. The empirical evidence that
is brought to bear is largely from non-human primates and from humans, because of the con-
siderable similarity of their cortical systems, and the overall aims to understand the human
brain, and the disorders that arise after brain damage.
In selecting the research findings on ‘what’ is computed in different brain systems and
on ‘how’ it is computed, to include in this book, I have selected pioneering research that
has helped to identify key computational principles involved for different brain systems. Dis-
coveries that have laid the foundation for our understanding as research has developed are
emphasized. That has meant that much excellent neuroscience research could not be included
in this book: but the aim of the book instead is to identify computational principles of op-
What and how the brain computes: plan of the book | 3

eration of brain systems, providing some of the key research discoveries that have helped to
identify those principles. I hope that future research will extend this aim further.

1.2 What and how the brain computes: plan of the book
In the rest of Chapter 1, I introduce some of the background for understanding brain com-
putation, such as how single neurons operate, how some of the essential features of this can
be captured by simple formalisms, and some of the biological background to what it can be
taken happens in the nervous system, such as synaptic modification based on information
available locally at each synapse.
Each of the following Chapters has parts concerned with what is computed in a brain
system, followed with parts concerned with how it is computed.
Chapters 2–7 consider the main sensory processing systems of the brain: the ventral visual
system with its functions in invariant object recognition (2); the dorsal visual system involved
in spatial visual processing, actions in space, and the spatial coordinate transforms that are
needed (3); the taste system (4); the olfactory system (5); the somatosensory system (6); and
the auditory system (7).
Chapter 8 considers the functions of the temporal lobe, including its functions in semantic
representations.
Chapter 9 considers the functions of the hippocampus, which combines neocortical ‘what’,
‘where’ and reward information to implement episodic memory, and its later recall back to
the neocortex which is implicated in memory consolidation.
Chapter 10 considers the functions of the parietal cortex, which receives much input from
the dorsal processing streams concerned with the location of stimuli and actions in space –
the ‘where and action’ processing stream, and which is also involved in navigation.
Chapter 11 describes the functions of the orbitofrontal cortex and amygdala, which com-
pute reward value from inputs received from the ventral processing streams, and are there-
fore central to understanding the implementation in the brain of emotion and reward-related
decision-making. Chapter 12 shows how the anterior cingulate cortex has functions related
to the initiation of goal-directed action given its inputs from the orbitofrontal cortex about
reward value and its connections with the mid-cingulate motor parts of the cingulate; and
how the posterior cingulate cortex has major connections that link ‘what’ and ‘where’ neo-
cortical systems with the hippocampus, making it an important component of the episodic
memory system, and perhaps with connected cortical regions involved in navigation.
Chapter 13 describes the prefrontal cortex, which provides off-line short-term and work-
ing memory, and the associated functions of planning and top-down attention.
Chapter 14 describes brain systems involved in language and syntax.
Chapter 15 describes motor cortical areas. Chapter 16 describes the basal ganglia, and
Chapter 17 the cerebellum, which are both involved in motor function, but which are now
also implicated in emotional, social, and cognitive functions.
Chapter 18 describes factors that influence the dynamics and stability of cortical systems,
and how these are different in some mental disorders including Schizophrenia, Obsessive-
Compulsive Disorder and Depression.
Chapter 19 describes how the functions of each of these systems provide a foundation
for understanding functions implemented by the whole brain, including many types of com-
plex and useful behaviors. This Chapter also compares computations in the brain with those
performed in digital computers including with machine learning algorithms. This is intended
to be a useful guide for engineers and artificial intelligence experts who may wish to learn
4 | Introduction

from how computations are performed in the brain, and how to emulate the computations per-
formed by the brain. This Chapter also compares the systems-level organisation of comput-
ations in rodent brains with those in primate including human brains, and provides evidence
that there are many differences in the systems-level computations between rodent and prim-
ate including human brains, which is why this book focusses on computations in primate
including human brains.
The Appendices are updated somewhat from those in Cerebral Cortex: Principles of Op-
eration (Rolls, 2016b), and are provided in this book for those who do not have access to
Cerebral Cortex. Appendix A provides an introduction to the linear algebra that is useful
background for the description of the fundamental operation of key networks found in the
brain. Appendix B describes the fundamental operation of key networks of the type that are
likely to be the building blocks of brain function. Appendix C shows how information theory
can be applied to analyze what is represented by populations of neurons and thus to under-
stand brain function. Appendix D describes Matlab software that has been made available
with this book to provide simple demonstrations of the operation of some key neuronal net-
works related to cortical function; to show how the information represented by neurons can
be measured; and to provide a tutorial version of the VisNet program for invariant visual ob-
ject recognition described in Chapter 2. The neural networks programs are also provided in
Python. The programs are available at https://www.oxcns.org.

I start the rest of Chapter 1 with some history showing how recently it is that our under-
standing of the (computational) principles of operation of the brain and cerebral cortex has
started to develop.
Before the 1960s there were many and important discoveries about the phenomenology
of the cortex, for example that damage in one part would affect vision, and in another part
movement, with electrical stimulation often producing the opposite effect. The principles may
help us to understand these phenomena, but the phenomena provide limited evidence about
how the cortex works, apart from the very important principle of localization of function (see
Rolls (2016b)), and the important principle of hierarchical organization (Hughlings Jackson,
1878; Swash, 1989) (see Rolls (2016b)) which has been supported by increasing evidence on
the connections between different cortical areas, which is a fundamental building block for
understanding brain computations.
In the 1960s David Hubel and Torsten Wiesel made important discoveries about the stim-
uli that activate primary visual cortex neurons, showing that they respond to bar-like or edge-
like visual stimuli (Hubel and Wiesel, 1962, 1968, 1977), instead of the small circular re-
ceptive fields in the preceding stage the lateral geniculate nucleus neurons. This led them to
suggest the elements of a model of how this might come about, by cortical neurons that re-
spond to elongated lines of lateral geniculate neurons (Fig. 2.5). This led to the concept that
hierarchical organization over a series of cortical areas might at each stage form combina-
tions of the features represented in the previous cortical area, in what might be termed feature
combination neurons.
However, before 1970 there were few ideas about how the cerebral cortex operates com-
putationally.
David Marr was a pioneer who helped to open the way to an understanding of how the
details of cortical anatomy and connectivity help to develop quantitative theories of how cort-
ical areas may compute, including the cerebellar cortex (Marr, 1969), the neocortex (Marr,
1970), and the hippocampal cortex (Marr, 1971). Marr was hampered by some lack in detail
of the available anatomical knowledge, and did not for example hypothesize that the hip-
pocampal CA3 network was an autoassociation memory. He attempted to test his theory of
Another random document with
no related content on Scribd:
CHAPTER VII.
HANNIBAL’S FIRST TRIUMPH.

“Now, Maharbal,” quoth the chief, “I would talk to thee no longer as


one general to another, but simply as man to man. What about my
daughter Elissa? Hast thou forgotten her? Hath not perchance all
this terrible fighting for the last six months knocked all the love
nonsense with which thou wast imbued out of thy warlike head, or is
there still left paramount therein the memory of that girl of mine?
Now, wilt thou answer plainly, for I have something to propose to
thee which may be of importance?”
Maharbal made no answer, but Hannibal rose, unlocked a small
casket and drew out a scroll, which he perused while waiting for a
reply.
“Well,” he remarked, seeing that no reply came, “I would know thy
mind on this matter, my friend.”
“Hannibal,” said Maharbal, rising in turn, and confronting his Chief,
“Hannibal,”—then he paused and threw down his sword with a
somewhat angry and impatient movement—“by what right dost thou
talk to me of Elissa? What is it to thee if I should think of her still or
no? As a warrior and my Chief I may listen to thee, ay, both must
and would listen to thee; but what have women to do with me now? I
am here to fight for Carthage and mine honour, ay, and for thine own
honour, too, Hannibal, but nothing more.”
“Honour is honour, but friendship is friendship. War is also war,
and we are all for our country; but private interests, nevertheless,
rule us all at times. Thou knowest this as well as I, therefore, as
friend to friend, tell me now the truth, Maharbal.”
“Well, the truth is this, Hannibal. When I meet a foe and he
confronts me,”—Maharbal excitedly arose and seized his sword and
shook it savagely—“I say to myself, ‘This for Elissa.’ Then I strike
home. When my foe is struck down and bleeding at my feet, and the
point of my weapon is at his throat, again I sometimes say to myself,
‘I spare thee for the sake of Elissa,’ and thus it ofttimes haps that a
human life is saved. When again I charge into the battle, the one
sweet name Elissa is ever on my lips. When I was day after day in
those terrible passes of the Alps, and the rocks and boulders falling
all around me slew so many of my friends and fellow-warriors, but
one thought arose to my brain, and it was this, I care not for death
itself, but will Elissa regret me? And now, Hannibal, my friend and
my Chief, thou hast thine answer; I need say no more.”
“Nay,” said Hannibal, “thou hast said enough; I understand thee—
thou art constant. But will she be equally constant? It may be years
ere thou see her again. But young is she, and springing from a very
passionate stock; her mother was an Iberian woman. What wouldst
thou do supposing that she proved inconstant unto thee and loved
another?”
“Do?—why, fight for Carthage still. What else could I do? Ay, and
scorn her, too, if inconstant; nought else could be done than that.”
“Well, listen, I have something to propose. Thou seest this scroll.
’Tis a letter to my daughter Elissa. To-morrow we may have a fight,
not with these Gauls alone, who have already learned to fear me, but
with Scipio himself, with Roman legions at his back.
“It may be that, owing to the somewhat demoralised condition of
our army, Scipio should prove victorious; it may be otherwise; yet my
belief in the gods is so great, that I think I shall overthrow him.
Whatever may chance, I must send my messengers with tidings
back into Iberia to say that we are now safely across the Alps; to
inform first my daughter Elissa at New Carthage, and then to bear
intelligence unto my brother Hasdrubal, in those northern parts
beyond the Ebro, where, I have learned, that Cnœus Scipio hath
proceeded to attack him. Now, Maharbal, I need a trusty messenger;
wilt thou be the bearer of the message? I have in the camp a
chieftain of the Insubrian Gauls, who hath promised me a small fleet
of ships at Genua, by which thou couldst proceed to New Carthage,
where Elissa will doubtless warmly welcome thee to her loving arms.
Wilt thou go? that is if thou shouldst survive to-morrow’s fight. Think,
my lad, after all this hardship, how sweet will be the delights of love.
Moreover, now that thou hast, by thine incessant toil and valour,
brought the cavalry, or rather a great part of it, over the mountains in
safety, know this, I will no longer withstand thy wish. Thy nuptials
with my daughter shall be recognised, and thou and Elissa shall be
man and wife.”
Maharbal’s face flushed as he leaned forward on his seat and
gazed at Hannibal with undisguised astonishment.
“My lord Hannibal, hast thou then forgotten the toast we drank to
one another at Saguntum? ’Tis true I love Elissa more than all the
world as a woman; but I love thee, mine own honour, and my country
Carthage, more than any woman living, be she even thine own
daughter and mine own beloved bride. Therefore must I decline to
leave thy side, oh Hannibal, to seek repose in thy dearest and most
beloved daughter’s arms. My duty is here, and here, with all
deference unto thee, I stay. Through life and death I am thine, but to
do thy bidding in our country’s cause. My personal longings and
lovings are now things unknown. I fight for Hannibal and for
Carthage; all else is forgotten, or, if not forgotten, must and shall be
crushed from out my breast. And yet, the gods are my witness, I love
Elissa far more than all.”
Maharbal rose, and sought to leave the tent; but as he rose,
Hannibal detained him by placing a hand on his shoulder.
“I admire thee, Maharbal, far more than if thou hadst acceded to
my request. In truth thou hast enacted a right noble part. But should
we both live, I shall not forget it, and thou thyself wilt not regret it.
Should we die, I feel convinced that all the gods who, in my vision,
promised me a future reward, will recognise thy virtue, and bestow a
far higher reward upon thee. Ay, Melcareth shall smile and shower
his blessings upon thee. Tanais shall make thee happy with such a
thrilling and heavenly love as never yet mortal hath known; while the
great Moloch, god of war, shall surely exalt thee to high rank in his
celestial armies. Thou hast, indeed, chosen the higher and the better
path. I pray the great god Melcareth, that, in my prolonged absence
from her side, he inspire Elissa’s heart also with devotion to her
country, with constancy and virtue. But ’twill be merit great, indeed,
to be worthy of a virtue so great as thine. And now, let us go forth,
since I must seek some other messenger to bear her my scroll, and
thou thyself canst, if thou wilt, despatch a letter by the same hand.
But nothing can be done before we have first once met the Romans
here on Italian soil.”
The rest of the day was spent in that camp by the Ticinus in
preparations for the great cavalry reconnaissance, which Hannibal
intended to conduct in person, on the morrow. Armour was furbished
up, swords and darts were sharpened, bits and bridles were seen to.
All the cavalrymen, even although the infantry suffered somewhat in
consequence, were thoroughly well fed, and also supplied with a
liberal allowance of wine. Olive oil was dealt out to them all,
wherewith to anoint themselves and make their limbs supple, and a
day’s rations, to be carried on the morrow, was served out to each
man. Scouts from a reserve body of horse, that was not to be
employed in any fighting on the morrow, were sent out to watch the
enemy’s movements, with instructions to leave a series of detached
posts at intervals, by whom, owing to there being thus relays of fresh
horsemen ready, news could be swiftly conveyed to the Carthaginian
Chief of Scipio’s slightest movements. All these details Hannibal, not
content with trusting to subordinates, saw to personally, for as he
was, saving only Maharbal, the strongest man in the army, so was
he also the most indefatigable. Throughout the whole of that day,
therefore, he scarcely rested, but visited every part of the camp and
every troop in turn, seeing that his instructions were carried out
absolutely to the letter, and speaking grand and noble words of
encouragement wherever he passed.
When, therefore, after a substantial breakfast, the whole of the
cavalry paraded on the following morn, it was no longer a starveling,
dispirited body of men that fell beneath his gaze, but a gallant band
of warriors bearing confidence in their glance, self-reliant, proud, and
anxious for the fray. Well satisfied with the noble bearing of his
followers, after having first ridden round the ranks and complimented
his men upon their brave and soldierly appearance, Hannibal bade
his trumpeter sound the advance, and himself led the way. As he put
his horse in motion, the whole army burst spontaneously into a cheer
—“Hannibal! Long live Hannibal.”
Clad in gorgeous but serviceable armour, with dancing plumes
waving from the crest of his helmet, the gallant Carthaginian General
was indeed, as he marched forth that morn to meet the Romans for
the first time in his career, such a leader as the world had never
seen.
As the cavalry, by troops and squadrons, filed off in succession
after him, silence fell among their ranks. But the infantry soldiers and
also the Gauls remaining behind in camp continued the shouts of
enthusiasm. “Hannibal! Long live Hannibal!” For it was for him, the
General alone, and his personal influence, more than for a Carthage
which was to most of them but a name, that these troops, drawn
from many mixed tribes and nations, were willing and ready to lay
down their lives. And it was ever so; it was Hannibal’s army, not a
Carthaginian army that so often defeated Rome.
Thus gallantly encouraged by the shouts of their comrades did the
whole of the Carthaginian horse march off down the banks of the
river Ticinus. For a considerable distance the ground was broken
and rocky, and considerably wooded, and not at all suitable for the
manœuvring of mounted troops. But, at length, after a march of
some three hours’ duration, just at the very spot which suited his
purpose, Hannibal was met by the first of his scouts coming in and
informing him that Scipio had started with his forces, and was now
crossing the river. They gave the further information that there was
no occasion for hurry, as each relay of messengers had galloped
back hard, and that the remainder of the vedettes, now retiring
leisurely before the advancing foe, would give timely warning of their
nearer approach.
The ground whereon the front of the column was standing when
this news arrived was at the edge of an extensive copse of detached
oak trees, beyond which was a large open plain of what had earlier
in the year been cultivated ground. Drawing up all his forces just
inside the edge, and in the shelter of the trees, in the order of battle
which he intended to employ later on, the Chief now gave the order
for the men to dismount to rest the animals, and to eat a portion of
the rations that they had brought with them. He had a short time
previously made each troop halt in turn at a convenient shallow
place in the river Ticinus to give their horses a mouthful of water—
therefore no possible precaution had been neglected to bring his
forces fresh into battle. While halting in the shelter of the oak trees,
several more groups of messengers came in from the front. From
these Hannibal learned without a doubt that Scipio was advancing
only with his cavalry and light-armed footmen, and that from all
accounts the numbers of the former were, if anything, somewhat
inferior to his own.
He was in high glee as he stood talking to a group of his officers,
and rubbed his hands cheerfully.
“Melcareth hath surely delivered them unto us,” he said, “for this
foolish Scipio hath, by leaving his heavy-armed infantry behind,
played beautifully into my hands. And thou, my bloodthirsty
Monomachus, shalt soon have thy fill of slaying.”
“I care nought so that I slay but Scipio himself! So that ere I perish
myself, I dip my hand into the life-blood of the Roman Consul, I shall
die happy.”
Hannibal laughed, then gave the order to mount, but for the troops
still to remain within the fringe of the covert underneath the
spreading oaks.
While waiting thus, there suddenly arose a fleeting squall of driving
wind and rain, which tore the leaves with fury from the oaks, sending
them whirling in all directions, while the acorns, which abounded, fell
pattering and clattering in shoals upon the armour of the warriors.
The men, who were superstitious, knew not how to interpret the
augury; but their Chief was equal to the occasion.
“A glorious omen!” he proclaimed aloud, with a laugh. “Even as the
falling of these acorns, so shall fall the hordes of the advancing
enemy beneath the storm of our attack. See how the elephants are
calmly picking them up, and devouring them! so shall we also devour
the foe.”
The word was taken up and passed round, and now the men,
reassured and in high good humour, caught the falling acorns and
tossed them gleefully at one another. Meanwhile, as Hannibal had
said, the huge elephants calmly waiting and swinging their fore-legs,
picked up one by one with their trunks all the fallen acorns within
reach and ate them with apparent relish. The sight of these huge
painted and horrible beasts, thus picking up and eating the little
acorns, was in itself so ludicrous that all those who could see the
sight now roared with laughter like very children. For, as all know
who have been present at a battle, during the time, be it short or
long, that men are waiting anxiously for the signal that shall engage
them in mortal conflict, the nerves are on edge. ’Tis then often the
slightest and most trivial circumstance that will sway a whole host to
tears or laughter, to reckless courage, or to shameful retreat. On this
occasion the whole force was overcome with exuberance of spirits,
and thus hearty and boyish laughter rippled along the ranks.
It was, in sooth, a strange sight to see these gigantic animals so
peacefully enjoying themselves at such a moment with such trivial
food. For to look at them they seemed more like monsters ready to
devour men.
There were that day twenty-four of them present, twelve being
placed at intervals on each flank. Every elephant had its head and
trunk painted crimson with vermilion; the body was painted black,
with white stripes on the ribs, and the legs white to the knee and red
below. To each tusk, and all were tuskers, were fastened huge two-
edged swords; round the upper part of each of the fore-legs was
clasped a band set with terrible and glistening spikes; and, to crown
all, each elephant had round its trunk, near the lower extremity,
several heavy iron rings. With these they were trained to strike. An
ordinary elephant is to those with weak nerves a sufficiently
appalling sight. Judge, then, what would be the effect upon those,
who had never seen the beasts before, of the appearance of these
awful monsters. The elephants were that day in light marching order,
and therefore they carried no castles. Instead, on the back of each
was a light pad, on which sat, secured with ropes, and having also
rope foot-rests, eight archers, four on each side, plentifully supplied
with heavy arrows. Seated upon the neck of every beast was an
Indian driver, completely covered with light armour of chain mail, and
carrying, in addition to the goad with which he urged his beast, a
quiver full of short and heavy darts to hurl at the foe at close
quarters.
The order of battle was fully formed. The heavy Spanish cavalry
were in the centre; then on each side were twelve elephants; beyond
them again on each flank large bodies of the Numidian horse,
Maharbal commanding the right wing, and Chœras the left of the
Numidians.
Presently the last of the scouts came in, saying that the Romans
were now close at hand, and almost instantly their light-armed
javeline men and archers appeared, swarming on the plain. In rear of
them, at a short distance, could be seen the allied Gallic and Roman
cavalry advancing at a slow pace in a splendid line, with small
intervals between the separate troops. Hannibal allowed the light-
armed men to advance well out into the plain before he stirred from
the shelter of the oak trees. Thus he was well able to take stock of
the Roman numbers before engaging, while they could not possibly
form any idea of the numbers that he had at his own command.
Speedily sending an aide-de-camp to both Maharbal and Mago with
orders to remain for a short time longer concealed in the forest, he
now caused the “advance” and “the trot” to be sounded by his
trumpeter. In a steady line, with himself and Monomachus at the
head of the centre, the Iberian cavalry instantly emerged from their
shelter, the huge elephants lumbering along with ungainly tread at an
equal pace on each flank, and by command of their drivers, raising
their trunks on high and loudly trumpeting as they advanced. The
sudden appearance of this hitherto unseen army of cavalry, coupled
with the awesome sight of these terrible and frightful creatures, at
once struck terror into the heart of the light-armed Roman troops.
Seeing that certain death awaited them if they remained, they
instantly broke and fled, retreating through the intervals between the
troops of cavalry, and re-forming into companies in rear. But Scipio
himself, whose presence was denoted by a standard surmounted
with the Roman eagle, was with the main body. He had previously
exhorted them: saying that the Carthaginians were but curs with their
tails between their legs, who had fled before his cavalry on the banks
of the Rhone, and they believed him. Hence there was no flinching in
the ranks.
Hannibal now ordered his trumpeter to sound “the gallop,” and in a
second, with thundering footfall, the whole of his heavy cavalry,
armed with both spears and swords, the latter being attached to the
saddle, advanced with the speed of lightning.
Scipio also sounded “the charge,” but the retreat of his light-armed
footmen had somewhat delayed him. Therefore, when the two forces
met in shock of battle, the greater impetus being with the
Carthaginians, the Romans were at first borne backwards, hundreds
of men and horses being cast to the ground at the very first dreadful
onslaught. It was a terrible and an awful sight, that fearful rush and
the meeting between the two cavalry forces. And now it was every
man for himself, and both sides were equally determined. The
Romans, at first borne back, soon rallied; spears on both sides were
cast away as the horses fell in hundreds, and, foot to foot and hand
to hand, the whole plain was soon one seething and struggling mass
of murdering humanity. Now, however, the elephants came in upon
each flank of the Romans. Charging down, striking with their trunks,
destroying with their horrible scythe-like swords, frequently kneeling
upon and crushing their opponents, they carried all before them on
the flanks. All the time, the archers on their backs discharged from
close quarters the heavy-headed arrows from their bows. The effect
of this charge on the flanks, however, was but to consolidate the
Roman centre. Gallantly the Romans fought, never yielding a foot,
and many were the Iberians who fell before their dreadful valour
never to rise again. In the midst of all, mounted on a splendid bay
charger, was Publius Scipio. Separated from him at but a short
distance was Hannibal, above whom, borne by his ensign, waved
the flag of Carthage, the white horse upon a purple ground. The two
Generals could, for a while, plainly see each other, and, each
grinding his teeth and shaking his fist with rage at the other, they
tried to come to mortal combat. But it was in vain: the struggling,
thrusting, killing throng of men, swaying first this way and then that,
swept them apart, and in a short time the rush of battle had severed
them completely. Each had now to defend himself, and they saw one
another no more. At length the tide of battle seemed going to the
Romans. The Carthaginians slowly but gradually began to give
ground. A few of those in rear of the centre began to retire. The
wounded were pouring back in streams; riderless horses, many of
them with their entrails hanging out, were, while shrieking with agony
as only a horse can, wildly careering about the plain.
And now came the opportunity for which the young warrior
Maharbal had been waiting, and he took it. For fully an hour he had,
from his shelter in the oak wood, been watching the ebb and flow of
the battle, and terrible had been his anxiety and great his
impatience. But he had in himself the makings of a great general,
and he wisely refrained from interfering too soon. He sent a
messenger to summon Chœras to him, and imparted certain
instructions before sending him back to his own wing.
“Now, Chœras,” he said, “notice this and no jesting! Keep thou all
thy men carefully concealed until thou shalt see me issue alone from
the wood with this scarf waving on my sword. Then if I raise it slowly
and wave it once downwards, gallop with thy party and attack the
enemy’s right flank. If I raise it again slowly and wave it downwards
twice, do thou, avoiding the enemy’s flank, sweep completely round
and attack the rear. Never mind what mine own movements may be,
thou hast but to obey mine orders. Of one thing be careful, that is, if I
wave twice to slay all those footmen who retreated through the
enemy’s ranks before thou makest any attack upon the cavalry from
the rear. For they are Romans—the cavalry are chiefly Gauls, and I
will dispose of them myself. But do thou be careful to despatch the
footmen first; dost thou understand?”
“Ay, ay, Maharbal, I understand full well; but may I not start now?
The clash and conflict of the battle are getting too much for me; I
cannot, I fear, control either my feelings or my men much longer; I
long to be at them. I can only express my feelings in verse.

“A poet I, yet thirst for blood,


For Roman blood I call!
And I would storm as mountain flood,
E’en though as flood I fall.

“There, would that not do for Sosilus? He would be delighted to


parse, dissect, and destroy that little verse of mine, if he were only
here. But, by Melcareth, Maharbal! versifying apart, do let me get at
them soon; I can hardly keep my seat upon my horse for
impatience.”
“Well spoken, my gallant poet,” replied Maharbal, smiling; “thou
shalt, I wager, soon have plenty of mountaineering work to do in
climbing over the bodies of the slain, so be not impatient. But one
word. In attacking, look thou well to thy guard—you poets are so
incautious. After striking, ever raise thy wrist and cover thyself again
before another blow. Now farewell, and the gods be with thee; return
to thy troops.”
Very shortly after these orders had been given, Maharbal issued
from the wood and waved his sword with the attached handkerchief
twice. Like arrows Chœras and his men started from their cover, and
sweeping over the plain, avoiding the flanks, fell upon the rear of the
Romans. Dreadful then was the slaughter of the footmen who had
retired behind the Roman cavalry. At the same instant Maharbal
swept round the other flank, and, leaving the footmen alone, fell with
his men upon the rear of the Roman cavalry. The whole aspect of
the battle was changed in a moment. The whole Roman force, with
the exception of a band that rallied around Scipio, broke and fled,
and the greater part of the Carthaginian horse pursued them.
Maharbal, however, charged the serried foes around the Roman
leader, and with his terrible blows had soon cut a lane right through
their ranks up to the Commander himself, slaying the standard-
bearer and seizing the standard. Vainly did Scipio raise his already
tired arm to ward off the fearful blow that Maharbal now dealt. He
struck at the Consul with all his force, but his sword, glancing off the
Consul’s helmet, clove his shoulder. It cut clean through his armour
and half-severed his right arm. The shock threw Scipio from the
saddle, and his attendants now fled to a man. Maharbal was about to
dismount, and secure the person of the Consul as his prisoner, when
suddenly Monomachus burst upon the scene, wounded and
dismounted, brandishing a bared dagger which he stooped to strike
into the prostrate Roman’s throat. But first he plunged his hand in the
Roman’s blood.
“Hold! hold, Monomachus!” cried Maharbal. “Slay him not, he is
the General and a noble foe, and he is moreover my prisoner. Thou
shalt not slay him!”
“Curse thee!” cried the slaughterer furiously, “I tell thee I will slay
him, ay, and thee too if thou interfere,” and he knelt over the body of
Scipio to despatch him.
At that moment there came a shock. A young Roman officer with
five or six followers charged in upon the scene. With one spear-
thrust the leader transfixed the neck of the bloody Monomachus, and
then he instantly turned upon Maharbal.
“Ha! we meet again,” he cried, and rushed upon him. It was young
Scipio! But he was no match for Maharbal, who easily avoided the
spear-thrust, and then with a scornful laugh charged him in turn.
Striking him merely with his fist, he knocked him off his saddle.
“Ay, once again we meet and once again I spare thee, Roman!” he
cried, “and again I spare thee simply for thy valour’s sake.”
Then seeing himself surrounded by Romans, he struck down a
couple of them, and pressing his horse out of the throng, escaped,
bearing the Roman eagle with him.
CHAPTER VIII.
EUGENIA.

What a glorious career of successes it was that was inaugurated


with this cavalry victory near the Ticinus! Had not the blood-thirsty
monster Monomachus been slain by young Scipio at the very end of
the action, he might not only have been able to gratify his wish of
plunging his arm into the blood—although it proved not indeed the
life-blood—of the Roman General, but he might have bathed
repeatedly during the next twelve months in rivers of blood. Success
followed on success. Like rats that leave a sinking ship, the Gauls in
the Roman army deserted in their thousands and joined Hannibal;
the Boii and other Cis-Alpine tribes first sent him ambassadors and
hostages, and then themselves came over with all their fighting men,
and, after the battle of the Trebia, the whole of Cis-Alpine Gaul was
in the hands of the invading Carthaginians.
Now the battle of the Trebia was in this wise. After the cavalry
action just described, the wounded consul Publius Scipio fled with
his army first across the Po, and then, after considerable losses at
the crossings of all the rivers, to some high ground near the River
Trebia, where the now-dreaded Numidian Cavalry were unable to get
at him. Here he was joined in his camp by the other consul, Tiberius
Sempronius, who brought his army from Ariminum, the combined
Roman forces amounting to over forty thousand men. The two
consuls were utterly unable to decide upon a concerted plan of
action; the wounded General wishing for time to get well, and to
accustom his newly-raised legions to campaigning for a while before
risking a general action. Tiberius, however, was anxious to gain
some military glory for himself, apart from his brother consul; he was,
moreover, jealous of the possible successes of the succeeding
consuls, the time for whose election was near at hand. He was
therefore only too anxious to risk a combat, and was constantly
urging Scipio to allow him to take the combined consular forces
against Hannibal.
But now mention must be made of the youthful Eugenia and her
amours with Mago.
Mago, the younger brother of Hannibal, was immediately, on the
death of Hannibal Monomachus, appointed to the supreme
command of all the Carthaginian cavalry. After the affair of the
Ticinus, he was ordered on to the pursuit, and he ceased not to
harass Scipio, even when he had encamped under the walls of the
city of Placentia. Here, one day in a bold foray, he destroyed a
portion of the camp outside the town, and in the tent of one of the
generals of the Roman allies, which he looted, he captured an Italian
maiden named Eugenia, a girl about twenty years of age, and of
surpassing beauty. She was by birth an Etruscan, and by some is
reported to have been the daughter, by others the niece, of the
general in whose tent she had been found.
Terrified at first, she was soon fascinated by the charm of Mago,
who was a young man of most handsome appearance, and who
loaded her with gifts and gold, which were liberally supplied by
Hannibal for a purpose. After a very few days, as Mago was well
acquainted with the Latin tongue, he had so completely won her to
him that she would have readily allowed herself to be burned with
fire for his sake, or, at least, to have given her life for his. Not being a
Roman, but an Etruscan, she had no particular love for Rome; but
youthful and ardent, and anxious to prove to Mago her great love for
him, she readily fell in with his wishes. Accordingly, after Scipio had
encamped upon the heights on a spur of the Apennines, she one
day, being aided by Mago, pretended to escape to the Roman camp,
where she arrived with torn raiment, dishevelled hair, and an
appearance of the greatest misery. Her curses and invectives
against the Carthaginians, and the story which she related of her
imaginary wrongs, utterly and at once disarmed all suspicion. The
fair maid Eugenia was therefore received with open arms and made
a most welcome guest in the Roman camp, where everyone pitied
her for the terrible misfortunes she had endured. Thus, there was
nothing that transpired there that she did not know, and when she
had found out all that was going on, she one night took an
opportunity of escaping, and rejoined her lover Mago. To him she
revealed everything, and Hannibal was at once put in possession of
all the information he required. Having thus gained knowledge of all
the cabals that were going on in the Roman headquarters, Hannibal
knew how to act, and determined to precipitate a conflict before the
Roman infantry were really sufficiently seasoned.
This he easily managed.
To Mago, through whose pretty lover he had gained so much
useful information, he assigned the most arduous post—that of
hiding by night in ambush in a wet water-course, with a party of two
thousand chosen horse and foot, to appear at a seasonable
moment.
Then early in the morning, after his own men had breakfasted by
the camp fires, a party of light-armed troops were sent out to draw
out the Romans from their camp, which was easily accomplished,
the Romans, under Sempronius himself, thronging out, all unfed as
they were, and pursuing the apparently flying foe across the swollen
River Trebia. What followed is but history. After a bloody and
prolonged fight, Mago and his hidden troops suddenly appeared
from the water-course in rear of the Romans, and a frightful
slaughter ensued, the Numidians under Maharbal and Chœras
charging, as usual, in small groups, and advancing and retiring,
utterly disconcerting the enemy. The result of this terrible battle,
which was fought in a fearful snow-storm, was that out of forty
thousand troops engaged, only ten thousand survived and escaped
to Placentia. But the savage elephants, despite the snow, pursued
and slaughtered for hours and hours, until they could slay no more.
And this was the last fight in which the elephants were of any avail,
for the bitter cold weather which now set in soon killed them all but
one, and killed also thousands of the allied Carthaginians. As,
however, during the battle, the greatest losses among the
Carthaginians had taken place among the Celts or Gauls, Hannibal
still retained a large number of his original army of Libyans and
Iberians with whom to continue the campaign.
After this battle in the early spring, there were some terrible times,
during which, over and over again, it must be confessed, Maharbal
longed, but longed in vain, that he had taken the opportunity offered
him by the Chief of returning to New Carthage with the letter to his
beloved Elissa.
But there was no going back now, and, as he had cast in his lot
with his wonderful Chief, so was he compelled to go on. Therefore,
in the early spring, he crossed the Apennines, and for four
consecutive days and nights marched through the horrible swamps
and morasses between Lucca and Fæsulæ, where the only dry
places to be found at nights was upon the bodies of the dead
baggage animals. Here daily and nightly he strove to minister to
Hannibal, who, sorely afflicted with a terrible attack of ophthalmia,
which cost him an eye, nevertheless concealed his own agony, and
daily and hourly, riding upon the sole elephant that survived,
encouraged by his presence and example the troops under his
command.
Then came a short period of rest at Fæsulæ, during which the
Carthaginian troops contrived to regain a little of their lost vigour; but,
as many of the horses had lost their hoofs in the awful march
through the swamps, it fell to Maharbal and Mago, during this period,
to continually make sudden excursions where least expected, and
seize upon all the horses they could find.
It was now that Eugenia, the mistress of Mago, became most
useful. Carefully nurtured by Mago, and being accorded by Hannibal
all the comforts possible during this terrible march, she had not only
survived all its terrors, but was as strong and well, and, moreover, as
beautiful at its termination as she had been at its commencement.
She had, for concealment and convenience, been disguised as a
boy, and did not look while attired in male raiment more than about
fourteen years of age. She was strong and hardy, and being herself
an Etruscan, was well able to give every kind of information about
that country of Etruria wherein they now were. She was, however, no
longer entirely devoted to Mago, and this for a variety of reasons.
Perhaps the strongest of all was that she was a woman, and
consequently unable to continue ever in one mind. Therefore,
although she had deliberately given herself to Mago, under no
compulsion, but merely for love’s sake, she now had, so she
discovered, some religious scruples about her conduct. She talked to
Mago of the possible anger of the gods; at times, also, she became
cold to him, and reproached him with the sacrifices she had made for
his sake in leaving the Romans to wander about with him.
Mago for a time put up with these whims and vagaries, for he had
truly learned to love Eugenia; but there was one reason, at present
unexpressed, which daily made her more dissatisfied and
discontented.
Mago at length resented this treatment, as he wearied of her ill-
merited complaints. He now became, or at any rate pretended to be,
neglectful and cold in turn, and appeared to be entirely devoted to a
recently-captured slave girl.
Eugenia had, in reality, recently set her affections upon Maharbal,
and, as the days advanced, she fell madly in love with him.
Accordingly, during the rides through the districts of Etruria in search
of horses, Eugenia, now that Mago neglected her, attached herself
daily to the party led by Maharbal, and trying, indeed, did the young
warrior find these rides. For, faithful himself to Elissa, and quite as
faithful to his friend Mago, he was, nevertheless, violently moved by
the passion inspired by the beautiful Eugenia; and at length, so
overcome was he by her charms, her sweetness, and her very
evident admiration and love for himself, that he felt he must fall, and
yet he would not fall.
A crisis came. One day, when far away upon one of these raiding
expeditions, they had encamped for the mid-day meal in a wood,
and Eugenia, on some pretext, persuaded Maharbal to wander with
her through the most shady parts of the grove. Then suddenly losing
all reserve, she fell with tears upon his neck, and declared that she
loved him. Maharbal was, for a moment, overcome by the passion
with which he thrilled at the contact with her, and, for a fleeting
second, pressed her violently to his heart. But then, with a virtue
almost unknown in those times, he remembered his faith to his lover
—his almost wife—Elissa, and his faith to his friend—almost brother
—Mago, and a feeling of fury rose within him. He loosed the girl’s
arms from his neck with an angry movement.
“May the gods forgive me!” he cried, “but I know not what to do. I
desire thee intensely, thou beautiful Eugenia, and, indeed, I almost
love thee. But thou art not yet mine, and shalt never be mine, for I
will not be false as thou art thyself. And I should be doubly false
should I yield. Therefore, for thine own honour and mine own, thou
must die!”
Swiftly he drew his dagger, and, in spite of her one appealing cry,
slew her there on the spot, striking his dagger into her warm young
breast. Then covering his eyes with his hands, he hurried from the
dreadful place, horrified at what he had done, and yet feeling that the
omnipotent gods alone had guided the hand that struck the fatal
blow. And this, then, was the tragic end of poor Eugenia, who knew
not, when she had won a man’s love and given unto him all the most
precious gift of woman, how to remain faithful.
Maharbal was not the man to conceal an occurrence of this sort.
Upon his return to camp, he straightway repaired to Hannibal’s tent,
and begged for Mago to be summoned. After relating exactly what
had occurred, he bared his breast, and, presenting the hilt of his own
sword to Mago, said:
“Strike, Mago! for I have taken a life that should be dear to thee.
Strike! for I feel myself indeed worthy of death for having sent that
fresh young soul to Hades. But the gods are my judges, and if thou
wilt but strike, I shall soon appear before them to answer for my
conduct. Therefore, I say strike, and strike home!”
Mago did not strike. He burst into tears, and threw himself into
Maharbal’s arms.
“It is mine own fault,” he cried, “oh Maharbal, and deeply do I
grieve for the poor girl Eugenia. But far more deeply, friend, do I
grieve for thee, upon whom the gods have laid such a cruel burden,
as to compel thee, for thine honour’s sake and mine, to slay a
woman. For thou hadst no other alternative, save to become a base
villain.”
And again he wept, and Hannibal, who had in a fatherly way
himself loved the young maiden, being moved beyond words, silently
joined his brother and Maharbal in their tears.
There was much sadness that night in the camp of the generals,
and if virtue be, as the proverb saith, its own reward, then, for the
unhappy Maharbal, that reward was nought but misery.
After this, Maharbal became gloomy and morose. He quite lost the
youthful gaiety which had so often borne him up, and with which he
had in evil days encouraged others. For this affair preyed upon his
mind. In his dreams he would see the dead girl, by turns stretching
out her arms to him in imploring love, and then in imploring despair
as he was about to strike the fatal blow. He refused his food; indeed,
he was quite unable to eat. It was soon evident to all who knew him
that his mind was preying on his body, for daily he looked more wan
and ill, but he could not be comforted despite the efforts of his
friends to cheer him. It was in vain that the worthy and kind-hearted
Sosilus related for his benefit various real or imaginary histories, all
given with due chapter and verse, of some of the god-like heroes of
old, who, in similar cases, had behaved exactly like unto Maharbal
himself; these well-meant histories afforded him no comfort
whatever. He listened in silence. In vain also did Chœras force
himself to the most sprightly jests, or write the most comic and witty
verses, and read them to him aloud.
From sheer politeness’ sake Maharbal would force a smile, and
compliment his friend, but immediately afterwards would relapse into
moody silence as before.
Hannibal and Mago soon became quite alarmed, fearing that his
mind was becoming unhinged by his grief. But although they were as
brothers to him, and showered on him their brotherly love, nought
that they could do was of any avail. For Maharbal was haunted day
and night by the spirit of Eugenia. Throughout all his career, although
many women, in fact nearly all the women whom he met, as Melania
once wrote to Hannibal, fell in love with him, this secret and powerful
attraction which he had experienced for the lovely Eugenia was the
only passion for any other, save Elissa, that ever came to torture
Maharbal. But he kept his thoughts to himself and suffered in
silence, although ever haunted by the phantom of the slaughtered
girl.
Had it not been that the battle of Lake Thrasymene occurred just
about this time to divert the current of his thoughts, there is not the
slightest doubt but that Maharbal would shortly have died a raving
maniac. But Melcareth was merciful, and, by means of the distraction
of active warfare, withdrew the heavy hand which he had laid upon
the young man to try him; so that after the battle of Lake
Thrasymene Maharbal gradually, to the delight of his comrades,
recovered his health and spirits once more.
CHAPTER IX.
THRASYMENE.

The two new Consuls that year were Cnœus Servilius and
Flaminius, and they had between them some sixty thousand troops.
By so unexpectedly crossing the Apennines, and marching through
the awful marshes of the overflowed Arno, Hannibal had entirely
evaded Cnœus Servilius, who was left at Ariminum on the Adriatic,
but he now found Flaminius in Etruria, blocking the way to Rome
with a large force at a place called Arretium. Accordingly, Hannibal,
while still collecting horses and resting his troops, held one day a
council of war to consider the situation.
There were present, Mago, General Hanno, Maharbal, Hasdrubal
the pioneer, who had succeeded Monomachus as head of the
engineering department, Silenus the scribe, with writing materials
handy, and last, but by no means least upon this occasion, Sosilus,
who had, by this time, entirely undertaken the duties of head of the
intelligence department of the army, which were duties in which he
excelled. He had recently organised an excellent system of spies on
a large scale, and the scheme was working admirably. Nothing took
place in the Roman camp, or, for that matter, in Rome itself, without
his being informed; and of all occurrences of interest, Sosilus made
voluminous notes under alphabetical headings, with a view both to
present utility and to incorporation in his future history of the war.
Chœras returned from a horse-raiding expedition just as the
proceedings were commencing. He was not entitled by his rank in
the army to be present at a council of war, but Hannibal, who was
very partial to him on account of his ready humour, called him in as
he saw him marching, with a band of captured horses, past the open
door of the council tent. Hannibal was still suffering agonies from the
attack of ophthalmia, which had already cost him one eye.
Nevertheless, with his usual indomitable courage, as he sat at the
head of the council table, with a silk bandage over the diseased eye,
he looked as unconcerned and jovial as possible. No one could, for a

You might also like