Introduction to Neural and Cognitive

Modeling 3rd Edition Daniel S. Levine

Visit to download the full and correct content document:
https://textbookfull.com/product/introduction-to-neural-and-cognitive-modeling-3rd-edit
ion-daniel-s-levine/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Introduction to Linear Algebra for Science and

Engineering, 3rd Edition Daniel Norman

https://textbookfull.com/product/introduction-to-linear-algebra-
for-science-and-engineering-3rd-edition-daniel-norman/

Cognitive Informatics, Computer Modelling, and

Cognitive Science: Application to Neural Engineering,
Robotics, and Stem: Volume 2: Application to Neural
Engineering, Robotics, and STEM 1st Edition Ganesh R.
Sinha
https://textbookfull.com/product/cognitive-informatics-computer-
modelling-and-cognitive-science-application-to-neural-
engineering-robotics-and-stem-volume-2-application-to-neural-
engineering-robotics-and-stem-1st-edition-gan/

Cognitive Science An Introduction to the Science of the

Mind 3rd Ed 3rd Edition José Luis Bermúdez

https://textbookfull.com/product/cognitive-science-an-
introduction-to-the-science-of-the-mind-3rd-ed-3rd-edition-jose-
luis-bermudez/

Cognitive Phase Transitions in the Cerebral Cortex -

Enhancing the Neuron Doctrine by Modeling Neural Fields
1st Edition Robert Kozma

https://textbookfull.com/product/cognitive-phase-transitions-in-
the-cerebral-cortex-enhancing-the-neuron-doctrine-by-modeling-
neural-fields-1st-edition-robert-kozma/
Brain Evolution by Design From Neural Origin to
Cognitive Architecture 1st Edition Shuichi Shigeno

https://textbookfull.com/product/brain-evolution-by-design-from-
neural-origin-to-cognitive-architecture-1st-edition-shuichi-
shigeno/

Introduction to Mathematical Modeling 1st Edition Mayer

Humi

https://textbookfull.com/product/introduction-to-mathematical-
modeling-1st-edition-mayer-humi/

An Introduction to Applied Cognitive Psychology 2nd

Edition David Groome

https://textbookfull.com/product/an-introduction-to-applied-
cognitive-psychology-2nd-edition-david-groome/

An Introduction to Applied Cognitive Psychology 2nd

Edition David Groome

https://textbookfull.com/product/an-introduction-to-applied-
cognitive-psychology-2nd-edition-david-groome-2/

Introduction to Linear Algebra for Science and

Engineering Daniel Norman

https://textbookfull.com/product/introduction-to-linear-algebra-
for-science-and-engineering-daniel-norman/
“Levine’s book achieves an impressive synthesis of historical trends and
current research results in both biological and artificial neural network research.
This synthesis clarifies that the currently popular Deep Learning is just one
contribution to this burgeoning field, and one that does not incorporate many
of the most powerful properties of biological learning. Levine’s book provides
an accessible introduction to many of these properties, while also reviewing
important properties of neural models of vision and visual attention, sequence
learning and performance, executive function, and decision-making, among its
other expository accomplishments.”
—Stephen Grossberg, Boston University, USA
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
INTRODUCTION TO NEURAL
AND COGNITIVE MODELING

Providing a thorough introduction to the field of neural networks, this edition

concentrates on networks for modeling brain processes involved in cognitive
and behavioral functions. Part I explores the philosophy of modeling and the
field’s history, starting from the mid-1940s, and then discusses past models of
associative learning and of short-term memory that provide building blocks for
more complex recent models. Part II of the book reviews recent experimental
findings in cognitive neuroscience and discusses models of conditioning,
categorization, category learning, vision, visual attention, sequence learning,
behavioral control, decision-making, reasoning, and creativity. The book
presents these models both as abstract ideas and through examples and concrete
data for specific brain regions.
The book includes two appendices to help ground the reader: one reviewing
the mathematics used in network modeling, and a second reviewing basic
neuroscience at both the neuron and brain region level. The book also includes
equations, practice exercises, and thought experiments.

Daniel S. Levine is Professor of Psychology at the University of Texas at

Arlington. He is a fellow and former president of the International Neural
Network Society. His research involves computational modeling of brain
processes in decision-making and cognitive-emotional interactions.
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
 INTRODUCTION
TO NEURAL
AND COGNITIVE
MODELING
Third edition

Daniel S. Levine

Routledge
ROUTLEDGE

Taylor & Francis Group

NEW YORK AND LONDON

Third edition published 2019
by Routledge
52 Vanderbilt Avenue, New York, NY 10017
and by Routledge
2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
Routledge is an imprint of the Taylor & Francis Group, an informa
business
© 2019 Taylor & Francis
The right of Daniel S. Levine to be identified as author of this work
has been asserted by him in accordance with sections 77 and 78 of the
Copyright, Designs and Patents Act 1988.
All rights reserved. No part of this book may be reprinted or reproduced
or utilised in any form or by any electronic, mechanical, or other means,
now known or hereafter invented, including photocopying and recording,
or in any information storage or retrieval system, without permission in
writing from the publishers.
Trademark notice: Product or corporate names may be trademarks or
registered trademarks, and are used only for identification and explanation
without intent to infringe.
First edition published by Psychology Press 1991
Second edition published by Routledge 2000
Library of Congress Cataloging-in-Publication Data
Names: Levine, Daniel S., author.
Title: Introduction to neural and cognitive modeling / Daniel S. Levine.
Description: Third edition. | New York, NY : Routledge, 2019. | Includes
bibliographical references and index.
Identifiers: LCCN 2018018582| ISBN 9781848726475 (hbk) | ISBN
9781848726482 (pbk) | ISBN 9780429448805 (ebk)
Subjects: LCSH: Neural networks (Neurobiology) | Neural networks
(Computer science)
Classification: LCC QP363.3 .L48 2019 | DDC 612.8/2–dc23
LC record available at https://lccn.loc.gov/2018018582

ISBN: 978-1-84872-647-5 (hbk)

ISBN: 978-1-84872-648-2 (pbk)
ISBN: 978-0-429-44880-5 (ebk)

Typeset in Times New Roman and Adobe Garamond

by Florence Production Ltd, Stoodleigh, Devon, UK
CONTENTS

Flow Chart of the Book ix

Preface to the Third Edition xi
Acknowledgments xiv
Notation Used in the Book xv

PART I
Foundations of Neural Network Theory 1

1 Neural Networks for Modeling Behavior 3

2 Historical Outline 13

3 Associative Learning and Synaptic Plasticity 40

4 Competition, Lateral Inhibition, and Short-Term

Memory 89

PART II
Computational Cognitive Neuroscience 137

5 Progress in Cognitive Neuroscience 139

6 Models of Conditioning and Reinforcement

Learning 167
viii Contents

7 Models of Coding, Categorization, and Unsupervised

Learning 216

8 Models of Supervised Pattern and Category

Learning 250

9 Models of Complex Mental Functions 286

Appendices
Appendix 1: Mathematical Techniques for Neural Networks 343
Appendix 2: Basic Facts of Neurobiology 367
References 381
Author Index 438
Subject Index 451
FLOW CHART OF THE BOOK

Here is the structure of the book. Chapters depend in part on previous chapters
that have arrows pointing to them.

Ch. 1:
Modeling philosophy

Exercises Ch. 2
in these History
chapters

Ch. 3: Ch. 4: Ch. 5:

Learning and Lateral inhibition and Cognitive neuroscience
long-term memory short-term memory

Ch. 6: Ch. 7:
Conditioning and Coding and Ch. 8:
reinforcement learning categorization Category learning

Ch. 9:
Complex cognitive
functions
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
PREFACE TO THE
THIRD EDITION

How do you eat an elephant? One bite at a time.

Beverly Johnson, after climbing the rock face of
El Capitan, Yosemite National Park

Since the second edition of Introduction to Neural and Cognitive Modeling

appeared in 2000, the growth of neuroscience, and of cognitive neuroscience
in particular, has been explosive. For example, I did a search of the library
database PsycInfo under the keywords “fMRI or functional magnetic resonance
imaging,” which is the most frequently used experimental method for studying
the relationship between brain physiology and human cognitive function. This
search yielded only 929 results between the years 1943, the date of the first
seminal article in neural networks, by McCulloch and Pitts, and 2000. Yet, the
same search yielded 31,107 results between 2001 and 2017.
The growth in computational models of neural and cognitive processes has
not been quite as spectacular as the growth in experimental studies but is still
impressive. One result has been the formation of distinct niches within the field
of neural networks. The changes in the field over 17 years were sufficient to
warrant a new edition of this book. The first and second editions were intended
for the varied audience of the field, those primarily interested in using neural
networks for intelligent computing and engineering applications as well as
those primarily interested in understanding brain processes. By contrast, this
edition primarily targets those interested in understanding brain processes.
However, most of the more general foundational material from the earlier
editions remains with updating in Chapters 2–4, so the neural engineer or
computer scientist can still find the book a valuable source.
xii Preface

This book is intended both as a textbook for a graduate or advanced

undergraduate course in neural networks and as a general introduction to the
field. Its focus is on the relationships between neural structure and cognitive
function. Details of neuronal biophysics are included only in so far as they
illuminate the cognitive and behavioral implications of neural structures: other
books (e.g., Dayan & Abbott, 2005) present more computational and math-
ematical models at the neuronal level.
The cognitive functions discussed herein include learning, perception, atten-
tion, memory, pattern recognition, categorization, executive function, decision-
making, and inference. The neural networks modeling these functions
sometimes incorporate organizing principles such as competition, association,
opponent processing, and error correction, principles that can be suggested
either by the exigencies of modeling psychological data or by the description
of known neuroanatomical structures. These principles are developed in early
chapters and appear throughout the book.
In keeping with the goal of accessibility to a varied audience, technical
prerequisites in any one discipline are kept to a minimum. Recent advances in
computing make the field accessible to many more people than before. For
those needing additional background in mathematics or in neurobiology,
appendices in those fields are included; the appendices also list sources for
more detailed coverage.
A word should be said here about equations. The last section of each of
Chapters 2–4 and 6–9 includes differential or difference equations for some
of the networks discussed in that chapter, so that the reader can gain hands-on
experience in computer simulation of the networks. On first reading, the
student without mathematical background can skip these equations and follow
the development of networks by means of the figures. On second reading, the
same student can turn to Appendix 1 for explanations of how equations reflect
the qualitative relationships in networks, and simple algorithms for simulating
such equations. The first half of Appendix 1 is written so as not to require
previous background; notions needed from calculus are redefined and
motivated in the context of neural network applications.
The book is divided into two sections. The first section, Chapters 1–4, is
more foundational. Chapter 1 gives the underlying philosophy of the book and
Chapter 2 gives the historical background for its main ideas. The next two
chapters describe models of common “building blocks” for network models
of cognitive functions: associative learning in Chapter 3 and lateral inhibition
in Chapter 4.
The second section, Chapters 5–9, discusses models of cognitive functions,
building on the work of the previous chapters. Chapter 5 reviews recent
advances in experimental cognitive neuroscience. The networks discussed
in Chapter 6–9 simulate some of the data discussed in Chapter 5 and
often incorporate simpler network structures developed in Chapters 3 and 4.
 Preface xiii

Chapter 6 on conditioning and reinforcement learning, Chapter 7 on unsuper-

vised categorization, and Chapter 8 on supervised categorization include much
of the material of Chapters 5 and 6 of the second edition but add a considerable
amount on models developed since 2000. Chapter 9 is entirely new, covering
models of a variety of complex cognitive functions: vision and visual attention,
sequence learning and performance, executive function, decision-making,
reasoning, and creativity.
Chapters 2–4 and 6–9 contain homework exercises. Some exercises are
thought experiments and others are computer simulations of various neural
network models. The exercises herein are a small sampling of the possible ques-
tions that can be asked about the material discussed in the book, and the
instructor is encouraged to supplement them as he or she sees fit. Many of the
thought questions asked here do not have right and wrong answers, only a
variety of better and worse answers. The reader should approach the field with
at least as much intellectual flexibility and curiosity as possessed by the
systems we model. A solution manual has been provided for instructors, with
links from the book’s website. The manual can provide hints for exercises,
particularly for the harder ones in Chapters 6 through 9.
The flowchart in the front of the book illustrates how the understanding
of each chapter depends on previous chapters. After the introduction in
Chapter 1 and the historical account in Chapter 2, Chapters 3 through 9
(excluding Chapter 5, which is a review of experimental data) reflect a
hierarchy in models from simpler to more complex neural and cognitive
processes. Chapters 2–4, which are foundational for the more complex models
in Chapters 6–8, are the chapters least changed from the second edition,
though they are updated based on recent scientific advances.
ACKNOWLEDGMENTS

The psychology editor at Taylor & Francis, Paul Dukes, encouraged this book
project from the very beginning. When Taylor & Francis became the parent
company of Lawrence Erlbaum Associates in 2007, Paul noted that previous
editions had been among LEA’s most successful books until about 2005, when
the field had grown large enough to spawn competing books from other pub-
lishers. He enlisted me to update my book for a twenty-first-century audience.
He has worked hard with other Taylor & Francis staff members to obtain
helpful reviews both before and after my signing of the book contract, and to
provide infrastructure both for publicizing the book and making it suitable for
classroom use. Claudia Bona-Cohen, the editorial assistant at Taylor & Francis,
saw the book through efficiently at the production stage.
Several leading neural network researchers gave feedback both on the book
proposal and on earlier drafts of parts of the text that markedly improved its
content. These researchers included Jeffrey Bowers, Joshua Brown, Daniel
Bullock, Gail Carpenter, Colin Davis, Stephen Grossberg, Sebastian Hélie,
Ali Minai, and Paul Werbos.
Bakur AlQaudi, who is a faculty member at Yanbu Industrial College (Saudi
Arabia) with a PhD in electrical engineering from the University of Texas at
Arlington (UTA), tested all the computational exercises in MATLAB on a Lenovo
P40 Yoga Laptop (ThinkPad) Windows 10 (64-bit). He worked with me to “vet”
the exercises and helped me edit their text to make them as suitable as possible
for the level of the graduate students who are likely to run them in class work.
In addition, the UTA Department of Psychology provided me with comfortable
office, laboratory, and computing space that expedited the book’s production.
My student Amandeep Dhaliwal helped with the quality of the figures.
Finally, my wife, Lorraine Levine, lived patiently with the highs and the
rebound lows associated with the book’s composition. She combined an
appreciation of the project’s value with a warm sense of humor that kept me
on course but helped me avoid the perils of overly grim determination.
NOTATION USED IN THE BOOK

In figures
An arrow (→) denotes an excitatory connection
A filled circle (䊉) denotes an inhibitory connection
A filled semicircle (◗) denotes a modifiable connection

In exercises
A single asterisk (*) denotes a computer simulation exercise
A double asterisk (**) denotes a mathematical problem
An open circle (䊊) denotes an open-ended conceptual exercise
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
PART I

Foundations of Neural
Network Theory
Taylor & Francis
Taylor & Francis Group
http://taylorandfrancis.com
1
NEURAL NETWORKS FOR
MODELING BEHAVIOR

My mind to me a kingdom is,

Such perfect joy therein I find
As far exceeds all earthly bliss
That God or nature hath assigned.
Edward Dyer

What is mind? No matter. What is matter? Never mind.

Thomas Hewitt Key (epigram in Punch)

1.1. What Are Neural Networks?

The birth of the current field of neural network modeling can be traced to the
cybernetic revolution of the 1940s and 1950s. At that time, scientists across
many disciplines became excited about the notion that neurons are digital
on–off switches (either firing or not firing), and thus that brains and the newly
emerging digital computers had similar structural organizations (Wiener,
1948). Before long, biologists discovered that the digital metaphor was an
inadequate one for capturing what was known about neurobiology and
psychology. It was found necessary to understand the graded (or continuous,
or analog, or grayscale) as well as the all-or-none (or digital) components of
neuron responses (see, e.g., Thompson, 1967, Ch. 1).
Yet, if one seeks to understand cognitive and behavioral functioning, it is
not enough simply to understand neurons and their assemblies. It is just as
important to understand architectural principles guiding network connection,
principles that enable key psychological processes to occur and thereby allow
the assembly of neurons to instantiate significant classes of cognitive and
behavioral data.
4 Foundations of Neural Network Theory

How do we develop theories of brain–behavior relationships? Elsewhere

(Levine, 1999) I have described the theory building process as analogous to
the building of the tunnel across the channel between England and France. The
theorist starts at both the behavioral and the neural ends, and then builds
“tunnels” of connection from each end toward the other (Figure 1.1).
Neural network models have gradually evolved in the direction of greater
correspondence with brain structure and function, from the 1980s to the
present. Techniques such as taking magnetic resonance scans of whole brain
regions and recording with electrodes from up to 50 neurons at once have made
neurophysiology more amenable to quantification. At the same time, advances
in computing (personal computers, supercomputers, and interfaces with
recording devices) have made simulation of biological data easier and more
practical. These technical developments are enabling theorists building on
earlier, more abstract cognitive models to create theories with real explanatory
and predictive power. The review article of Ashby and Hélie (2011) calls the
newer models that incorporate sophisticated neuroscience data by the name
of computational cognitive neuroscience models, to distinguish them from
more loosely brain-related cognitive models that are called by the older name
of connectionist models (e.g., Feldman & Ballard, 1982; Rumelhart &
McClelland, 1986a).
The development of neural network or connectionist theories has been
spurred not only by interest in how the brain works but equally by the interest
in potential engineering applications of intelligent computing. The International
Neural Network Society’s several hundred members, and several annual
meetings including the International Joint Conferences on Neural Networks,
span research in both the neuroscientific and engineering applications of the
field, and some studies that overlap neuroscience and engineering.

NEUROSCIENCE

NEURAL NETWORKS

PSYCHOLOGY

FIGURE 1.1 “Tunnel” metaphor for integrating neuroscience and psychology;

see the text.
 Neural Networks for Modeling Behavior 5

Designers of machines for performing cognitive functions generally are

interested in learning, without slavishly imitating, how the brain performs those
functions. Consequently, such machines have often been built like simulated
brain regions, with nodes corresponding to neurons or neuron populations, and
connections between the nodes; at times, their designers have borrowed ideas
from recent experimental results on the brain’s analog responses. The industrial
applications of connectionist theory are often called artificial neural networks
or ANNs (e.g., Hecht-Nielsen, 1986).
The much older term neural networks is usually considered to encompass
both theoretical and applied models that provide mechanistic bases for cogni-
tive functions. The functional units in neural networks have alternatively been
called “nodes,” “units,” “neurons,” and “populations.” The first two terms are
most commonly used, in this book and elsewhere, because they do not commit
the user to an assumption that units correspond to either single or multiple
neurons. Both the earlier connectionist networks and the networks in compu-
tational cognitive neuroscience models include nodes, connections, and
equations describing the interactions of node activities and connection
strengths.

1.2. What Are Some Principles of Neural Network Theory?

While the neural network field has experienced upward and downward surges
in popularity, these surges conceal the field’s maturity. The early history of
neural network models, summarized in the review article of Levine (1983) and
discussed further in Chapter 2, shows that most modern ideas in network design
have much earlier antecedents. For example, the popular distinction between
input, hidden, and output units (Rumelhart & McClelland, 1986a), which led
to back propagation and then to deep learning, owes much to the early work
of Rosenblatt (1962) on networks with sensory, associative, and response units.
Rosenblatt, in turn, combined extensions of the linear threshold law of
McCulloch and Pitts (1943) with various learning laws, some of them inspired
by the work of Hebb (1949). A precise mathematical formulation for many
examples of this type of network was given in the dissertation of Paul Werbos
(1974; see Werbos, 1993, for a more accessible version). Moreover, several
researchers who published seminal neural network articles in the late 1960s or
early 1970s, including Michael Arbib, Jack Cowan, Walter Freeman, Stephen
Grossberg, and Teuvo Kohonen, either remain active in the field today or did
so through the early twenty-first century.
As the neural network literature grows, it is essential to find criteria for
making distinctions among competing models. Meeter, Jehee, and Murre
(2007) observed that fitting both behavioral and neural data is one of the
important criteria but not the only one. Ideally, these authors note, in addition
to fitting existing data and making nontrivial predictions, a model should be
6 Foundations of Neural Network Theory

based on assumptions that make sense from a biological and/or behavioral

viewpoint. Also, it is desirable to have several interconnected and mutually
consistent models for the same process at different levels of representation, and
such hierarchies of models are particularly useful when some of the relevant
data are unavailable. A similar point was made by the neural network pioneer
Stephen Grossberg (Grossberg, 2006): “One works with large amounts of data
because otherwise too many seemingly plausible hypotheses cannot be ruled
out.” The purpose of interconnected models, Grossberg goes on to say, is to
capture the fact that “the brain . . . can be successfully understood as an organ
that is designed to achieve successful autonomous adaptation to a changing
world.”
In order to understand the behaviors leading to autonomous adaptation, we
need first to understand subsystems that perform parts of the tasks we want
the system to perform. Subsystem identification is dramatized by the parable
of the watchmakers (Simon, 1969, pp. 90–93). One watchmaker tries to fashion
a whole watch simply by fitting parts together. Another watchmaker instead
starts with the same parts but puts some of them together into subsystems. Not
until the subsystems are working does he then join them into a watch. The
second watchmaker prospers, while the first has to start all over again whenever
he is interrupted.
Likewise, any major cognitive process needs to be analyzed into sub-
processes. The understanding of the subprocesses then suggests principles that
can also be used in models of a wide variety of other processes. Agreement
on the principles of how to organize neural networks into subnetworks is far
from universal. In part, this reflects the incredible variety of ways in which
any given cognitive function is organized across many biological species
(indeed, phyla) and individuals within these species. In part, it reflects the
variety of theoretical perspectives brought to bear on these problems. Yet, it
is possible to see through this diversity a few subnetwork organizing themes
that are common to network models arising from many sources.
Levine (1989) developed a thought experiment that illustrates two generic
examples of analyzing a mental process into subprocesses. Both examples
relate to main topics of later chapters in this book.
The first example, discussed more fully in Chapter 7, is the categorization
of sensory patterns. Categorization is necessary for an organism or network to
make sense of its environment and make predictions about novel stimuli. For
definiteness, say that the network is processing hand-printed characters and
attempting to match each one to a known letter of the alphabet. The simplest
(although not the only) way to understand this process is to include in the
network some units (“feature nodes”) that respond to presence or absence
of writing at particular locations, and other units (“category nodes”) that
respond to patterns of feature node activation representing particular letters (see
Figure 1.2).
 Neural Networks for Modeling Behavior 7

To carry our thought experiment further, how are feature nodes and category
nodes likely to be connected? If these connections were hard-wired, the
network could not adjust to significant changes in its patterns of activation –
for example, receiving inputs of Japanese or Russian instead of Roman
characters. Hence, it is usually desired that the strength of the connection
between any specific feature node and any specific category node be allowed
to change over time, as a result of repeated activation of the connection. Such
a change is often accomplished by a principle called associative learning;
associative learning laws in neural networks are the main topic of Chapter 3.
Hence, associative learning has been one of the common subnetwork
organizing principles in neural networks from the early 1960s to the present.
There are other important subnetwork principles: many categorization models
particularly rely on a principle called competition. To motivate the idea of
neural competition, consider a sloppily written letter that is ambiguous (, say;
it could either be an E or an F. The network needs a method for deciding which
of the two letters is the more likely one. The feature nodes are activated, to
varying degrees, by the incoming letter, and in turn activate their category
nodes via the internode connections. As shown in Figure 1.2, there is mutual
inhibition between the category nodes. Thus, if both the “E” and “F” nodes
are activated but the activation of the “E” node is greater than that of the “F”
node, the system makes a decision that the letter is an E. Inhibition between
nodes at the same level of the network (in this case, the category level) is often

E F F2 (CATEGORIES)

wij

Signal to jth F2 node is

sum of xiwij

F1 (FEATURES)
xi

INPUT

FIGURE 1.2 Generic categorization network combining associative learning and

competition.
Source: Modified from Levine, 1989, with permission of Miller Freeman Publishers.
8 Foundations of Neural Network Theory

regarded as competition between the different cognitive entities coded by those

nodes; competition laws in neural networks are the main topic of Chapter 4.
Associative and competitive principles are likely to combine in many other
cognitive processes besides categorization. Our second example is attentional
modulation of conditioning, which is discussed more fully in Chapter 6.
Suppose that a neutral stimulus such as the sound of a bell has become
associated with food. Then how does the animal learn to pay more attention
to the bell than to other neutral stimuli in its environment?
Again, the psychological results suggest the neural network principle of
competition. If different nodes develop representations of sensory stimuli, then
the bell node should somehow “win” a competition with other sensory nodes
for storage in short-term memory.
But how does the competition among sensory representations become
biased in favor of the bell? One plausible answer suggests the neural network
principle of associative learning. Other things being equal, the bell node tends
to be activated because of prior association between the bell and food, or, more
abstractly, between the bell and satisfaction of the hunger drive (see Figure
1.3). Hence, if there is another node representing the primary reinforcer of food
or the hunger drive itself, repeated pairing of the bell and food tends to activate
the bell representation if the animal is hungry. Strengthening pathways based
on pairing of stimuli is an example of associative learning.
The thought experiment described here illustrates the potential of network
theories to unify disparate areas of psychology. In the early to middle twentieth
century, the behaviorist school was ascendant in academic psychology, with
its focus on outwardly measurable performance and its distrust of any
consideration of internal states, beliefs, or self-reports. But Behaviorism was
challenged in the 1960s and 1970s with the rise of cognitive psychology,
whereby internal states became of paramount importance (Reisberg, 2016,
Chapter 1). The thought experiment tells us that classical conditioning, a major
concern of the behaviorists, and categorization, which falls into the cognitive
realm, can be understood using different combinations of the same theoretical
“building blocks.”
In the course of this exposition, a few other major subnetwork organizing
principles emerge besides association and competition. One of these principles
is opponent processing. This means that neural architectures are organized into
pairs of pathways with opposite significance (e.g., light and dark, or reward
and punishment) in such a manner that a sudden decrease in activity of one
pathway transiently activates the opposing pathway. Like association and
competition, opponent processing plays a role in some of the conditioning
models discussed in Chapter 6 and some of the categorization models discussed
in Chapters 7 and 8.
All these principles have been suggested by a heterogeneous database that
is partly physiological and partly psychological. In some cases, the theory was
 Neural Networks for Modeling Behavior 9

FOOD

LIGHT TOUCH

BELL

FIGURE 1.3 Generic selective attention network combining associative learning and
competition. Filled semicircles represent learnable connections. The larger semicircle
on the path from food to bell represents a connection strengthened by learning, based
on pairing of the bell with food. This biases competition among light, touch, and bell
nodes in the bell’s favor.
Source: Modified from Levine, 1989, with permission of Miller Freeman Publishers.

first suggested by psychological results and later, at least qualitatively,

supported neurophysiologically. An example is associative learning theory.
Hebb (1949), inspired by Pavlovian conditioning data, proposed that if one
neuron connects to another via a synapse, and the firing of the first neuron is
repeatedly followed by firing of the second, then the synapse should become
strengthened. As discussed in the early part of Chapter 3, examples of neuronal
behavior consistent with this general hypothesis (though not in the precise form
that Hebb had proposed) were discovered first in invertebrates (e.g., Kandel
& Tauc, 1965) and later in vertebrates (e.g., Bliss & Lømo, 1973). More recent
experimental studies, moreover, have partially supported mathematical
variations of Hebb’s learning law proposed by modelers in the 1960s and
1970s.
This book talks rather freely of “representation” by network nodes of
features, categories, or other concepts. There is substantial debate among
philosophers of science as to what a true representation is, either in the brain
or a model (see Bechtel, Mandik, Mundale, & Stufflebeam, 2001; Bechtel,
2008). Bechtel and his colleagues note that the different disciplinary
perspectives of cognitive scientists and neuroscientists have led to different
concepts of representation. They also note that a school of modelers has arisen
that study large networks by means of the mathematical theory of dynamical
systems and have concluded that considering representations of part of a
system detracts from considerations of the whole system (see, e.g., Freeman
& Skarda, 1990).
This book presents models originating from a variety of perspectives and
disciplines, so does not take a position in the philosophical debates about
10 Foundations of Neural Network Theory

representation. I believe, though, that the “tunnel-building” approach described

here helps to unify the varying disciplinary perspectives from neuroscience,
psychology, and cognitive science. Also, the models described in this book
make extensive use of dynamical systems, but in some cases the modelers still
find it a useful simplification to talk about some part of their network being a
representation of some concept.

1.3. Problems, not Trends

This book’s organization is unlike that of most other introductory books on
neural networks. In keeping with its emphasis on modeling the connections
between neuroscience and psychology, the primary organization of the chapters
is by the behavioral and cognitive problems that the neural models were
designed to address. Organization by different trends, approaches, and schools
of modeling is secondary and shows up mainly in the headings of sections and
subsections.
For this reason, such recent trends as deep learning and Bayesian learning
are not treated in separate chapters but as aspects of models that reproduce
specific behavioral and neural capabilities or datasets. Also, because the book
more than its previous editions emphasizes neuroscience and psychology
applications, engineering and machine learning applications are discussed
only insofar as they illuminate the modeling of behavioral and neural
phenomena.
Modelers of psychological phenomena frequently make a distinction
between tokens (specific instantiations of models) and types (generic
instantiations of models) (see, e.g., Wagenmakers, Ratcliff, Gomez, & Iverson,
2004). There has been considerable work on model mimicry, the ability of
one model to account for data generated by a competing model, at the level
of tokens, that is, different parametrizations of a model with a given set of
assumptions. Much of that work is beyond the scope of this book, because the
book concentrates more on types than on tokens. The primary focus is on
discerning the right set of model interactions, before trying to optimize the
numerical strength of those interactions.

1.4. Methodological Considerations

Subnetworks incorporating principles such as associative learning, competition,
and others to emerge in later chapters can be thought of as part of the neural
network modeler’s “tool kit.” So can larger networks that have been developed
by major researchers in the field. Hence, when neural networks are used in
brain modeling, new discoveries on the brain or on biological cognition may
force modifications of a theory rather than abandonment of the entire structure.
 Neural Networks for Modeling Behavior 11

Also, a general qualitative theory may be widely applicable but the detailed
instantiation of that theory may vary enormously between individuals and
species.
The ability to break down many complex networks into functionally
significant subnetworks does not necessarily mean that these subnetworks
should be treated as independent modules, as in early artificial intelligence
models. The function and even structure of each of the smaller networks can
be strongly affected by its interactions with the other smaller networks. For
example, in many models of vision the systems for processing shape, color,
and motion all depend in part on interactions with the others, as is seen in
Chapters 4 and 9.
The models discussed in this book vary in how faithful they are to known
neuroanatomy and neurophysiology. The models discussed in Chapters 6–9
tend to be more neurobiologically correct or detailed than those discussed in
Chapters 3 and 4, yet many of the models in Chapter 6–9 include parts that
are modifications of subprocess models from the earlier chapters. The units,
or nodes, in neural networks are often regarded as populations of neurons that
are unified in some functional sense (e.g., those cells responsive to light in a
particular part of the visual field, or to the hunger drive). Analogies of these
units to averaged cell types in particular brain areas are quite close at times,
more remote at other times. Hence, the book title uses the broad term neural
and cognitive modeling to encompass models with different degrees of fidelity
to real brain structure. The boundary between more and less “brain-faithful”
networks is a fluid one. In fact, as the above example of associative learning
shows, models that are more “cognitive” than “neural” sometimes lead to
“neural” predictions that are later supported by data.
As neural networks have become popular, it is often asked how much neural
network theory has really accomplished in its efforts to explain neurobiological
data. The problems that neural network theory addresses are complex, and no
single model has yet “cracked” the problem of categorization, or memory, or
decision-making, or emotion. What has happened, instead, is that neural
network theory has been part of a slow, steady increase in overall understanding
of brain and cognitive functions. Hence, I believe that researchers in the field
are justified in saying, in the words of the neural network pioneer Warren
McCulloch, “Don’t bite my finger, look where I am pointing” (McCulloch,
1965, Page xx).
The technological applications of neural networks are somewhat ahead of
the biological modeling applications. Artificial neural networks have achieved
considerable success in diverse areas, such as robotics, speech recognition and
synthesis; decisions on whether to grant mortgage insurance; classification of
some types of medical information, and classification of radar patterns. Yet in
the last ten years the neurobiological modeling has begun to catch up to the
technology.
12 Foundations of Neural Network Theory

Neural network models have led to experimental predictions, most notably

in vision, motor control, and classical conditioning but to a lesser extent in more
opaque areas such as decision-making and the effects of specific mental
illnesses. Since the early 1990s, an increasing number of neuroscience and
psychology laboratories are including computational modeling as part of their
operations and actively testing some of the predictions of their models.
The scientific approach to knowledge, broadly speaking, argues that mental
phenomena should have some mechanistic basis that will eventually be
understandable by human beings. As Wiener (1954, p. 263) said, the faith of
scientists is that nature (including mind) is governed by ordered laws, not by
the capricious decrees of a tyrant like Lewis Carroll’s Red Queen. Neural
network modeling provides the best methodology now available for building
mechanistic theories of mental and psychological functions.
Since all current neural models are subject to modification, this book is
written to give the student or other reader hands-on experience in thinking
about, simulating, and ultimately designing neural networks. It begins in
Chapter 2 with a historical overview of major trends and the roots of current
key ideas. Chapters 3 and 4 review the most important established models of
associative learning and competition, respectively. Chapter 5 reviews some
of the most significant recent findings in cognitive neuroscience. Chapters 6–9,
building in part on some of the network structures introduced in Chapters 3–4,
develop recent models (many of which Ashby and Hélie, 2011, would call CCN
models) that reproduce some of those cognitive neuroscience results.
Another random document with
no related content on Scribd:
 “I would like to have a drink of good, cold water. I am a deal of
trouble, am I not?”
“If I were sick I would like some one to wait upon me,” I said, and
ran down stairs.—Ann was at the end of the garden, picking berries;
so I drew the water myself; and as I brought the bucket up to the
curb, the woman of Samaria came into my mind. If I could give any
such comfort of living water! Did I really desire to? So far I had done
barely what was required of me. It did not look half as enchanting as
reading to and amusing Miss Lucy Churchill. But wasn’t there a good
work in it as well?
I entered the room with a glass pitcher, through which the water
shone and sparkled. There was such a thirsty, longing look in his
eyes that I was glad to minister to him.
“Thank you a thousand times. I have been wanting that for the last
half hour.”
“Why did you not ring sooner?”
 Rose’s Visit to the Invalid. Page 121.

“You had a visitor; and I call for so many things. There appears to
be no end to my wants. I am ashamed of myself. But it is so tiresome
to lie here helpless.”
“What would be the next thing, if I were a fairy?” I asked,
laughingly.
Was it my mood that made him smile?
“There is a tantalizing smell of honeysuckle somewhere about.
Nay, don’t run down stairs again.”
“It is just on the porch.”
Mamma was emptying Miss Churchill’s basket. A bowl of custard,
a jar of wine jelly, fresh eggs, a great, creamy pot-cheese, and the
pears. I just took a whiff of the fragrance and passed on.
“Let me have a piece of it here in my hand.”
“How odd that you should be so fond of flowers!”
 “Is it? Sweet blooms only. May be you would not approve of such
a love. I like to crush them and have them about me. Not but what I
admire them in vases, too, but then they do not come into my very
life.”
“Or die for you.”
I had said it, and then I paused in a great tremble, thinking of the
other death that came through love, greater than which hath no man.
“Miss Endicott,” he said, slowly, “are you very religious?”
I colored, and turned my face away; then I thought of “confessing
before men.” What should make me afraid here, except the sense of
personal unworthiness?
“I try a little. I have not gone very far in the way.”
“I know some people who are very religious,” he went on, “and I—
dislike them. That was another reason why I did not want to come
here—because your father was a clergyman. But you always appear
to have such nice, enjoyable times. You talk over everything with him
and your mother.”
“Why should we not? It would be strange if they were not
interested in all that concerns us. And bringing home a bit of
pleasant talk or some bright and amusing incident is like adding a
sheaf to the general granary. Does it not seem as if each one ought
to contribute to the fund of happiness?”
“I suppose it is a good deal in the way you look at it. And the
having a home, may be.”
“Yes,” I said, “that is the great thing, or next to the having a
mother.”
“What if I were seized with a fit of confessing my sins? Would that
be added to the ‘general fund?’”
“I think we have all been brought up to respect a confidence,” I
answered, a trifle wounded. “But it would be better to confess them
to papa.”
 “I might not want to;” and he gave a short laugh that did not seem
at all natural. “In fact, there are very few people who suit me, or
attract me—the same can, doubtless, be said of me. Do you know—
and I have never owned it before in my life—I am sometimes jealous
of Stuart? Every one takes to him, likes him; and he is no better than
—other people. He is not always truthful; he is awfully selfish, and
heartless, too. Only he has that sunny, glowing way with him; and
most people are such fools that they cannot see through it. So he
gets credit for sweetness, when it is only—”
“A matter of temperament,” I returned, filling up the long pause.
“Exactly. Why cannot others understand that it is so?”
“Because nearly every one likes roses better than thorns. We
naturally shrink from a rough, prickly outside. No matter if the kernel
is sweet, every one, you know, cannot wait years and years for it to
open. And you seem to shut yourself up—”
“There is nothing to show, so I make no pretence,” he answered,
in a dry, hard tone.—“I hate froth, and all that.”
“Yet I suppose the waterfall is much prettier for the spray and
bubbles. Frail as they are, they reflect many beautiful tints. And I
suppose God could have made apples just as well without such
showers of fragrant blooms, and He may put some people in the
world for the sake of the blossom and the sweetness rather than the
fruit.”
“What an idea! I should think you would be educated to consider
the strictly useful.”
“But all things that God has made have their uses.”
“You keep to the text—that God does it all. It is a woman’s
province to believe, I fancy.”
There was a little sneer in that.
“Does unbelief render men so much happier that they love to cling
to it?” I asked.
 “Oh, they are dipping into science and philosophy, and see so
much more of the world,” he replied, loftily.
I turned to the window—and was silent.
“There!” he exclaimed, “I have vexed you.”
“No.” I returned, “I am not vexed. I only wish I knew the right words
to say to you; mamma might.”
“You are all very kind. I wonder that you were so good, when the
beginning was so unpromising. You must have thought us a couple
of brutes.”
“Stuart apologized handsomely to papa.”
“O, I dare say. He is up to that sort of dodge;” and a smile of scorn
curled his thin lip.
“I wish you loved your brothers better,” I could not forbear saying.
“It is their loss, no doubt.”
What could I do with him in such a mood? “Preaching,” as Fan
called it, was useless. Then I bethought myself of Miss Churchill’s
call, and told him what she had brought for him.
“And now,” I said, “it is time you had your supper. You must be
nearly starved.”
With that I ran down stairs. Yes, I did like bright, pleasant people.
Mamma’s cheery ways and papa’s sweetness were worth more than
doses of science and philosophy, since we have to live in a work-day
world, and cannot soar up to the clouds. It is just the every-day being
that is life, not the grand dreams that never come to pass.
I prepared the tray and took it, standing it on a table at the bed’s
side. When I returned the little group were in their accustomed
places, with papa ready to ask the blessing. I slipped quietly into the
circle.
When I went to bring the dishes down I remarked a peculiar
expression upon Louis’ face.
 “Miss Rose,” he began, “I want to know how it feels to be
generous; therefore I shall give you a holiday this evening. I must
resolve to stay alone now and then.”
“Are you quite sure—?”
“Quite;” and he waved his hand, smilingly.
I did want to go down to the store with Fan; so I was glad of the
permission. Stuart started to accompany us but two of the village
boys came to call on him. I was relieved, for I wanted to stop on the
way and see one of my Sunday School children.
Fan bought her braid, and we found the baby at the Day’s was
sick, and Betty had to stay at home to help take care of it. Poor thing,
how wild and wan it looked, so different from our rosy Edith.
Mrs. Day’s house was generally in disorder. She was a hard-
working woman in some respects, for she was always at it. Her
husband was a gardener and day-laborer, earning his twelve dollars
a week pretty regularly, and they owned a small cottage and garden,
that Mrs. Day senior had left them. Yet they always looked very poor.
“Yes,” Mrs. Day was saying, “I couldn’t spare Betty on Sunday.
Husband went over the river to see his cousin, and took little Jem. I’d
been hard at work all the week, and was clear beat out, up half the
night, too. And I don’t see as the baby gets a bit better. You don’t
know what it is to look after a baby all alone by yourself, and not
have a soul to raise a finger for you.”
“But Betty helps a good deal.” I returned, for I could not bear to
have the child so underrated.
“A girl like that can’t do much at the best. Now, if I had one or two
grown up, as your mother has!”
She always thought if she only had something another person
possessed, she should be happier. I wondered a little how she would
get along with mamma’s cares and worries, and sewing, to say
nothing of the demands from outside.
“Ask your ma if she cannot come over. Hardly a soul has been in,
and I can’t go anywhere for a bit of change. But poor people have to
do the best they can in trouble.”
I promised, and spoke a few words of cheer to sad-eyed Betty.
“That woman always does try me!” declared Fan. “If I was a
minister’s wife she would be a thorn in my side. How many poor,
inefficient people there are in this world, and the worst feature
appears to be their inability to learn anything! I do not believe they try
in good earnest.”
“Yet I feel sorry for her.”
“Well, yes, and the poor sick baby. But if her room had been
swept, her dishes taken to the kitchen, and her hair combed, and a
collar on, how it would have altered the aspect of the place! And she
seems to think every one else in the world has it so much easier.”
“This is one of the places where one must not weary in well-doing,
papa would say,” was my rejoinder.
“You are a good little girl, Rose. I have not half your faith or
patience. I wonder if I shall be of any real and sensible use in this
world?”
“You can try to-morrow. The house will be clean.”
“I am afraid I should not want to go, otherwise,” she returned,
laughingly.
The man came over for her the next morning, quite early, having
been to the village on business. We felt that she was going off in
state, but I suppose it was on account of its being the West Side and
the Churchills, for Fan somehow was fortunate in having plenty of
rides fall to her share. She uttered a laughing good-bye and they
drove away.
It seems odd how one event comes out of another, like the
wonderful Chinese transformations. You open a ball and the article
inside is one you would never have guessed at. You go to some
place, and one trifling incident changes the course of one’s whole
life, or a few words that some person utters carelessly brings about a
new train of thought and action, and your life is not quite the same
afterward.
 The Churchill mansion had a look of the old nobility. It was two
stories, with a great, double pitched roof, and wide, overhanging
eaves. Just the old fashion of white and green. But the blinds were
never faded, and the exterior never soiled. A porch on the front and
one side, upheld by square, white columns, and on the other side the
graveled roadway to the barn. A lawn in front, terraced twice, with
clumps of blossoming shrubs, or dainty beds cut out sharply in the
grass. For the house stood on a slight hill which gave it a still more
commanding appearance.
But around, just a trifle removed, to let in the sunshine, stood the
glory of it all. Great trees, elms, maples, a giant black-walnut,
hemlocks that must have grown nearly a hundred years, firs, spruce
and larches waving their long fringy arms. No modern sacrilegious
hands had come near to disturb them. Birds built in their branches
year after year, and the sunshine sifted through on the grass.
It was a warm morning and Miss Lucy’s reclining chair had been
wheeled out to the shady side of the porch. She was dressed in
white with pale pink roses in her hair and at her throat. Just turned of
thirty she had the Churchill maturity with a certain delicate
girlishness. You could imagine Miss Esther being a handsome and
stately old woman, but it seemed as if Miss Lucy must always stay
where she was.
Mr. Churchill came to help her out, Miss Esther welcomed her
warmly, and Lucy put forth her hand with a smile.
“I was afraid we might be too early, but Abner had to go over on
business, and we told him to wait if you were not ready. Did you
have a pleasant ride?”
“It is very kind of you to come. I hope I shall not tire you out before
the day is over,” said Miss Lucy.
“O, you will not, I am sure,” Fanny returned with her bright smile. “I
am delighted to come.”
“And the sewing?” Miss Churchill exclaimed.
“I did all the long seams last evening, and Rose is to bind my
trimming for me, but I am much obliged.”
 “What industrious girls you are. I am almost conscience smitten.
Are you quite sure you could be spared?”
“Yes, indeed. Don’t think of that please, Miss Churchill.”
“Will you sit here awhile? The air is so fresh and fragrant. The
greater part of my going out amounts to this only, so I am thankful for
the beautiful prospect. Look at those woods over there.”
Another knoll dark with evergreens as tall as those around the
house. At a little distance an adjoining hill, but in the level opening
between, there was a field of ripening wheat which looked like a
golden sea. Fan spoke of it.
“How odd,” returned Miss Lucy, “I have had the same thought
dozens of times in the last fortnight. I sometimes imagine that there
is a lovely undiscovered country just beyond, and what it is like. I am
glad that I cannot go out to discover it, that would take away half the
charm.”
Fanny smiled at the quaint conceit, so satisfying.
“And now tell me all about the children at home, and the sick
young man? What do you think Dr. Hawley said to me a few days
ago?—Miss Lucy, you need some one to bring you a good dish of
gossip.”
“Good gossip at that;” laughed Mr. Churchill with a humorous
twinkle in his eye. “If Miss Endicott does not acquit herself well, I’ll go
for some of the village cronies.”
“I’ll begin with the baby then,” and Fanny moved her seat a trifle.
“She is just the cunningest baby you ever saw. We were all smart
children, but she is a prodigy. She sits alone, and creeps a little
sideways, and when she gets in a glee, flaps her wings, i. e. her
arms, and crows.”
Mr. Churchill shook his head solemnly. “That will hardly do for a
girl,” he said, “and a clergyman’s daughter.”
“We think it best for her to do her crowing while she is small,” was
Fannie’s playful answer.
 She talked about the others—Fan had a way of brightening up
everything that was very amusing. Not that she ever made it out
better or worse—it was only the quaint touches of harmless
pleasantry.
Miss Lucy laughed softly and a pink tint came to her pale cheeks.
Miss Esther in the meanwhile made several journeys to and fro.
Mr. Churchill took up his paper and pretended to read, but his eyes
wandered to the fair young girl whose simple homelikeness was her
greatest charm. Presently the sun came around, and Miss Lucy’s
chair was wheeled to the sitting room, which was cool and shady.
All their entertainments were not kept for the great drawing-room.
Here were pictures, a well filled book-case, articles of virtu, a cabinet
of shells, minerals and precious stones, and portfolios of fine
engravings. Here an album filled with notable authors, artists and
musical people, another with eminent men of Europe, and
remarkable women. Fan had enough to entertain her there.
Suddenly a bell rang.
“This is the shortest morning that I have known for some time,”
exclaimed Miss Lucy. “It hardly seems possible that is the dinner
bell. We are old fashioned in our hour, you see.”
Fan was astonished as well. Mr. Churchill gave Lucy his arm, as
she could walk with a little assistance. Miss Esther led Fanny.
The dinner table was like a picture. The quaint old china, delicately
flowered, and the antique silver was set off by the snowy cloth and
the brilliant bouquets with trailing stems that looked as if they might
have grown in the vases. Fan enjoyed it all to the uttermost, and was
too happy to envy aught of it.
“You have been sitting up all the morning;” said Miss Churchill,
“and you do not look a bit tired now! Shall we give Miss Fanny the
credit?”
“I think she deserves it. Indeed I hardly noticed how the time
passed. You see I get so tired of staying alone, or talking over the
same old subjects with Essie.”
 “You are a grateful young woman I must say!” and Miss Churchill
laughed.
“Kenton, couldn’t we have a drive to Round Hill about sunset? I
think I could go.”
“What, more dissipation?”
“Please don’t undertake too much,” said Fanny. “I am well content
to stay here.”
“But the sunset is so lovely there.”
“You must have a good long rest;” said her sister, “and we will see
how you feel then.”
“I dare say Miss Endicott will be glad for I have kept her talking
steadily.”
“I am used to it,” laughed Fan, “and somehow I never understood
the charms of solitude, or perhaps was born incapable of
appreciating them.”
“There is no doubt of that;” Mr. Churchill returned with a quiet
smile.
They sat over their dessert a long while, talking of various subjects
that were exceedingly entertaining. The quiet and air of formal
courtesy that was far removed from stiffness, pleased and interested
Fanny greatly.
But Miss Churchill was inexorable afterward, and would not even
consent to Fan’s going up stairs with Lucy. Instead she took charge
of her and they inspected the house and the clean, fragrant dairy,
and lastly found themselves in Miss Churchill’s room. This was large
and airy, looking cool in its summer dress of matting and furniture of
cane or delicate chintz covers.
The visit was so different from the formal little calls that we had
been in the habit of making with either of our parents. Indeed, Fan
always declared that this day’s experience took her right into the
Churchills’ lives, and I think it did.
 A dress of fine white India muslin lay on Miss Churchill’s bed. At
least, the skirt which had three ruffles edged with delicate
needlework. The rest had been ripped apart and ironed out.
“Do you think it pretty?” asked Miss Churchill.
“It is lovely. What exquisite muslin! I wonder what makes these old
things so much more beautiful than what we have now?”
“They are neater and not so showy.”
“But this would be noticeable anywhere.”
“Yes, yet it has an air of quiet refinement. Twenty or thirty years
ago ladies bought dresses to keep, now they are unpardonably old
after one or two seasons, therefore it does not pay to make them so
elegant. My dear Miss Fanny, I may as well confess to a conspiracy.
I brought out a lot of old dresses yesterday—too pretty to give to the
absolute poor. I selected this and altered the skirt. It is all done but
the band. I did not know precisely what to do with the waist, so I shall
have to give you the material. And if you will accept these for
yourself and your sister—there is a great quantity in them, and you
will find it a nice, serviceable fabric, as it will save washing. Please
do not consider me officious.”
“Oh, Miss Churchill!” was all that Fan could say.
“This pine-apple will be good for afternoon wear, and I believe to
some extent in useful gifts. The other I wanted you to have because
it was so pretty. I have two more, which will last me my life time.”
“You are too generous! Oh, Miss Churchill, how can I thank you?”
“By wearing and enjoying them, my dear, and not having any fussy
feeling over them. Just as if they had come from an aunt, for
instance. I do not believe your mother will object. She is too truly a
lady to fancy that I desire to place you under any obligation.”
I should have stood silent and abashed. Fan did the best thing of
all, just clasped her arms around Miss Churchill’s neck and kissed
her for thanks.
 The stage came lumbering along at that moment. Miss Churchill
glanced out of the window with one arm still around Fanny.
“Of all things! Here is Winthrop Ogden looking too merry for any
misfortune. It is like him to take us so by surprise. My dear, I will run
down a moment.”
 CHAPTER VIII.

AN heard the sound of the voices without distinguishing

the words, and turned to an inspection of the dresses.
There was a dainty apron overskirt of the muslin, with
the same lovely ruffling around it, and plenty for the
waist and sleeves. The others were blue and white
striped, one very narrow, the other about an inch wide, with a kind of
embroidered figure in the stripe. The skirts were long and full, and
with one there was a mantle.
Miss Churchill returned presently.
“You have not mutinied in my absence, have you? My dear girl, I
do not want to place you under any obligation, yet I thought these
garments might be of some use to you and your sister. It would not
do to send them to the sewing society to be cut up for the very poor.
Don’t give yourself an anxious thought. Now go in to Lucy who is
waiting for you, and when she is ready you must come down stairs
and see our nephew.”
Miss Lucy was much rested. “I have had three or four naps which
did me a world of good;” she said. “You see I was very tired, and that
made me rest delightfully. All my good things come at once.
Winthrop is here.”
She looked bright and cheerful.
“Can I not do something for you?” asked Fanny. “I might brush
your hair, I often do mamma’s.”
“If it would not trouble you—Essie does it in the morning, and I
generally manage it in the afternoon. But I am afraid—”
“No,” returned Fan cordially, divining the delicate fear. She took up
the brush and soon had it in order.
Usually Miss Lucy walked about without any assistance, but being
somewhat weak now, she had to help herself with a cane which she
always kept in her room. Fan tried to anticipate her wants, and she
was ready so soon that she rang the bell for her brother, who came
to assist her down stairs.
“This is our nephew, Mr. Ogden;” announced Miss Churchill, “Miss
Endicott.”
Fan remembered seeing him at church occasionally. He was about
twenty-two, and had matured considerably in a year. He was
medium height, with a rather handsome, rollicking face. There was a
laugh in his hazel eyes, in his curly chestnut hair, and it seemed to
play hide and seek about his mouth, the upper lip being shaded by a
soft brown moustache.
“Ah, Miss Endicott,—though I ought to know you without a formal
presentation, only I could not save your life I suppose if I were not
introduced. How are you, Aunt Lucy? Why you have roses, actually! I
thought from Aunt Essie’s letter that you must be a pale shadow!”
“The roses are in your honor, and not very durable. I am glad to
see you, but oh how you have—changed!”
“For the better allow me to hope!”
She laughed. “But how did you come to take us so by surprise? I
thought you were at Newport.”
“I was yesterday. But I had to dance so much night before last,
that I was afraid of impairing my constitution. I began to sigh for
simple country life, and came hither, thinking of uncle Kenton’s
horses.”
“They are in a fine condition;” said Mr. Churchill.
“And I can have only a fortnight’s vacation now, so I mean to make
the most of it. The other two weeks will come in September.”
“Your mother and Helen—?”
“Are delightfully well and charmingly entertained—can you ask
more?”
He gave a droll little smile at this.
 We heard sometime afterward that Miss Ogden was engaged, and
her lover’s mother being very fond of her, they had gone to Newport
somewhat on her account.
“Come out on the porch. Here is your chair Lucy,” and Mr.
Churchill wheeled it round. “Why you do look quite bright. Miss
Endicott, we must thank you for part of it.”
“I have not done much, I am sure.”
“Oh, Aunt Lucy, have you a protege, or has Miss Endicott kindly
consented to rule you for the nonce? Will my occupation be gone?”
“What nonsense, Winthrop! Miss Endicott came over to spend the
day, taking pity on me. I have been so forlornly lonesome of late.”
“Then I have arrived just in the nick of time, if that word has any
meaning or relation to anything above or under the waters. Let me
feel your pulse. Quite reduced, I must admit. Beef tea and camomile
flowers three times a day. A long walk morning and evening.
Cheerful society—a new bonnet—and—but try that first. My
knowledge is not exhausted.”
“Could you take the ride, think, Lucy?” asked Miss Churchill.
“Kenton we will have the large carriage and all go.”
“What conspiracy have you planned?” inquired Winthrop.
“A harmless drive,” returned Miss Lucy mirthfully. “If it looks
suspicious we will leave you at home in Hugo’s charge.”
Hugo was a handsome English hound, as aristocratic as his
master.
They all asked and answered questions, drawing Fan within their
beautiful circle by the fine tact of thorough breeding. She was so gay
and charming, and withal natural without any aiming at position or
special notice. Indeed she and Mr. Ogden had two or three passages
of sharpness between them that made their elders laugh.
It came supper time so soon, that Miss Lucy declared gaily she
had been defrauded; the day certainly was shorter than usual.
 “Yes,” returned Mr. Churchill, “they are. We have passed the
longest days.”
“Have you? Sometimes I feel as if I were just coming to mine,” and
there was a graver look in her face.
“Aunt Lucy low spirited! Why I thought you were a very princess of
philosophy!”
“One’s heart does fail sometimes.”
“But I am to be married you know, and you are to make me long
visits. I’ll save my buttons for you to sew on, you shall embroider my
initials, and mend my gloves. Will not that be happiness enough?”
“What is your wife to do meanwhile?”
He affected to be puzzled. “Why I suppose she will not know how
to do anything. Is not that the accomplishment of the girl of the
period?”
“There may be girls of the semi-colon who do not go quite so far;”
answered Miss Lucy drolly.
Winthrop glanced up at Fan who colored vividly.
“Excuse me, Miss Endicott, I—”
 Stephen Duncan.

“Winthrop you are not to tease Miss Endicott, nor to classify her,
either. I take her under my especial protection.”
 “I lay down my arms at once, Aunt Lucy. I am your most obedient.”
And so it went on with bits of fun and pleasantness cropping out
now and then. Mr. Churchill unbending, Miss Churchill straying from
the little hedge of formalities, sweet as a wild briar blossom. And
Lucy was nearly as bright as Fan.
The carriage came around soon afterwards. Mr. Ogden insisted
upon driving, so the man was dispensed with. The Churchill estate
was very large, including the mountainous track and a good deal of
woodland. It was not a much frequented drive, although Round Hill
was one of the curiosities of the town. But the Churchills and the
Garthwaites seemed to fence it in with their sense of ownership, and
it was not common property like Longmeadow and the Cascades.
But it was very beautiful in the low lying light. Here was a field in
deep gloom, shadowed by yonder trees, here a strip of waving grain,
then long sweeps of grassy hillsides broken by clumps of young
cedars or hemlocks. An irregular wooded chain—the mountains,
Wachusetts’ people called them, divided us from the quaint little town
lying in the next valley. Here was the delightful opening that
appeared more level by contrast with the tall trees on both sides, and
next, symmetrical Round Hill, in a flood of golden red light, for the
sun was going down between this and the next eminence.
Fan just turned to Miss Lucy and put out her hand. But the
eloquent words and the intense appreciation were in her fluttering
color, her swelling lip and kindling eye, and the simple gesture.
“I knew you would like it;” said Lucy just as quietly.
Miss Churchill looked over at them. Was she thinking of what Mrs.
Endicott had said—how she kept young in her children’s lives? For
Lucy’s face was like a girl’s again.
“I sometimes think there can be nothing in Europe more beautiful,”
Miss Lucy said at length. “It is my Alps. Ah, if one could paint that
glow!”
Winthrop glanced back. “I wish you could have seen some
pictures in the Academy, Aunt Lucy;” and then he went on to
describe them in an eager manner, evincing much genuine love for
beauty, and a kind of fitness, for his tone was low and earnest,
without any assumption of manishness.
Meanwhile as they wound slowly along, the sky changed from the
crimson gold, to orange, then to a yellow tint, sending out long rays
into the frost-white, not unlike an Aurora. All the edges of the hills
were purple and blue, with a peculiar velvety softness, losing
themselves presently in hazy indistinctness.
“Kenton,” Lucy said, “this place ought to be re-christened. Sunset
Hill would be more appropriate. There is no such enchanting sunset
for miles around.”
“But it isn’t always that,” in his dry humorous way. “And it is always
round.”
“Then mine shall be its holiday name, a kind of golden
remembrance.”
“It is beautiful;” Miss Churchill said with deep feeling. “Miss Fanny,
your father preached a good sermon last Sunday morning, about our
longing for loveliness and grandeur which was far away, and not
enjoying that right beside us, and our desiring to do some great
thing, waiting years for the opportunity, when we might have made
our lives rich with the small daily deeds that are at our very finger
ends. And how many of us long for Italy when we have clear skies
and glowing sunsets at home that we know nothing about!”
“Because such lives are crowded full to repletion. I sometimes
wonder if we do not have too much instead of not enough? I find a
large world right around here, because I can’t get out comfortably to
any larger one.”
“And because you see the beauty in every thing,” returned Fanny
softly. “It’s just like daily bread, the now, and here. We need not
starve to-day because of a famine coming a hundred years hence.”
Mr. Churchill raised his grave eyes and smiled, just a little. They
moved on quietly again, the wide glory of the twilight heavens falling
gently over, clasping hands with the indistinct outlines of the beautiful
earth. The creek went rippling and winding around, making a