You are on page 1of 54

How the mind comes into being :

introducing cognitive science from a


functional and computational
perspective First Edition Butz
Visit to download the full and correct content document:
https://textbookfull.com/product/how-the-mind-comes-into-being-introducing-cognitive
-science-from-a-functional-and-computational-perspective-first-edition-butz/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

The Psychology of Learning An Introduction from a


Functional Cognitive Perspective Jan De Houwer

https://textbookfull.com/product/the-psychology-of-learning-an-
introduction-from-a-functional-cognitive-perspective-jan-de-
houwer/

The Embodied Mind Cognitive Science and Human


Experience Francisco J. Varela

https://textbookfull.com/product/the-embodied-mind-cognitive-
science-and-human-experience-francisco-j-varela/

Happy Brain Where Happiness Comes From and Why First


Edition Dean Burnett

https://textbookfull.com/product/happy-brain-where-happiness-
comes-from-and-why-first-edition-dean-burnett/

Cognitive Science An Introduction to the Science of the


Mind 3rd Ed 3rd Edition José Luis Bermúdez

https://textbookfull.com/product/cognitive-science-an-
introduction-to-the-science-of-the-mind-3rd-ed-3rd-edition-jose-
luis-bermudez/
Brentano’s Philosophical System: Mind, Being, Value
First Edition. Edition Uriah Kriegel

https://textbookfull.com/product/brentanos-philosophical-system-
mind-being-value-first-edition-edition-uriah-kriegel/

A Comparative Study on the Translation of Detective


Stories from a Systemic Functional Perspective Yan Wang

https://textbookfull.com/product/a-comparative-study-on-the-
translation-of-detective-stories-from-a-systemic-functional-
perspective-yan-wang/

How the Right Lost Its Mind First Edition Sykes

https://textbookfull.com/product/how-the-right-lost-its-mind-
first-edition-sykes/

Cognitive science and the New Testament : a new


approach to early Christian research First Edition
Czachesz

https://textbookfull.com/product/cognitive-science-and-the-new-
testament-a-new-approach-to-early-christian-research-first-
edition-czachesz/

Tibetan Yoga for Health Well Being The Science and


Practice of Healing Your Body Energy and Mind Alejandro
Chaoul

https://textbookfull.com/product/tibetan-yoga-for-health-well-
being-the-science-and-practice-of-healing-your-body-energy-and-
mind-alejandro-chaoul/
How the Mind Comes Into Being
How the Mind Comes
Into Being
Introducing Cognitive
Science from
a Functional
and Computational
Perspective

Martin V. Butz and Esther F. Kutter


Department of Computer Science and Department of Psychology
Faculty of Science, Eberhard Karls University of Tübingen
Tübingen, Germany

3
3
Great Clarendon Street, Oxford, OX2 6DP,
United Kingdom
Oxford University Press is a department of the University of Oxford.
It furthers the University’s objective of excellence in research, scholarship,
and education by publishing worldwide. Oxford is a registered trade mark of
Oxford University Press in the UK and in certain other countries
© Oxford University Press 2017
The moral rights of the authors have been asserted
First Edition published in 2017
Impression: 1
All rights reserved. No part of this publication may be reproduced, stored in
a retrieval system, or transmitted, in any form or by any means, without the
prior permission in writing of Oxford University Press, or as expressly permitted
by law, by licence or under terms agreed with the appropriate reprographics
rights organization. Enquiries concerning reproduction outside the scope of the
above should be sent to the Rights Department, Oxford University Press, at the
address above
You must not circulate this work in any other form
and you must impose this same condition on any acquirer
Published in the United States of America by Oxford University Press
198 Madison Avenue, New York, NY 10016, United States of America
British Library Cataloguing in Publication Data
Data available
Library of Congress Control Number: 2016946807
ISBN 978–0–19–873969–2
Printed and bound by
CPI Group (UK) Ltd, Croydon, CR0 4YY
Oxford University Press makes no representation, express or implied, that the
drug dosages in this book are correct. Readers must therefore always check
the product information and clinical procedures with the most up-to-date
published product information and data sheets provided by the manufacturers
and the most recent codes of conduct and safety regulations. The authors and
the publishers do not accept responsibility or legal liability for any errors in the
text or for the misuse or misapplication of material in this work. Except where
otherwise stated, drug dosages and recommendations are for the non-pregnant
adult who is not breast-feeding
Links to third party websites are provided by Oxford in good faith and
for information only. Oxford disclaims any responsibility for the materials
contained in any third party website referenced in this work.
v

To our families
Preface

More than 2000 years ago — maybe as the first humans in the world — Greek philosophers
have thought about the puzzling introspectively assessed dichotomy between our physical
bodies and our seemingly, non-physical, mental minds. How is it that we can think highly
abstract thoughts, seemingly fully detached from the actual, physical reality? Despite the
obvious interactions between mind and body (we get tired, we are hungry, we stay up late
despite being tired, etc.), until today it remains puzzling how our mind controls our body,
and vice versa, how our body shapes our mind.
This textbook focuses on the embodiment of the mind in all its facets. Many other
books on cognitive science focus on sensory information processing, motor output genera-
tion, reasoning, symbol processing, and language in separate rather disconnected chapters.
This book integrates these aspects of cognition sketching-out their interdependencies and
their potential ontogenetic (over a life-time) and phylogenetic (over centuries, millennia, and
beyond) development. As a consequence, after giving a multidisciplinary background on the
development of cognitive science as a research discipline, the book starts from an evolu-
tionary developmental perspective, shedding light on how behavior traits and genetic codes
may have developed on our earth. Next, we proceed with cognitive development during a
life-time, focusing on reinforcement learning and the development of flexible behavioral capa-
bilities by learning about and selecting amongst redundant alternative interactions with the
environment. An excursion into a functional perspective on the human brain then leads to
principles of perception and action generation, including the relations between these prin-
ciples, leading to multisensory and multi-information interactions. Processes of attention
and the different aspects of attentional processes then lead to principles of decision making,
language, and abstract thought. One goal of the book is thus to sketch-out a developmental
pathway towards higher, symbolic, abstract cognitive capabilities starting bottom-up with
principled processes of bodily representations and body control, offering a novel perspective
on cognitive science by taking a sensorimotor, integrative, embodied stance.
Meanwhile, the textbook offers a principled mathematical, functional, and algorithmic
background about the processes that apparently underly neurocognitive representations,
processes, adaptations, and learning in our brains. In particular, when considering phy-
logenetic, evolutionary development of the human mind, we introduce genetic algorithms
and their functionality, revealing potentially rather simple developmental pathways and
contrasting these with situations of highly unlikely evolutionary developments. Moving to
reward-oriented adaptations of behavior, reinforcement learning is introduced and analyzed
once again in terms of its capabilities and challenges. Interestingly, evolutionary learn-
ing algorithms appear to have much in common with a particular type of reinforcement
learning algorithm. We also show, nonetheless, that our behavior and behavioral decision
making cannot be purely reward-driven. We are able to make context-dependent decisions
and choose alternative environmental interactions given current circumstances. Thus, we
are highly flexible in our interactions with the environment. Where does the flexibility of
our mind come from? We show how it is possible to alter interactions on demand – so to
say, effectively doing what seems right given the current context – by means of abstract,
predictive models.

vii
viii PREFACE

With cognitive flexibility in mind, we then have a look at the current neuroscientific
knowledge from a functional perspective. Several of the rather dedicated modules in our
brain are then further evaluated and analyzed in further computationally-oriented detail.
In particular, we proceed by bottom-up visual information processing and the possibility
to include top-down influences into this processing stream. Once again computationally
oriented, we show how such interactive information processing can be accomplished by means
of Bayesian probability and predictive coding principles. The same principle also applies
when information from various sensors needs to be integrated into one fused percept of
both the environment and the own body. To interact flexibly and selectively with mental
concepts, attention for behavior needs to be enhanced to principles of internal attentional
processes, which select those mental representations that are currently most appealing. Once
again, the bottom-up processes are combined with top-down processes to guide information
processing for behavioral- and mental control.
Equal processes expand to principles of planning and decision making, for which an
embodied value system needs to be integrated, and finally to language and abstract thought.
Even with respect to language, computational principles can explain aspects of language
evolution, including benefits of communication, the need to individualize your conversation
partners, and principles of basic grammatical structures. Moreover, structures that are
highly suitable for accomplishing behavioral flexibility and enabling more complex planning
and decision making may determine the principle underlying the universal grammar, offer
solutions to the symbol grounding problem, and bootstrap the very flexible compositionality
offered by human languages.
In closing, we hope this book provides a new, intriguing perspective on cognitive science.
While being a textbook with educational contents, the book also aims at shedding light onto
the recent movement towards embodied cognitive science, offering answers to the questions
of what embodied cognitive science may mean and what it may imply.

Acknowledgments
This book would not have come into being without the help of many people, including our
families, friends, and many colleagues. Thank you for all the support and understanding.
As the first author, I would like to particularly thank my colleagues at the University of
Würzburg for supporting me during the time when I gave the lecture – upon which large
parts of the book are based on – for the first time in the summer term of 2008. In par-
ticular, professor Frank Puppe has encouraged me to present my perspective on artificial
intelligence and cognitive systems back then. Moreover, professor Joachim Hoffmann has
always provided his complete trust and support throughout my time at the University of
Würzburg and beyond that until now. I am immensely grateful for all the inspiring discus-
sions, comments, and constructive criticism throughout my career. Only due to his thorough
and farsighted work in cognitive psychology – and beyond that towards cognitive science
– have I understood and internalized the importance of predictions and anticipations for
cognition. I am equally in debt to professor David E. Goldberg for all his trust and support
throughout my time at the University of Illinois at Urbana-Champaign, IL (UIUC) and
beyond that. Due to the highly inspiring discussion and analytical sessions with him, I have
learned – amongst many other things – to appreciate the importance of understanding a
complex system’s functionality by means of facet-wise analysis. I would also like to thank
my colleagues at the University of Tübingen for supporting my work here in many ways.
Finally, I would like to thank my wife for her understanding for all the late nights and her
continuous support, and my kids for always welcoming me home and reminding me in their
beautiful ways that a very important part of life has nothing to do with research.
As the second author, I would like to express my deepest gratitude to Martin for all
the trust and support he put in me. In countless passionate and constructive discussions
with him he helped me understand how to see and forge links between all the different
aspects of cognitive science. During the past years in his team I have internalized the
PREFACE ix

impact and importance of a holistic, unifying understanding of cognition in all its facets. I
am immensely grateful that he offered me this unique opportunity to become coauthor of
this book. Finally, I am so grateful to my family, who provided me with unfailing support
and continuous encouragement throughout the project.
Comments and suggestions for the actual book came from numerous people. We found
the enthusiasm that was returned when we detailed the planned book’s content particularly
encouraging. Several anonymous reviewers from OUP have provided very useful suggestions
including to keep a balanced perspective, which is hopefully maintained successfully. More-
over, we would like to thank the Cognitive Modeling team for reading book chapters and
providing very useful contents suggestions. We are particularly grateful also for PD Oliver
Herbort, who managed to give detailed and highly valuable comments on the first chapters
in these busy times. Moreover, we would like to thank Carolyn Kinney for proofreading
the book in immense detail and very thoroughly and also Kenneth Kinney for his detailed
proofreading effort.
In closing, we would like to thank professor Harold Bekkering, professor Wayne Gray, and
professor Karl Friston for their wonderful book endorsements and all the support beyond
that. We very much hope, of course, that the book will be well-received by many other
researchers, students, and anybody with the longing to understand “how our minds come
into being.”
Contents

Preface vii

1 Embodied Cognitive Science 1


1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Our brain controls our body . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Our body controls our brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Our body and our world shape our brain . . . . . . . . . . . . . . . . . . . . . 3
1.5 Our brain develops for a purpose . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.6 Computational knowledge is necessary . . . . . . . . . . . . . . . . . . . . . . 5
1.7 Book overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Cognitive Science is Interdisciplinary 9


2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.2 Philosophy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2.1 Epistemology and its development . . . . . . . . . . . . . . . . . . . . 11
2.2.2 Philosophy of mind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2.3 Philosophy of language and forms of representation . . . . . . . . . . . 21
2.3 Biology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.3.1 Neurobiology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.4 Psychology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.4.1 Behaviorism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.4.2 Constructivism and developmental psychology . . . . . . . . . . . . . 33
2.4.3 The cognitive turn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.4.4 Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.5 Bringing the pieces together . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3 Cognition is Embodied 45
3.1 Computers and intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.2 What is intelligence anyway? . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.2.1 Early conceptualizations of intelligence . . . . . . . . . . . . . . . . . . 47
3.2.2 Further differentiations of intelligence . . . . . . . . . . . . . . . . . . 48
3.3 Symbolic artificial intelligence and its limitations . . . . . . . . . . . . . . . . 49
3.3.1 Symbolic problem solving . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.3.2 Symbolic linguistic processing . . . . . . . . . . . . . . . . . . . . . . . 51
3.4 Hard challenges for symbolic processing systems . . . . . . . . . . . . . . . . 53
3.4.1 Symbol grounding problem . . . . . . . . . . . . . . . . . . . . . . . . 53
3.4.2 Frame problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.4.3 Binding problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
3.5 Neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.6 Embodied intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.6.1 Embodied biological processing . . . . . . . . . . . . . . . . . . . . . . 62

xi
xii CONTENTS

3.6.2 Embodied artificial intelligence . . . . . . . . . . . . . . . . . . . . . . 65


3.6.3 Embodied cognitive agents . . . . . . . . . . . . . . . . . . . . . . . . 69
3.7 When have we reached artificial, human cognition? . . . . . . . . . . . . . . . 71
3.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

4 Cognitive Development and Evolution 77


4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.2 Ontogenetic development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.2.1 Prenatal development . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.2.2 Cognitive development after birth: the first few years . . . . . . . . . 81
4.3 Phylogenetic development and evolution . . . . . . . . . . . . . . . . . . . . . 88
4.3.1 A brief history of evolution science . . . . . . . . . . . . . . . . . . . . 89
4.3.2 Genetics in a nutshell . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.3.3 Evolutionary mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.4 Evolutionary computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.4.1 Basic components of evolutionary computation algorithms . . . . . . . 96
4.4.2 When do evolutionary algorithms work? . . . . . . . . . . . . . . . . . 102
4.5 What can we learn from evolution? . . . . . . . . . . . . . . . . . . . . . . . . 105
4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

5 Behavior is Reward-oriented 109


5.1 Introduction and overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.2 Reinforcement learning in psychology . . . . . . . . . . . . . . . . . . . . . . . 109
5.3 Reinforcement learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.3.1 RL problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
5.3.2 Temporal difference learning . . . . . . . . . . . . . . . . . . . . . . . 114
5.3.3 Speeding up temporal difference learning . . . . . . . . . . . . . . . . 118
5.3.4 Behavioral strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.3.5 Actor-critic approaches . . . . . . . . . . . . . . . . . . . . . . . . . . 123
5.4 Policy gradients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
5.4.1 Formalization of policy gradients . . . . . . . . . . . . . . . . . . . . . 125
5.4.2 Gradient estimation techniques . . . . . . . . . . . . . . . . . . . . . . 126
5.4.3 A racing car example . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
5.4.4 Conclusions and relations to cognition and behavior . . . . . . . . . . 128
5.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

6 Behavioral Flexibility and Anticipatory Behavior 131


6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
6.2 Flexibility and adaptivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.2.1 Niches and natural diversity . . . . . . . . . . . . . . . . . . . . . . . . 133
6.2.2 Beyond behaviorism . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.2.3 Redundancies and complements . . . . . . . . . . . . . . . . . . . . . . 136
6.3 Sensorimotor learning and adaptation . . . . . . . . . . . . . . . . . . . . . . 139
6.4 Anticipatory behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.4.1 Forward anticipatory behavior . . . . . . . . . . . . . . . . . . . . . . 142
6.4.2 Inverse anticipatory behavior . . . . . . . . . . . . . . . . . . . . . . . 144
6.5 Motivations and curiosity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.5.1 Intrinsic reward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.5.2 Extrinsic reward and motivations . . . . . . . . . . . . . . . . . . . . . 148
6.6 Summary and outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
CONTENTS xiii

7 Brain Basics from a Computational Perspective 155


7.1 Introduction and overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
7.2 The nervous system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
7.3 Brain anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
7.3.1 Neurons and neural information processing . . . . . . . . . . . . . . . 157
7.3.2 Modules and areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
7.3.3 Basic brain and body maintenance . . . . . . . . . . . . . . . . . . . . 165
7.4 General organizational principles . . . . . . . . . . . . . . . . . . . . . . . . . 166
7.4.1 Function-oriented mappings . . . . . . . . . . . . . . . . . . . . . . . . 167
7.4.2 Cortical columns and topographies . . . . . . . . . . . . . . . . . . . . 169
7.4.3 Neural tuning and coordinated communication . . . . . . . . . . . . . 172
7.5 Brain mechanisms and brain imaging . . . . . . . . . . . . . . . . . . . . . . . 174
7.5.1 Brain lesion studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
7.5.2 Active methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
7.5.3 Passive methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
7.5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
7.6 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

8 Primary Visual Perception from the Bottom Up 183


8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
8.2 Light and reflections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
8.3 The eye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
8.4 Visual processing pathways . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
8.5 Redundant and complementary visual processing . . . . . . . . . . . . . . . . 190
8.5.1 Receptive fields, columns, and hypercolumns . . . . . . . . . . . . . . 191
8.5.2 Smoothing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
8.5.3 Edge detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
8.5.4 Motion detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
8.5.5 Integrating edge information . . . . . . . . . . . . . . . . . . . . . . . 198
8.5.6 Further sources of visual information . . . . . . . . . . . . . . . . . . . 200
8.6 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
8.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

9 Top-Down Predictions Determine Perceptions 203


9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
9.2 Top-down predictive, generative models . . . . . . . . . . . . . . . . . . . . . 204
9.3 Bayesian information processing . . . . . . . . . . . . . . . . . . . . . . . . . 206
9.3.1 Probability theory: a short introduction . . . . . . . . . . . . . . . . . 207
9.3.2 A simple example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
9.3.3 Bayesian networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
9.3.4 Probability distributions and densities . . . . . . . . . . . . . . . . . . 215
9.4 A Bayesian model of visual processing . . . . . . . . . . . . . . . . . . . . . . 219
9.5 Visual illusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
9.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

10 Multisensory Interactions 231


10.1 Introduction and overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
10.2 Body-relative spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
10.2.1 Redundant, body-relative, multisensory spaces . . . . . . . . . . . . . 234
10.2.2 Simple population encoding in a locust . . . . . . . . . . . . . . . . . . 237
10.2.3 Learning peripersonal spaces . . . . . . . . . . . . . . . . . . . . . . . 238
10.2.4 Optimal information fusion . . . . . . . . . . . . . . . . . . . . . . . . 240
10.2.5 Spatial, topology-preserving transformations . . . . . . . . . . . . . . 242
10.3 Multisensory recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
xiv CONTENTS

10.3.1 Object recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243


10.3.2 Behavior recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
10.4 Cognitive maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
10.4.1 Hippocampus and episodic memory . . . . . . . . . . . . . . . . . . . 247
10.4.2 Behavior-oriented cognitive map . . . . . . . . . . . . . . . . . . . . . 249
10.5 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
10.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

11 Attention 255
11.1 Introduction and overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
11.2 Top-down and bottom-up attention . . . . . . . . . . . . . . . . . . . . . . . . 256
11.3 Phenomena of attention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
11.3.1 Visual search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
11.3.2 Attention over time . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
11.3.3 Change blindness and inattentional blindness . . . . . . . . . . . . . . 261
11.3.4 Other attentional capabilities . . . . . . . . . . . . . . . . . . . . . . . 264
11.4 Models of attention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
11.4.1 Qualitative models of attention . . . . . . . . . . . . . . . . . . . . . . 265
11.4.2 Bundesen’s theory of visual attention . . . . . . . . . . . . . . . . . . 267
11.4.3 Saliency maps and eye saccades . . . . . . . . . . . . . . . . . . . . . . 268
11.4.4 Dynamic neural fields of attention . . . . . . . . . . . . . . . . . . . . 270
11.5 Summary and outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
11.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

12 Decision Making, Control, and Concept Formation 275


12.1 Introduction and overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
12.2 Compositional motor control in the brain . . . . . . . . . . . . . . . . . . . . 277
12.2.1 Muscles and the spinal cord . . . . . . . . . . . . . . . . . . . . . . . . 278
12.2.2 Motorcortex and beyond . . . . . . . . . . . . . . . . . . . . . . . . . . 280
12.3 Computational motor control . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
12.3.1 Models of online motor control . . . . . . . . . . . . . . . . . . . . . . 284
12.3.2 Models of decision making . . . . . . . . . . . . . . . . . . . . . . . . . 287
12.3.3 Action and motor control components . . . . . . . . . . . . . . . . . . 291
12.3.4 Decision making components . . . . . . . . . . . . . . . . . . . . . . . 293
12.4 Event-oriented conceptualizations . . . . . . . . . . . . . . . . . . . . . . . . . 295
12.4.1 Events and event segmentations . . . . . . . . . . . . . . . . . . . . . . 296
12.4.2 Event taxonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
12.4.3 Event conceptualizations and abstractions . . . . . . . . . . . . . . . . 298
12.5 Summary and conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
12.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

13 Language, Concepts, and Abstract Thought 305


13.1 Introduction and overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
13.2 Introduction to linguistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
13.2.1 Historical sketch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
13.2.2 Speech sounds: phonetics . . . . . . . . . . . . . . . . . . . . . . . . . 309
13.2.3 Words: phonology and morphology . . . . . . . . . . . . . . . . . . . . 309
13.2.4 Sentences: syntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
13.2.5 Semantics and pragmatics . . . . . . . . . . . . . . . . . . . . . . . . . 314
13.3 Language in the brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
13.4 Language evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
13.4.1 Shared intentionality, cooperation, and communication . . . . . . . . . 319
13.4.2 Gestural theory and verbal communication . . . . . . . . . . . . . . . 320
13.4.3 Mind and grammar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
13.4.4 Further implications of language evolution . . . . . . . . . . . . . . . . 322
CONTENTS xv

13.5 Language development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323


13.5.1 Prelinguistic concept development . . . . . . . . . . . . . . . . . . . . 323
13.5.2 Phonological and lexical development . . . . . . . . . . . . . . . . . . 325
13.5.3 Grounding and developing grammar . . . . . . . . . . . . . . . . . . . 328
13.6 Common and individual meaning . . . . . . . . . . . . . . . . . . . . . . . . . 329
13.6.1 Word meaning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330
13.6.2 Meaning in conversations . . . . . . . . . . . . . . . . . . . . . . . . . 331
13.7 Conclusions and implications . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
13.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334

14 Retrospection and future perspectives 337


14.1 Retrospection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
14.2 Some underrepresented aspects . . . . . . . . . . . . . . . . . . . . . . . . . . 341
14.3 Consciousness and cognitive modeling . . . . . . . . . . . . . . . . . . . . . . 342

References 345

Index 367
List of Figures
3.1 Necker Cube and Rubin Vases . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.2 Navon Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3.3 Braitenberg Vehicles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.4 Didabots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.1 Numerical confidence in infants: the Ernie & Elmo experiment . . . . . . . . 85


4.2 Fitness functions in real-valued evolutionary algorithms . . . . . . . . . . . . 98
4.3 Control map for innovative evolutionary process . . . . . . . . . . . . . . . . . 104

5.1 Q-learning in a simple maze . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116


5.2 Hierarchical RL in a four-rooms problem. . . . . . . . . . . . . . . . . . . . . 121
5.3 Polcy gradient example: a race-car controller . . . . . . . . . . . . . . . . . . 127

6.1 Sense→think→act cycle contrasted with bidirectional cognition . . . . . . . . 140


6.2 Explicit anticipatory behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.3 The Reafference principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

7.1 Information flow sketch during visual-based motor decision making . . . . . . 156
7.2 Neural cell body . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7.3 Neural cellular boundary interactions . . . . . . . . . . . . . . . . . . . . . . . 160
7.4 Action potential dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
7.5 Synaptic interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
7.6 Anatomical terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
7.7 Basic brain anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
7.8 Brodmann areas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
7.9 Annotated cross-section . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
7.10 Brain area function map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.11 Six-layered neocortex structure . . . . . . . . . . . . . . . . . . . . . . . . . . 170
7.12 Primary homunculi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
7.13 Neuronal tuning in visual and temporal cortical cells . . . . . . . . . . . . . . 173
7.14 Brain imaging techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

8.1 Spectrum of electromagnetic waves and visible light . . . . . . . . . . . . . . 184


8.2 Exemplar reflection distributions over the color spectrum . . . . . . . . . . . 185
8.3 Eye anatomy and light processing . . . . . . . . . . . . . . . . . . . . . . . . . 186
8.4 Pinhole camera illustration as a simple model of the eye’s lens and retina. . . 186
8.5 Distribution and sensitivity of photo receptors in human retina . . . . . . . . 187
8.6 The blind spot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
8.7 Primary visual processing pathway . . . . . . . . . . . . . . . . . . . . . . . . 188
8.8 Cortical visual processing areas and main pathways . . . . . . . . . . . . . . . 189
8.9 From simple to complex visual cells . . . . . . . . . . . . . . . . . . . . . . . . 192
8.10 Image smoothing with Gaussian filter . . . . . . . . . . . . . . . . . . . . . . 193
8.11 Types of edges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
8.12 Sobel filter example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
8.13 Optical flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
8.14 Reichardt detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

xvii
xviii LIST OF FIGURES

8.15 Four ways of forming trihedral corners . . . . . . . . . . . . . . . . . . . . . . 199


8.16 Huffman–Clowes edge labeling example . . . . . . . . . . . . . . . . . . . . . 199
8.17 Huffman–Clowes junction types . . . . . . . . . . . . . . . . . . . . . . . . . . 200

9.1 Generative perceptual models . . . . . . . . . . . . . . . . . . . . . . . . . . . 205


9.2 Generative model of a traffic light . . . . . . . . . . . . . . . . . . . . . . . . 206
9.3 Conditional probability masses and densities . . . . . . . . . . . . . . . . . . . 210
9.4 Illustration of the Zebra challenge. . . . . . . . . . . . . . . . . . . . . . . . . 211
9.5 Example of a simple Bayesian network . . . . . . . . . . . . . . . . . . . . . . 212
9.6 d-separation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
9.7 Deduction in Bayesian networks . . . . . . . . . . . . . . . . . . . . . . . . . . 214
9.8 One- and two-dimensional Gaussian probability densities . . . . . . . . . . . . 217
9.9 Gaussian mixture model examples . . . . . . . . . . . . . . . . . . . . . . . . 218
9.10 Example of a generative Gaussian mixture model . . . . . . . . . . . . . . . . 219
9.11 Neural model of visual processing . . . . . . . . . . . . . . . . . . . . . . . . . 220
9.12 Parallels between a model of visual processing and cortical brain areas. . . . . 221
9.13 Visual, spatial, and feature-oriented, interactive information processing . . . . 222
9.14 Examples of visual illusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
9.15 Terrace illusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
9.16 Fun with Necker cubes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
9.17 Bistable foreground/background and consistency of multiple figures . . . . . . 227
9.18 Simple ball detection Bayesian network . . . . . . . . . . . . . . . . . . . . . 228

10.1 Rubber hand illusion experiment . . . . . . . . . . . . . . . . . . . . . . . . . 234


10.2 Peripersonal spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
10.3 Peripersonal spatial encodings in a locust . . . . . . . . . . . . . . . . . . . . 237
10.4 Exemplar performance of Kohonen network, Neural gas, and GNG . . . . . . 239
10.5 Limbic system including hippocampus . . . . . . . . . . . . . . . . . . . . . . 248
10.6 Modeling learning a cognitive map . . . . . . . . . . . . . . . . . . . . . . . . 251

11.1 Object-oriented attention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257


11.2 Disjunctive and conjunctive visual search . . . . . . . . . . . . . . . . . . . . 259
11.3 Attentional blink paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
11.4 Change blindness examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
11.5 Inattentional blindness experiments . . . . . . . . . . . . . . . . . . . . . . . . 263
11.6 Feature integration theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
11.7 Model of visual attention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
11.8 Dynamic field theory mappings . . . . . . . . . . . . . . . . . . . . . . . . . . 272

12.1 Combination of motor output signals . . . . . . . . . . . . . . . . . . . . . . . 278


12.2 Muscle anatomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
12.3 Motor control in the spinal cord . . . . . . . . . . . . . . . . . . . . . . . . . . 280
12.4 Motor cortex and beyond . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
12.5 Cortical motor decision making . . . . . . . . . . . . . . . . . . . . . . . . . . 282
12.6 Development of reaching movements . . . . . . . . . . . . . . . . . . . . . . . 285
12.7 End state comfort effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
12.8 SURE_REACH architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
12.9 Perception of causality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

13.1 Speech organs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310


13.2 Parts of speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
13.3 Grammatic tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
13.4 Semiotic triangle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
13.5 Wernicke–Geschwind model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
List of Tables

4.1 Major approaches of evolutionary computation . . . . . . . . . . . . . . . . . 95


4.2 Fitness functions in genetic algorithms . . . . . . . . . . . . . . . . . . . . . . 99

5.1 Dynamic programming computation of the optimal state-value function V ? . . 117


5.2 Dynamic programming computation of the optimal Q-value function Q? . . . . 117
5.3 Q-value function example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

7.1 Taxonomy of brain structures . . . . . . . . . . . . . . . . . . . . . . . . . . . 166


7.2 Levels of investigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

8.1 Two visual processing pathways . . . . . . . . . . . . . . . . . . . . . . . . . . 190

12.1 Wirings from and to motor- and pre-frontal cortical areas . . . . . . . . . . . 283

xix
Chapter 1

Embodied Cognitive Science

1.1 Introduction
Over the last twenty years or so it has become increasingly clear that purely symbolic ap-
proaches to cognitive science, as well as to artificial intelligence are insufficient to grasp the
intricate beauty of our minds. We are able to draw analogies and understand metaphors
that go far beyond symbolic forms of representations and associations between them. For
example, we immediately understand a phrase such as our “circle of friends”, although our
friends never really come together in a circle or form a circle. Nonetheless, the term “circle”
suggests that our group of friends forms a close and interactive, circular relationship. Simi-
larly, we understand the phrase “grasping an idea”, although an idea is not a physical object
that can be grasped with our hands. Nonetheless, “grasping” suggests in this context that
we got hold of an idea and can thus handle it, manipulate it, and think about it.
Thousands of examples can be made in this respect (Bergen, 2012; Johnson, 1987; Lakoff,
1987; Lakoff & Johnson, 1999), suggesting that our brain cannot be a symbol processing
machine. Rather, it seems that our brain may be characterized as a highly distributed,
parallel, neuro-dynamic system. The problem with this latter characterization, however, is
that the functionality of such a system is very hard to imagine. How are analogies understood
or drawn by such a system? How can metaphors be understood by a highly distributed,
parallel neuro-dynamic cognitive architecture?
Analogy making, but also many other examples of innovative and intricate, interactive
thoughts, and behavior have led to the development of embodied approaches to cognition.
This turn to embodiment, indeed, has established itself or has at least been discussed in
various disciplines, including philosophy, neuroscience, psychology, artificial intelligence, and
the overarching field of cognitive science. Although an embodied mind that is grounded
in sensorimotor experiences may sound appealing, many open questions remain: What
are the implications of an embodied mind? How may embodied, sensorimotor processes
and representations lead to suitable abstractions and thus “higher-level” cognition? How
embodied is the human mind really?
The aim of this cognitive science textbook is to shed light on these questions by in-
troducing the basic concepts that constitute embodied cognitive science, and sketching-out
relations of these concepts and their involvement in shaping our human minds.
Two particularly puzzling facts are discussed in this chapter to further motivate the
need for embodied cognitive science approaches: the fact that the brain controls the body
and the second fact that, vice-versa, the body controls the brain. In order to establish
this symbiosis, embodied cognitive science suggests that the body and the surrounding
world in which we grow up and live in shapes the brain such that effective, bidirectional
body–brain interactions become possible. In order for us to understand such interactions
and implications on a deeper level, we strongly stress that cognitive science needs also to be
approached by means of computational techniques in the pursuit of a functional perspective.

1
2 CHAPTER 1. EMBODIED COGNITIVE SCIENCE

We believe that observations of particular, intricate cognitive interactions — such as those


revealed by analogies — alone are not sufficient to foster a deeper understanding of how the
mind may come into being. Rather, only with computational models based on mathematical
and information-theoretic principles, cognitive processes can be understood. Thus, while
sketching-out how the mind may come into being, we detail computational principles that
need to be involved to realize the different cognitive aspects necessary to reach our abstract
reasoning, language, and thought capabilities.

1.2 Our brain controls our body

From an introspective, self-conscious perspective probably every one of us is convinced that


we are responsible for our own bodily decisions and actions. This seemingly obvious fact,
however, has been highly challenged by philosophers and neuroscientists for decades. One
particular challenge was put forward by experiments conducted by the physiologist and
psychologist Benjamin Libet (1916–2007). Using brain-imaging techniques (EEG in this
case, cf. Section 7.5.3), Libet published the results of a series of experiments in 1979,
which indicate that the decision, when to press a button, was made by the brain before the
participant claimed that he or she had made the actual decision. This puzzling contradiction
between conscious decision making and the actual brain activities that are involved in the
decision-making process has led to many heated discussions and their implications are still
being questioned, reinterpreted, and debated today.
Nonetheless, it is clear that the neural signals that control our body (especially our
muscles, but also glands and other organs to certain extents) come from our brain. By
means of our brain, we make decisions and accordingly activate behavior by moving our
body in our world. Mostly fully subconsciously, our brain controls our heartbeat. Breathing
is controlled consciously and subconsciously to a certain extent. Even our intestines are
luckily not fully controlled by the organs alone, but they also communicate with the brain
to coordinate certain needs, such as when the stomach indicates hunger. Even the circadian
rhythm, that is, the night-time-day-time rhythm, which makes us sleepy at night and keeps
us somewhat awake during the day (even when we are actually really tired) is controlled
mainly subconsciously by our brain.
In addition, even those behaviors that we seem to choose consciously, such as grasping,
walking, or writing, are controlled to a much lesser degree by conscious processes as we typ-
ically tend to think. Actually, consciously thinking about it typically somewhat slows down
or even disrupts automatic behavior, indicating that conscious control is typically much
slower than subconscious control processes. Meanwhile, it seems that well-automatized con-
trol processes have access to bodily details that are hardly accessible by conscious awareness.
Try to walk a few steps while controlling every single muscle in the legs! It seems impossible
because the control access to our muscles is very indirect. Is this a problem? Should we be
scared of falling much more than we are because we cannot really control our muscles fully
consciously? Obviously not!
Indeed, we have to be more scared of stumbling when we suddenly become aware of
our walking or our posture, for example, when we sometimes become aware that we are
being watched. We will see that it is highly beneficial for our cognitive capabilities that
lots of our behavioral capabilities are automatized, thus actually relieving our higher-level
cognitive capabilities from the burden of taking care of such low-level control demands.
What is accessed by cognitive control are somewhat abstracted encodings of our behavioral
capabilities. Shedding light on the nature of these encodings and mechanisms, and thus
on how our brain controls our body, can give us insights about how abstracted forms of
cognition may develop in our brain.
1.3. OUR BODY CONTROLS OUR BRAIN 3

1.3 Our body controls our brain


Also the other direction seems obvious, but it implies rather different aspects about the
functionality of our brain. Our stomach indicates that we are hungry and leads us to search
for something to eat, to prepare food, and to eat. When our body indicates pain, we direct
attention to this uncomfortable pain and attempt to relieve this uncomfortable state of mind.
Pain thus causes discomfort, which in turn leads to behavior that we believe will help ease this
discomfort. Even subtle bodily postures, such as when we smile – regardless if intentionally
or unintentionally – induces a somewhat more positive perspective on our environment –
leading, for example, to a more positive classification of cartoon figures (Strack, Martin, &
Stepper, 1988). Thus, states of our body, as well as perceived states of the surrounding
environment generally influence our brain activity.
Over the last two decades or so, however, a much more far-reaching perspective has
started to establish itself in the cognitive science community and beyond. This “embodi-
ment turn” in cognitive science suggests that our cognitive capabilities are highly dependent
on bodily processes and on the environment with which our body continuously interacts
(Ballard, Hayhoe, Pook, & Rao, 1997; Barsalou, 1999; Clark, 1999, 2013; Engel, Maye,
Kurthen, & König, 2013). In particular, it suggests that the representations and processes
found in the brain and their ontogenetic development over a lifetime are based on the body
with its bodily morphology, including its sensory and motor capabilities. As a consequence,
seemingly highly abstract, somewhat symbolic cognition is grounded in and has developed
from sensorimotor, bodily interactions with and experiences about our environment. As
a result, abstract cognition inherently reflects the structure of our world and cognition is
inherently adaptive (Anderson & Schooler, 1991). This book thus focuses on how and which
abstract encodings and processes may develop, based on experiences about our own body
and about interactions with other objects, entities, animals, and humans.

1.4 Our body and our world shape our brain


The main implication for cognitive science itself then is essentially the need to not only
consider abstract, symbolic thought as the field of study, but to also work on revealing how
our brain controls our body and how our body controls our mind. In order to become a
success story, and to ultimately explain how our mind comes into being, it is necessary to
understand the development of language, abstract cognition, and conscious thought based
on our sensory and behavioral capabilities, and the experienced interactions with our world.
Even though our personality traits are predisposed by our genes, they are shaped by
the body and the experienced environment. Starting as a fetus, and continuing until and
throughout adulthood, cognitive development of the brain is inevitably grounded in a body
that experiences the outside world. Our mind grows and develops while interacting with
and by means of interacting with the world, and in particular with the present, particular
structures in the world. These worldly structures by no means only encompass physics and
objects, but also include other humans, especially our friends and family, and the societal
and cultural structures. Clearly, education, communication, scripture, photography, TV and
other media, and generally all sorts of training of a certain expertise also shape our mind,
and our capabilities to perceive and think about our world, including present, past, and
future. Thus, myriad worldly influences, mediated by our body, influence the way we think,
the way we perceive ourselves, including our perceived place in society, and our abstract
thought capabilities. Intelligence and consciousness certainly do not exist from birth in
their adult form.
Although already born with many clever mechanisms and capabilities, these capabilities
develop further and are shaped by the available experiences. The ultimate goal of cognitive
science is to unravel the development of our mind, and the concurrent development of brain
structures and neural representations dependent on the body and the environment in which
4 CHAPTER 1. EMBODIED COGNITIVE SCIENCE

each of us grows up, which each of us experiences, and which each of us actively explores
given individual behavioral capabilities, cognitive priorities, and environmental stimulations.

1.5 Our brain develops for a purpose


When considering such an interactive, developing mind from an ontogenetic (during a life-
time) and phylogenetic (over generations) developmental perspective (cf. Chapter 4), it has
to be kept in mind that the brain does not develop for the purpose of being intelligent or
being able to think symbolically per se. Rather, the massive, disproportional brain devel-
opment in humans must inevitably have a purpose, that is, it must have been and most
likely still is beneficial in terms of survival and reproduction. Seeing that most other species
are not as capable as humans to have abstract thoughts, intelligence does not appear to be
the ultimate goal that evolution strives-for, and also not the ultimate solution for ensuring
survival and reproduction.
Besides this phylogenetic, evolutionary puzzle, the following ontogenetic puzzle may be
considered the most challenging one in cognitive science: how can abstract thought develop
so naturally in most human beings? How is it that all of us start smiling, crawling (or
variations of it), and walking along our developmental path toward being a toddler? How
is it, that we become language-ready and are capable of learning any language while being
a toddler, seemingly without any extraordinary mental effort? How do we manage to inte-
grate ourselves in our communities, typically without any major or even deadly conflicts?
Embodied cognitive science emphasizes that these abilities develop naturally because of our
particular genetic predispositions, which influence the way in which the brain builds pro-
gressively more abstract, higher-level structures given embodied, sensorimotor experiences.
Moreover, embodied cognitive science emphasizes that our cognitive development is
driven by behavior, which in turn is driven by goals, which in turn are driven by bod-
ily and mental needs. Curiosity, which is already visible in newborns, is a great example:
newborns were shown to preferably look at biological motion stimuli, or at least stimuli
that somewhat resemble biological motion, when compared with random motion patterns
(Bidet-Ildei, Kitromilides, Orliaguet, Pavlova, & Gentaz, 2014; Pavlova, 2012). Also faces
are particularly interesting from very early on. Meanwhile, sensorimotor contingencies are
explored, such as how ones own hands look when moving them before ones eyes, or how
an object can be made to behave in a predictable manner, or how an object sounds when
manipulating it. This exploratory behavior starts very early in life—definitely at birth—but
it is probably already present while the infant is still in the womb.
Fetuses have been shown to suck on their own thumb and to know quite a lot about their
own body when born (Rochat, 2010). These and other results suggest that behavior is goal-
directed from very early on (von Hofsten, 2004). Goals can have various forms, may be ill-
specified by the brain, or may be highly abstract. Regardless how they are exactly encoded,
though, they seem to be responsible for selecting and executing most of our behavior. Thus,
cognition may be viewed as selecting goals, rather than behavior. Desirable and seemingly
currently achievable goals are the ones that are selected, and that then cause the execution
of the behavior that is believed to reach the selected goal.
Thus, the brain develops to achieve goals. Goals are selected and activated in turn for
the purpose of satisfying bodily and mental needs. Mental needs in turn appear to be driven
by genetic predispositions, such as a curiosity for face-like stimuli, which in turn is grounded
in the body, development, and the available sensor and motor capabilities. Thus, from this
perspective the brain develops to effectively select and control behavior – not to develop
abstract thoughts or intelligence. Abstract thoughts and “higher-levels” of cognition develop
in turn to be able to select goals in a maximally efficient and versatile manner, ultimately
in order to be able to adapt to and survive in diverse environments.
1.6. COMPUTATIONAL KNOWLEDGE IS NECESSARY 5

1.6 Computational knowledge is necessary


While the purpose of this chapter is to motivate embodied cognitive science and sketch-out
its implications, a secondary purpose is to motivate the functional, computational approach
pursued in this book. For studying cognition, including the involved mental development
and behavioral decision making and action control mechanisms, we believe it is absolutely
necessary to understand the underlying computational principles. While we hope that the
reader may find the previous paragraphs somewhat intuitive and possibly even obvious to a
certain extent, they only “describe” the embodied perspective on cognitive science. Details
about the implications of this perspective, such as exactly which structures can be expected
to develop, and exactly how these structures may develop, have not been given.
In order to show that and how the sketched out development toward higher levels of
cognition can actually take place, the involved learning processes and developing encodings
need to be specified further by means of computational models and implementations thereof.
By detailing and implementing computational mechanisms that can mimic particular aspects
of cognitive development and particular cognitive abilities, we can ensure that our brain can
implement similar mechanisms, at least in principle. Moreover, once we understand the
involved computational mechanisms, we may even be able to build smart machines and
intelligent agents in the real world, and possibly also in virtual worlds. In essence, we will
gain a deeper and more precise understanding of what cognition really is about. Thus, this
book introduces the necessary, fundamental computational mechanisms underlying cognition
and embeds them into the embodied perspective of cognitive science.

1.7 Book overview


The book is partitioned into two main parts: chapters 2–6 sketch-out how embodied cogni-
tive science has developed and then detail the key computational mechanisms that govern
evolution, ontogenesis, learning, and the development of anticipatory, goal-directed behavior
and cognition. Chapter 7 offers an introduction to the neuroscience of the brain in prepara-
tion of the second part of the book. Chapters 8–13 then go into more detail about how the
mechanisms and challenges put forward in the first part appear to be realized and solved by
our brain, moving from basic sensory, multisensory, and sensorimotor control processes, over
decision making and behavioral coordination, to language and abstract thought. The con-
cluding chapter (Chapter 14) wraps up How the Mind Comes Into Being, pointing-out many
aspects that could not be touched upon in detail, but that are nonetheless typically closely
related to the covered material and sketching-out an overall perspective on the put-forward
material, its inherent interrelations, and its implications.
In particular, the first part initially focuses on the historic development of our knowledge
about brain and mind (Chapter 2). The development of the philosophy of mind particularly
is discussed in detail, considering the mind–body problem from various perspectives and
emphasizing that the mind can only develop by means of a body, which interacts with and
thus experiences itself, as well as the outside environment. Based on these assumptions, it is
shown why the traditional approach to artificial intelligence, which had focused on logic and
symbol manipulations, has contributed only so much toward an explanation of how the mind
comes into being. To solve fundamental problems in cognitive science, such as the mind–
body problem, but also the symbol-grounding problem, the frame problem, or the binding
problem, the embodiment of the mind has to be acknowledged (Chapter 3). Embodied
cognitive science essentially puts forward that abstract, symbolic cognition is grounded in
sensorimotor experiences. The rest of the book essentially focuses on how abstract, symbolic
cognition may actually be grounded in these experiences.
Acknowledging the embodiment perspective, Chapter 4 suggests that the development
of the mind–body complex during ontogenesis, starting from conception and going beyond
adulthood, needs to be put into focus. Cognition dominantly develops by learning from
experiences gathered in interaction with the environment. Evolution has laid-out the right
6 CHAPTER 1. EMBODIED COGNITIVE SCIENCE

body and the right brain, with sufficient computational capacity and suitable structuring, to
develop the cognitive capabilities of a healthy human. Moreover, it appears that evolution
has laid out a suitable developmental pathway, along which body and mind co-develop. The
chapter thus first provides information about cognitive psychological development. Next, it
details evolutionary mechanisms and provides technical details on how evolution appears to
work from a computational perspective, highlighting its capabilities, as well as its limits. In
conclusion, the paper sketches-out how much evolution may have contributed to human cog-
nition and how actual human cognition appears to develop, co-determined by evolutionary
predispositions during ontogenesis.
Cognitive development goes hand-in-hand with behavioral development, such that Chap-
ters 5 and 6 consider development in detail focusing on behavior. Chapter 5 considers
reward-dependent learning, covering psychological learning theories as well as theories from
machine learning. In particular, the Rescorla–Wagner model is related to reinforcement
learning (RL). RL is then introduced in detail, contrasting model-based with model-free
RL, as well as value-function learning with direct, policy gradient-oriented behavioral opti-
mization. In the end, we ask the question how far we can get with RL and related techniques,
and we acknowledge that in order to be able to act flexibly, goal-directed dependent on the
current circumstances, and in order to develop a form of “understanding” of body and envi-
ronment, knowledge about action effects and thus predictive forward models are needed.
As a consequence, Chapter 6 contrasts different types of forward models and sketches-
out some typical learning techniques. More importantly, though, the chapter focuses on
the realization of flexible, anticipatory, goal-directed behavior, which is only possible by
forward model-based predictions and forward model inversions. Model-based predictions
enable forward anticipations, including information filtering, the generation of surprise, and
the identification of external entities and agents. The inverse application of forward models
results in goal-oriented, active inference and can thus generate highly flexible, dexterous
behavioral decision making and control processes.
With the help of the neuroscience overview provided in Chapter 7, sensory processing
is then considered in detail. Chapter 8 points out that visual processing undergoes a hier-
archical cascade of processing stages, extracting redundant and complementary aspects of
information about one’s own body and the environment from the visual stream. Different
information aspects typically contributed in different manners to the extraction about what
is going on in the outside environment, providing information about depth, surfaces, edges,
colors, motion, and so forth.
Chapter 9 then focuses on the interaction of bottom-up, incoming sensory information
with top-down expectations about this information. In particular, Bayesian, generative,
graphical models are introduced. Along these lines, it is also emphasized that predictions
can be forward in time, expecting changes due to motion and forces, but also top-down,
expecting corresponding information, such as incoming sensory information. The internal
estimations about the outside environment appear to dynamically change over time and they
adapt to the experienced regularities in the sensory information. Current models of visual
processing thus emphasize the interaction of top-down spatial and feature predictions with
bottom-up, incoming sensory information. This interactive information processing principle
is also the reason for various types of illusions, some-of-which are surveyed in the end of this
chapter.
After having considered a single sensory stream in detail, Chapter 10 emphasizes mul-
tisensory interactions, where multiple sensory modalities provide once again redundant and
complementary sources of information about body and environment. Thus, generally, sim-
ilar information processing principles apply as for single sensory processing. However, it
is emphasized that the learning of multisensory information first needs to focus on learn-
ing the structure of the own body, to be able to transform modal sensory information into
other modalities posture-dependently. To enable flexible information transformations, the
brain appears to learn multiple spatial encodings and spatial transformations. These enable
not only the effective correlation of multimodal sensory information, but also the effective
1.7. BOOK OVERVIEW 7

filtering of own bodily motion, as well as robust object and behavior recognition. Besides
body-relative spatial encodings, also multisensory cognitive maps develop, which support
behavioral planning and decision making in navigational tasks.
With the knowledge of all these types of multisensory information, their flexible correla-
tions across frames of reference, the involved spatial encodings, and the Gestalt encodings
in hand, the focus then falls on attention in Chapter 11. Attention is essentially necessary
to enable the selection of and focus on those information aspects that are currently behav-
iorally relevant. Seeing that we are, so-to-say, continuously bombarded with a huge amount
of sensory information, it appears that our brain needs to filter most of it in order to enable
the execution of focused, goal-oriented planning, decision making, and control processes.
Starting with psychological phenomena of visual attention, we point-out that world is its
own best model, such that attention needs to direct cognitive resources only to those aspects
of the environment that seem relevant. Dynamic neural fields and dynamic Bayesian infor-
mation processing are good candidates to realize such attention-based, focused information
processing.
One of the most important questions with respect to cognition, however, is how attention
decides on what is actually relevant. Chapter 12 shows how this relevance determination is
grounded in motor control. Motor control in our brain is, similar to visual information pro-
cessing, structured in a cascade of interactive processing loops. As a result, higher cognitive
processes do not need to fully control or even plan each individual muscle twitch; rather,
they can focus on higher-level planning, action decision making, and action control. On this
higher-level, segmentations of environmental interactions into events and event transitions
become important. Events can be characterized as a short or extended period of time dur-
ing which particular interactions take place. Event transitions are those cases where these
particular interactions change significantly. With suitable segmentations in hand, hierar-
chical planning and abstract, action- and event-oriented decision making becomes possible.
Moreover, relevance becomes determined by means of encoding events, event transitions,
and conditional structures, which specify when an event or event transition can typically
take place.
With such behavior-oriented abstractions in hand, finally, language comes into play.
After providing a short overview over the most important language components, Chapter 13
focuses on how language may be grounded in behavior and the abstractions of behavior for
enabling, goal-directed, hierarchical action decision making and control. It turns out that
there are several indicators that Chomsky’s generative universal grammar, may actually be
grounded in an action grammar, from which progressively higher abstractions can develop.
Following the path of language development, we then put forward how language enables
a new means to think about the environment, including other humans, animals, and also
the self, opening up a new level of cognition. Meanwhile, the symbol-grounding problem
is solved by grounding word meanings in the sensorimotor experiences, as well as in the
other, already gathered linguistic experiences. Language and cognitive control by means of
language thus essentially enables to think on a linguistic level, which is closely tied to the
experiences gathered, but which nonetheless enables the detachment of thought from actual,
concrete experiences. Thus, the human mind has come into being.
In the concluding chapter, we highlight several important aspects that should have re-
ceived more detailed coverage. We relate them to the mechanisms, processes, encodings,
and their interactions put forward in this book, and thus hope to put them into perspective.
Finally, we summarize the book’s overall point on how the mind comes into being and dare
a glimpse at highly relevant future research directions.
Another random document with
no related content on Scribd:
cents up, and club lunches, with an ample selection of good things to
eat, for fifty or sixty cents. You may pay more for your room and
bath, but you get more for your money, with better service. We made
it a rule to go to the newest, largest hotels, and indulge in every
comfort that was afforded. Why? Not to be extravagant, nor to say
that we had stopped at such or such hotels. After you have driven
day after day, and come in stiff and tired, there is no bed too soft and
no bathroom too luxurious to overrest your mind and body.
Economize in other ways if you must, but not on good food and
comfortable lodgings.
Our third day was still a drizzle; we would no sooner have the
top down than we would have to put it up again, and often the side
curtains as well. Our objective point was the charmingly quaint town
of Bedford, and the Bedford Arms. This part of Pennsylvania was
more beautiful than what we had been through, and every mile of the
day’s run was a pleasure.
I have not spoken of our lunches, a most important item by one
o’clock. We had brought a small English hamper, fitted with the usual
porcelain dishes, cutlery, tin boxes, etc., for four people, and unless
we were positive that a good place to eat was midway on the road,
we prepared a lunch, or had the hotel put one up for us. This latter
plan proved both expensive and unsatisfactory. Usually Toodles was
sent foraging to the delicatessen shops for fresh rolls, cold meats
and sandwiches, eggs, fruit, tomatoes, and bakery dainties, and the
hotel filled our thermos-bottles with hot coffee. We carried salt and
pepper, mustard, sweet and sour pickles, or a relish, orange
marmalade, or a fruit jam, in the hamper, and beyond that we took
no staple supplies on the whole trip. We met so many people who
carried with them a whole grocery-store, even to sacks of flour, that
you would imagine there was not a place to get food from the
Atlantic to the Pacific. Often later on we would meet these same
people and find that they had thrown or given away most of their
larder. Of course, the camping parties, which are legion, are houses
on wheels! Aside from the tents, poles, bedding, and cooking
utensils, we have seen stoves, sewing-machines, crates of tinned
foods, trunks full of every conceivable incumbrance they could buy,
strapped to the back and sides and even on the top of the car, and
usually the personal luggage jammed in between the mud-guards
and hood of the engine. A traveling circus is an orderly, compact
miniature in comparison. And the people!—sitting on top of a
mountain of baggage, or under it, the picture of woe and discomfort.
That may be fun, but I fear I have not developed a capacity for such
pleasure. Have you ever seen a party of this description unpack and
strike camp after a hot, broiling, dusty day of hard travel? You will do
as we did—drive right ahead until you come to a clean hotel and a
bath.
We have been told so often that one has to develop an “open-
air” spirit to really enjoy a long motor trip! Quite true! I can’t imagine
what the fun can be of touring in a closed limousine, and yet we
have met that particularly exclusive party more than once. On the
whole, an absence of flies, ants, mosquitoes, and sand and dust in
one’s bed and food does not detract from the pleasure of the trip. It
may be all right to endure such annoyances for a few days in the
woods, to fish or hunt—but weeks and more weeks of it! We admit
our “lack,” whatever it may be termed, and enjoy clean linen, hot
tubs, and tables that have legs not belonging to ants and spiders.
In Wisconsin we met a most unique and charming couple, both
past fifty, who had lived all over the world, even in South America, a
Mr. X and wife, from Washington, D. C. They were going on the
same route as we were, and back to Washington, via southern
California, the Yosemite, New Mexico, New Orleans, and then north.
So their trip would be twice as long as ours. They loved the open,
with that two-ton-equipment enthusiasm excelling all others we had
met. From an over-stocked medicine chest, so carefully stowed
away that they bought what they wanted en route rather than unload
everything to try to find it, to a complete wardrobe for every
occasion, which was never unpacked, they had every conceivable
utensil that a well-furnished apartment could boast of. They even
bought a small puppy, as a protection at night when camping; the
poor little beast caught cold and crawled under the pile and died.
They solved the lunch problem in a unique way. If they passed a
good corn-field, they “procured” a few ears and stopped at the next
farmhouse and calmly asked the loan of the kitchen for a short time,
and cooked their corn and bought bread and milk, etc. Mrs. X
remarked: “It is all so simple! We have all these things in case we
should need them, but they are so well packed in the car it is really
too bad to disturb them; so I live in one gown, and we buy what we
need, and it is most satisfactory.” Later we learned that they had
camped out just three nights in several weeks.
But I have digressed, and left you at the Bedford Arms, one of
the most artistic, attractive inns that we found. The little touches
showed a woman’s hand. Flowers everywhere, dainty cretonnes,
willow furniture, and pretty, fine china; in appearance, courtesy, and
efficiency, the maids in the dining-room might have come from a
private dwelling. Will someone tell me why there are not more such
charming places to stop at on our much-traveled main highways.
Why must hotel men buy all the heavy, hideous furniture, the
everlasting red or green carpets and impossible wall-paper, to make
night hideous for their guests—to say nothing of the pictures on their
walls? It is a wonder one can sleep.
There is much of interest to see in Bedford—really old, artistic
houses, not spoiled by modern gewgaws, set in lovely gardens of
old-fashioned flowers, neatly trimmed hedges, and red brick walks.
There were few early Victorian eyesores to mar the general beauty
of the town. As we were walking down the main street about sunset,
we heard a great chattering and chirping, as if a thousand birds were
holding a jubilee. Looking up, we found, on a projecting balcony
running along the front of all the buildings for two blocks, hundreds of
martins discussing the League of Nations and Peace Treaty quite as
vigorously as were their senatorial friends in Washington. They were
fluttering about and making a very pretty picture. It sounded like the
bird market in Paris on a Sunday morning, which, in passing, is an
interesting sight that few tourists ever see.
It was with regret that we left the next morning for Pittsburgh.
The day was clear and cool and the best part of the Lincoln Highway
was before us; in fact, the first real thrill so far, and one of the high
spots of the trip. This was a stretch of seven and a half miles of
tarvia road on the top ridge of the Alleghany Mountains, as smooth
as marble, as straight as the bee flies, looking like a strip of satin
ribbon as far as the eye could see. On both sides were deep ravines,
well wooded, and valleys green with abundant crops, and still higher
mountains rising in a haze of blue and purple coloring, making a
picture that would never be forgotten. The top was down and we
stopped the car again and again, to drink it in, and, as one of us
remarked, “We may see more grand and rugged scenery later on,
but we shall not see anything more beautiful than this”—and it
proved true.
We had come 442 miles, from New York to Pittsburgh, over fine
roads and through beautiful country. Approaching Pittsburgh, we
came in on a boulevard overlooking the river and “valley of smoke.”
Great stacks were belching out soot and smoke, obliterating the city
and even the sky and sun. They may have a smoke ordinance, but
no one has ever heard of it. We arrived at the William Penn Hotel, in
the heart of the business center of the city, a first-class, fine hotel in
every regard. We found the prices reasonable for the excellent
service afforded, which was equal to that of any New York hotel. The
dining-room, on the top of the house, was filled with well-dressed
people, and we were glad that we had unpacked our dinner clothes,
and appeared less like the usual tourist, in suits and blouses. It was
frightfully hot during our two days’ stay. You go out to drive feeling
clean and immaculate, and come in with smuts and soot on your
face and clothes, looking like a foundry hand. The office buildings
are magnificent, and out a bit in the parks and boulevards the homes
are attractive, and many are very handsome, especially in Sewickley.
But aside from the dirty atmosphere one is impressed mostly by the
evidences of the outlay of immense wealth. An enthusiastic brother
living there took us through a number of the business blocks, and
told us of the millions each cost and the almost unbelievable amount
of business carried on. I can only describe Pittsburgh as the
proudest city we visited. Not so much of the actual wealth
represented, but of what the billions had accomplished in great
industries. We went out in the evening and stood on one of the
bridges to look over the river lined with monster furnaces. The air
was filled with sparks, jets of flame bursting through the smoke. All
you could think of was Dante’s Inferno visualized. And what of the
men who spend their lives in that lurid atmosphere, never knowing if
the sun shone, nor what clean, pure air was like in their working
hours? I shall never look at a steel structure again without giving
more credit to the men who spend their waking hours in those hells
of heat and smoke than to the men whose millions have made it
possible.
The second day, nothing daunted by the heat, we went out to the
St. Clair Country Club for lunch and golf, about a twenty-mile run
through the suburbs. This is a comparatively small and new club, but
our host told us that they were soon to have a fine club-house and
improve the links. The location is attractive, and the luncheon was
delicious. We had brought our golf bags, tennis racquets, and
bathing suits with us, much to the amusement of our friends. After
sitting in the car day in and day out, I know of no better way to
stretch your legs and arms and to exercise your stiff muscles than to
put in a few hours at either game. My husband described this course
thus: “You have to hold on to a tree with one hand and drive with the
other, the bally course is so steep.” There are many more
pretentious country clubs and golf links about Pittsburgh, but this
small one had charm and a homelike atmosphere. Our last evening
we were taken to the “New China,” the last word in Chinese
restaurants—beautiful, clean, and artistic! You have your choice of
American or Chinese dishes. As we were looking for sensations, we
ordered some marvelous dishes with impossible names. One portion
was sufficient for three hungry people. The other two portions were
untouched. I do not know what we ate, but it was delicious. Truth
compels me to state that we were all ill for three days, and decided
to patronize home cooking in the future.
We did not get away until noon the next day, as our auto top had
been torn in the garage, and the manager kept out of sight until
noon, and then, after considerable pressure had been brought to
bear, he made a cash settlement of fifteen dollars, wishing us all the
bad luck his “Mutt and Jeff” mind could conjure.
III

OHIO AND DETOURS

We were assured that we should find good roads through Ohio to


Cleveland, where we were to take the D. & C. steamer to Detroit. If
we were to take this part of the trip again, we should certainly go to
Chicago, via Toledo and South Bend, Indiana. As we had relatives in
Detroit waiting in the heat to see us, and to depart for cooler climes,
we took the most direct route through Youngstown, Ohio, to
Cleveland. The roads were poor and the many detours were almost
impassable—over high hills, on narrow sandy roads, winding like a
letter S through the woods. One long stretch was so narrow that two
cars could not pass; so they had two roads, one going each way.
The Doctor remarked, “I wonder what would happen if a car broke
down on this detour.” Prophetic soul! He no sooner had said it than
we rounded a curve, and presto! there were six cars, puffing and
snorting, lined up back of an Overland car, which was disabled and
stuck fast in the sand. In half an hour there were ten cars back of
ours—and the sun setting over the hills, and fifty miles to
Youngstown! The owner of the car knew nothing of his engine.
Heaven save us from such motorists! But Heaven did not save us,
for we met dozens of men, headed for the wilds of somewhere, who
were as blissfully ignorant of what made the wheels go round as
their wives were.
It may have been a coincidence, but is nevertheless a fact, that
nearly every car we saw disabled, ditched, stuck in the mud or sand,
or being towed in, on the entire trip, was an Overland car. It really
became a joke. When we saw a wreck ahead of us, some one
exclaimed, “Dollars to doughnuts it is an Overland!”—and it generally
was. It used to be a common expression, “If you wished to really
know people, travel with them.” I would change it to “Motor, and grow
wise.” There were as many varieties of dispositions in that belated
crowd as there were people. Everyone got out of his car and went
ahead to the wreck, offering advice, growling, complaining, and
cursing Ohio detours. A few sat on the roadside and laughed,
chatted, or read the papers. As it was hot and dusty, we looked like
an emigrant train. My husband is an engineer with a knowledge of
cars. He suggested some simple remedy which enabled the man to
get his car to the next siding, and we all started with a whoop of joy
on the wretched road, leaving the Overland owner to spend the night
at a farmhouse near by.
Our troubles were not over. With a steep grade before us, I was
driving, going up steadily on second speed, when a real wreck
loomed up three-quarters of the way to the top of the hill. Two
drunken niggers had upset a rickety old truck loaded with furniture in
the center of the road, and their car had zigzagged across the road,
narrowly escaping a plunge down the steep embankment. You could
not pass on either side; so, with my heart in my mouth, I reversed,
backing our car into the farther side of the road, with two wheels in a
deep stony ditch, but safe from sliding down-hill on top of the cars
coming up back of us. It looked as if we were to share the fate of our
Overland friend and stay there indefinitely. We all jumped out and
tried to clear the house and lot out of our way. Those miserable
niggers just sat on top of the débris and refused to work. After
tugging at spring beds and filthy bedding, we succeeded in getting it
pushed to one side. I had had enough driving for one day, so gave
the wheel to my husband, and he started the engine. We did not
budge! The next half-hour was spent in filling up the ditch with
stones and making a bridge by covering the stones with boards.
Eventually the car started, pulling itself out of the slough of despair,
and narrowly escaped turning turtle. The Doctor, Toodles, and I all
called wildly, “Keep going! don’t stop!”—and on he climbed to the
top, while we trudged up through the dust, a quarter of a mile. All
that night I dreamt I was backing off the Alps into space.
Oh, what a tired, dirty party it was that drove up to the Ohio Hotel
in Youngstown that night! Someone had told us that there was a
good hotel in Youngstown, but we soon came to form our own
conclusions about hotels. This was a delightful surprise. Not only
good, but wonderful, for a city of the size of Youngstown. After we
were scrubbed and sitting down to a delicious dinner in the big cool
café, a broad smile spread over the table, and the Doctor suggested,
“You know, it really might have been worse!”
The next day we had more detours; but, in the main, the state
highways, when they could be traversed, were good. The rural
scenery through Ohio was pleasing, but we had left the Lincoln
Highway and the beautiful farms of Pennsylvania.
We reached Cleveland by four, driving directly to the D. & C.
wharves. The “Eastern States” was being loaded, and the monster
“City of Detroit III,” a floating palace, was starting out for Buffalo, I
believe. Although the week-end travel is always heavy, and this was
Friday, we were most fortunate in getting staterooms, with brass
beds (not bunks), running water, and a bathroom. It may be of
interest to state that the cost of shipping the car to Detroit, a night’s
run, was only $14.50. As we did not sail until nine o’clock, and we
could not go aboard nor leave the car, we drove out the Lake Shore
drive overlooking Lake Erie, through beautiful suburbs, with
attractive homes and gardens, and then something told us it must be
time to investigate the hotels. As we had all sampled the excellent
cooking at the Statler, we dined at the fine Cleveland Hotel—modern
in all its appointments, in good taste, and unexcelled service. We
remarked the appearance of the people. There was not a smartly
gowned woman in the dining-room, and the waiters had a monopoly
of the dress suits. Being hot, and in midsummer, and a more or less
transient gathering, might have been the reason. In many large
cities, in first-class hotels, we found the tired business men in
business suits and the women in skirts and blouses. Never did
anything taste more delicious than the broiled fresh whitefish, just
out of the lake, green corn on the cob, melons, and peaches. As long
as we remained in the Great Lakes region, we reveled in the
whitefish, broiled, sauté, or baked. It is the king of fresh-water fish.
I am beginning to realize that I am exhausting my descriptive
adjectives when it comes to hotels. Time was, not so very far distant,
when a hotel like the Cleveland was not to be found, except in
possibly half a dozen cities in this country. Now it is the rule. On all
our long trip, with the exception of three nights, we had perfectly
comfortable, clean double rooms, usually with twin beds, and private
baths with modern sanitary plumbing and an abundance of hot, not
tepid, water. We have been assured by the proprietors that the
change has been wrought by motorists who demanded better
lodgings. I think the farmer is the only member of society who still
holds a grudge against us as a class; but when he is the proud
possessor of a “Little Henry” he slides over to our side
unconsciously. A book could be written on “Motoring as an art, a
profession, a pastime, a luxury or a necessity, a money maker or a
spender, a joy or a nuisance”—and then much more!
Before leaving Cleveland I must speak of its fine municipal
buildings, its many industries, and its far-famed Euclid Avenue, once
the finest of streets, lined on both sides with massive, splendid
residences, many with grounds a block square; alas! long since
turned into boarding-houses, clubs, and places of business—the
inevitable transition from a small to a great city.
Our trip across Lake Erie was quiet and cooling. That is not
always the case, even on such big steamers as the D. & C. line
affords. I have seen that lake lashed into fury by waves that rocked
the largest boat like a cockleshell. Breakfast on the steamer was all
that could be desired. It was some time before we had the car on the
dock, ready to start to our hotel in Detroit. The ride up the river had
been interesting, past old Fort Wayne, the Great Lakes engineering
plant and dry docks, and the grain elevators; even at that early hour
(seven A. M.) the wharves were alive with the bustle of trade.
Here I pause. Detroit was my home city and that of my father
and grandfather in territorial days. My earliest recollections of it were
of broad streets, fine homes, and an atmosphere of dignified culture
and home-loving people. But now! It has outgrown recognition. It has
outgrown every semblance of its former charm. Like Cleveland, the
old homes on the principal avenues are all given over to trade, and
the streets down-town are overcrowded, noisy, and well-nigh
impassable. The Statler is a new and fine hotel. We went to the
Pontchartrain, formerly the old Russell House, which in its palmy
days, in the Messrs. Chittenden régime, was the center of the social
life of Detroit. It has passed through several hands, and is now
doubtless torn down. We found it run down and undesirable in every
way. Even then we felt more at home there and made the best of
things. We spent two and a half days, as hot as I ever experienced.
The nights were so hot that sleep was out of the question. A drive
around the Island Park, Belle Isle, cooled us off a bit. Thousands
were taking advantage of the municipal bathhouses or a swim in the
river.
If the city has been spoiled down-town, it has been equally
beautified in the outlying sections. The drive to Grosse Pointe along
Lake St. Clair has ten miles of residences unsurpassed in America.
The magnificent home of Senator Truman Newberry and dozens of
others that could be mentioned, set in acres of highly cultivated
grounds, commanding an unobstructed view of Lake St. Clair, are
worthy of a special trip to Detroit to see.
We lunched at the Country Club, but weakened when it came to
trying the celebrated golf links. It was too boiling hot! There were not
more than a dozen people at the club. Usually the place was
crowded. There are other fine clubs and links about Detroit, and the
city seems to have gone golf mad—a very healthful form of insanity!
The Detroit Athletic Club, in the business center, claims to be the
finest private city club in America. If patronage is any indication of its
excellence, this must be true. My brother, Mr. L., gave us a beautiful
dinner there, and we certainly have not seen anything to surpass it.
Our time was all too quickly spent, and the heat literally drove us out
of town. Before leaving, we paid our respects to the mayor, Mr. C.,
an old-time friend. While we were pleasantly chatting with him and
he was graciously offering us the keys of the city, my husband had a
summons served on him and the car locked for leaving it more than
an hour at the curb. He was taken to police headquarters and paid
his fine and then returned for us. As we were praising the efficiency
of the mayor, he gave us a knowing smile, and some days later
showed us his summons!
IV

ON TO CHICAGO

I realize that I am giving a most unsatisfactory picture of the


Eastern and Middle-West cities. Our time was limited, and space
forbids my giving anything but a cursory glance, a snapshot view, of
their size and beauty. And, then, most tourists visit these places and
the reading public have an intimate knowledge of them.
We left Detroit, having been told at the Michigan Automobile
Association that we should find excellent roads. As one prominent
broker remarked, “You can drive the length of the state on
macadamized roads.” Where were they? Surely not the way we
went, the way described in the Blue Book. And let me state right here
that we have never had much faith in that publication, and now what
little we had is nihil! A few miles out of the city we struck a detour
which lasted nearly to Ann Arbor. We had left at six o’clock, and
when we reached the university city all places to dine were closed.
We did not dine. We had pot-luck supper at a Greek restaurant, and
started for Jackson to spend the night. Ann Arbor is a beautiful
place, and the university buildings and fraternity houses are second
to none of all we saw in other states. The road did not improve, and
we arrived at Jackson very late and put up at the Otsego Hotel. It
was crowded, and we were given the “sample rooms,” in which the
traveling-men displayed their goods on long tables. We had
comfortable beds and private baths, but you felt as if you were
sleeping in a department store, with the counters covered with white
cloths. Otherwise, the Otsego is a good hotel, and we were perfectly
comfortable. By the time we were through breakfast, we asked to
have a lunch put up, and were kindly but firmly told that it was nine-
thirty, and the chef had gone home and locked up everything. We
pleaded for some hot coffee and anything cooked that was left from
breakfast. But no, not a sandwich nor a roll could we buy! We met
this condition time after time. If we arrived at a hotel after eight
o’clock in the evening, we were met with the same retort—“Chef
gone and everything closed.” A dozen times and more we were
obliged to go out and forage for supper—“due to the eight-hour law,”
we were always told. As it was nearly ten o’clock, we trusted to luck
to find a lunching-place en route. Fortune certainly favored us in the
most unexpected way—not in our roads, which still were poor, but in
the shape of two little girls on the wayside. As we were passing
through a hamlet called Smithfield—before reaching Albion—we
were attracted by two dainty girls with baskets of goodies waiting for
us. Their names were Evelyn and Willetta Avery, and they proved to
be fairy godmothers. Their mother owned the neighboring farm, and
these children were spending their vacation in supplying lunches to
passers-by. Everything was done up in fresh napkins and was real
home cooking. This is what we bought from them: a quart of fresh
blueberries (which Toodles, in her joy, promptly upset in the tonneau,
and we walked on blueberries for days!), fresh cake, pie, honey,
hard-boiled eggs, tongue sandwiches, hot bread and rolls, a pat of
sweet butter, and oh! such home-made pickles, raspberry jam (a pint
glass), and a bottle of ice-cold spring water, an abundance for four
hungry grown-ups, and all for $2.10. We gave them both liberal tips
and they smiled and waved us out of sight. That was a banner
luncheon, and the best but one on the trip.
We stopped in the interesting city of Albion. The college was
founded and endowed by General Fisk, of Civil War fame, whose
only daughter, Mrs. P., is one of New York’s most beautiful and
prominent women. That afternoon about four we came to Battle
Creek, and as the Doctor’s eyes were troubling him, from the heat
and dust, we drove to the sanatorium, where he could receive
treatment. It is an immense place and beautifully kept up. We were
sitting in the car outside, watching the crowds of patients with their
friends, when a number of wagons, like popcorn wagons, came into
view, pushed about by the white-robed attendants. The wagon itself
and the four uprights were covered with white cloth and festooned
with fresh vines and flowers. In the center, hidden from view, was an
ice-cream freezer, and young girls in white, carrying flowers, were
dispensing ice-cream cones at five cents each. It was as pretty a
sight as I ever saw. The carts were wheeled through the grounds
and everyone, sick or well, indulged. It was our first introduction to
ice-cream cones, but we acquired the habit; and thereafter our
afternoon tea consisted of ice cream, generally bought at a soda-
water fountain in some small town along our road. It may be
fattening, but it is nourishing and refreshing. Even in the tiny hamlets
on the plains of Montana we found good, rich ice cream. It is
certainly an American institution and a very palatable one.
We had come ninety miles over bad roads, and it was 160 miles
to Chicago, so we decided to stop at Paw Paw for the night. We
drove through the town and inquired which was the best hotel—our
usual question—and were told that they had two, but the Dyckman
House was first-class—a typical small country hotel, with little
promise of comfort. We were shown into big, comfortable rooms with
one private bath; but were told that “supper was over.” The manager
was a typical small-town person of importance, but had a kindly eye,
and looked amenable to persuasion. The others had given up hope;
not so with me! Then and there I invented a “sob-story” that would
have melted Plymouth Rock. It became our stock in trade, and many
a supperless night we would have had without it. After praising up
the town and his hotel, and saying that we had heard of its
hospitality, and so forth; that we were strangers, and had come all
the way from New York; that we were tired and hungry, and I really
was not very well; and that the price was no consideration, etc., he
walked out to the kitchen and caught the cook with her hat on ready
to depart, gave his orders, and in twenty minutes we were doing full
justice to a perfectly good supper. After we had finished, I went out
into the summer kitchen and found a good-natured Irish woman, as
round as she was pleasing, fanning herself. I gave her a dollar,
thanked her for staying, and made a friend for life.
Even in Michigan our New York license attracted much attention.
When we came out of a hotel or store, a crowd of people had
invariably gathered about the car and were feeling the tires. The size
seemed to astonish them. The fact that we had come from New York
filled them with awe, and when, in fun, we said we were going to San
Francisco, they were speechless! “Aw, gaw on!” or “By heck!” was all
that they could exclaim.
Our last taste of Michigan roads was worse than the first. We
went by the way of Benton Harbor, with sandy detours and
uninteresting country, until we struck the strip of Indiana before
coming into South Chicago. Our troubles were over for a long time. A
breeze had come up from the lake, and we slept under blankets that
night for the first time in two weeks. We were all familiar with
Chicago, and we wished to stop out on the Lake Shore, if possible.
We drove through the city, out on the North Shore Boulevard to the
Edgewater Hotel, of which we had heard charming reports. A block
below the hotel cars were parked by the dozens. It is built directly on
the shore, with the most remarkable dining-room at the water’s edge,
like the deck of an ocean liner, filled with palms, flowers, and smartly
dressed people, many in evening clothes. The tables were all
reserved, and so were the rooms, two weeks in advance—this was
the pleasant news that awaited us! Could they take us in the next
day? “No, possibly not for a week or more.” No “sob-story” to help us
here! But the clerks were obliging and advised our going about ten
miles farther out, to the North Shore Hotel in Evanston, which we
found delightful in every way—very near the lake, quiet, furnished in
exquisite taste, and good food at reasonable prices. But even here
we found the eight-hour law in force; we could not get a bite after
eight o’clock. We went to half a dozen restaurants—all closed! In
desperation we went into what looked to be a candy store, and found
they were closing up the café! They could serve nothing but ice
cream and sodas. We asked to see the manager and told him our
plight. He was an Eastern man, a long-lost brother. He said, “As you
placed your order just before eight o’clock, of course we shall serve
you.” It was quite nine by this time. He kept his face straight, and we
tried to do the same. That dinner certainly did touch the spot! It was
the “Martha Washington Café,” and certainly immortalized the
gracious lady for all time for us. Later we went back to the
Edgewater Hotel for our mail and to dine, and we were more
charmed with it than before.
We had come 1028 miles from New York. Our car had to be
thoroughly cleaned, oiled, and looked over; so we were without it for
two days. The street-car strike was on in full force, not a surface car
moving in the city. Consequently, we walked, rested, and saw but
little of the city. It was quite ten years since any of us had been there;
in that time Chicago had grown and been so improved that we hardly
knew it. If Pittsburgh people are proud of their city, Chicagoans are
the original “boosters.” Nature has done so much for its location. Its
system of parks and boulevards is not equaled by any city. There is
a natural, outspoken pride evinced by the people of the best class—
not ashamed of a humble beginning, but glorying in the vast
importance of the commercial and financial life. To quote from the
folder of the Yellowstone Trail, which we picked up here and followed
without any trouble to St. Paul, Minnesota, “Nothing need be said
about Chicago. Chicago is the heart of America and speaks for
herself.” Other cities may challenge this, but there is every evidence
of its truth. In time, Chicago will give New York a good race; in fact,
she is doing it now.
Our genial Doctor left us here, much to our regret. We went on, a
select party of three.
V

THROUGH THE DAIRY COUNTRY

“A good road from Plymouth Rock to Puget Sound.” Thus reads


the Yellowstone Trail folder. If you really believe a thing, you may be
excused for stating it as a truth. The trusting soul who wrote that
alluring statement has never been over the entire trail, or I am
greatly mistaken. Credit must be given for the system of marking the
trail. At every turn, right or left, the yellow disk is in plain sight.
On leaving Chicago, we went through Lincoln Park and up the
Sheridan Road to Milwaukee. The road is a wonderful boulevard,
with beautiful homes and estates and glimpses of Lake Michigan,
past the Great Lakes Naval Training Station, now the largest in the
United States. We had heard much of Zion City. Driving down its
main street was like a funeral. The houses were closed, the buildings
seemed deserted, and the only evidences of life were two men, a
horse and wagon, and a stray dog! We found a good macadam road
to Oshkosh from Milwaukee and many such stretches through
Wisconsin. At times the road followed closely the shore of Lake
Winnebago, and then would wind through fertile dairy country.
Trainloads of butter and cheese are shipped from here each year,
and high-bred dairy cattle are raised for the market. Was it not
strange that we did not have Wisconsin cheese on the menu at any
hotel in that state? Several times we asked for it, but no cheese was
forthcoming.
The first night we put up at Fond du Lac, at Irvine Hotel. It was
fairly good, but a palace compared with what we found the next night
at Stevens Point—the Jacobs Hotel. This was our first uncomfortable
experience—a third-rate house, with no private bath, hard beds in
little tucked-up rooms, a bowl and pitcher with cold water and two
small towels the size of napkins, and the most primitive table you
could imagine. The weather had kept cool and clear, but the sandy
roads with deep ruts were awful! As it had rained in the night, the
clerk assured us next morning that four cars were stuck in the road
west of the town, and we had better not start. We asked him if there
was a good hotel at Marshfield. “Good hotel! Well, you folks just wait
till you see it! They actually have Brussels carpet on the floor of the
dining-room! Good hotel, eh? Nothin’ better this side of Chicago!”
The cars were lined up in the street waiting to start. The clouds
looked heavy and threatening, and not a ray of blue sky. Everyone
was talking to someone. The formalities are discarded on such
occasions. We fell into conversation with a charming man, Mr. H.,
from Fargo, North Dakota. Later we found that he was the ex-
governor, and his name was sufficient to get anything you wanted in
the Northwest. He and his family were touring to New York; so we
exchanged maps and experiences, and he gave us a list of towns
and hotels that proved invaluable, with the kindly remark, “If you will
show the hotel clerks this list with my name, I am sure you will be
well taken care of.” We certainly were—and more!—from there to
Yellowstone Park.
We found the Blodgett Hotel at Marshfield—with a really, truly
carpet in the dining-room—a good hotel, clean and comfortable. The
next day we had two hundred miles to go to St. Paul, and were
promised good roads. Colby, Eau Claire, and Chippewa Falls are all
attractive towns. Wisconsin boasts of six thousand lakes. It certainly
is a paradise for the huntsman and the angler—“The land with charm
for every mile.” The method of numbering the state highways is the
best we have found. You simply can’t lose your way. We,
unfortunately, had several long detours and did not reach St. Paul
until one A. M., a very sleepy trio, in a disreputable-looking car.
VI

CLOTHES, LUGGAGE, AND THE CAR

We decided to take as little luggage as possible. In the end, we


found that we had more than ten people would need. Each of us had
a large dress-suit case, a small handbag with toilet articles, an extra
bag for soiled linen (which proved useful), two golf-bags, with
umbrellas and rubbers (which were never used), a case of tennis-
rackets and balls, a shawl-strap with a heavy rug, rain-coats and top-
coats for cold weather, the lunch-hamper, and a silk bag for hats.
The tonneau was comfortably filled, with still room for two, and even
three, people. The thermos-bottles were stowed away in the side-
pockets, easy of access. All the maps were in the right-hand front
pocket by the person sitting with the driver. We had an old rug which
was so disreputable that no one would steal it; we had been on the
point of throwing it away a dozen times, but after it came from the
cleaners we hadn’t the heart to leave it behind. That old relic proved
to be the joy of the trip. We sat on it when lunching on the roadside,
used it to protect the car from the bags and golf-clubs, and when we
had a puncture down it went under the car to avoid collecting all the
dust of the road on my husband’s clothes. We still have it, and
consider the old veteran deserves a pension for life. My advice—
take an old rug!
And our clothes: Of course, a silk or an alpaca dust-coat; linen
soon shows soil and looks mussy. This applies to the ladies. I won’t
attempt to advise men, for they will wear what best suits them. We
wore one-piece gowns of serge, and, when it was hot, voile or even
gingham. We each had a silk afternoon frock, which would shake out
and look presentable for dinner, a black evening gown for dress-up
occasions, a half-dozen crêpe de chine blouses, and a cloth suit. We
could have done without the suits. They were used but once or
twice. We all took heaps of under-linen, only to find that we could get
one-day laundry service in any good hotel, and could buy almost
anything in the cities, and even in the small towns. The color of our
linen resembled coffee at times, but, aside from that unpleasant
feature, we could keep clean and comfortable with no trouble. We
each had a sport skirt, a sweater, shoes, a pair of evening pumps, a
pair of heavy top-boots, and two pairs of Oxford ties, black and tan,
with sensible heels. In driving, I soon found the long-vamp, pointed
toe not only a nuisance, but dangerous, and used an old-fashioned,
round-toed low shoe. Hats! There every woman is a law unto herself.
We each had a good-looking hat in the hat-bag, which, after being
tied to the rug-rail, sat on, smashed by the bags, and wet a few
times, still kept our hats very presentable. Straw hats will break and
be ruined. Those made of ribbon or black satin will withstand the
weight of a ton of luggage and come out looking fairly decent. Wash
gloves proved practical, also white Shetland veils. Toodles was
swathed like an escaped harem beauty; but one good Shetland veil,
well tied and pinned in, kept my sailor hat in place comfortably, even
when the top was down and I was driving. The hat with a brim is a
necessity when the sun shines for weeks at a time. I did not wear
motor goggles, but the others did. Through all the Western states we
found the female population in khaki breeches and puttees, khaki
blouses, and hats like a sun-bonnet or a cowboy’s sombrero, and
occasionally a coat to match, which was short and of a most
unbecoming length. Often high tan boots were substituted for the
puttees. It was a sensible costume, and well adapted to the country
and life in the open that Western women lead. They all rode astride,
wisely. Often we met parties of four in a Ford just hitting the high
spots on the road.
The farther we went into the real West, the West of the movies
and the early days pictured by Bret Harte, we realized what part
these Western women had played, and were still playing, in their
unselfish, brave, industrious, vital lives, in the opening and
developing of that vast territory, and in making such a trip as ours
comfortable, safe, and even possible. I think, if I ever take the trip

You might also like