Cognition

Psychology
--- Study of mind and behavior
Cognitive Psychology
--- Study the inner workings of human mind
--- Study the cognitive system to understand mind and behavior
--- Study the mind as an information processor
Cognition/Cognitive system
--- All mental processes related to information acquisition, processing, storage, retrieval, and control.
 Behavior Cognition/Mind Brain

• Movement • Perception • Neuron

• Speech • Memory • Connection
• Facial Expression • Feeling/emotion • Structure/Anatomy
• Self-report (verbal, written) • Consciousness • Physiology/Function
• Cognitive performance • Attention • Nutrition, metabolism
(reaction time, accuracy) • Motivation • Tangible material
• Academic performance • Intelligence/Cognitive • Biological, chemical,
• Complex skills (e.g., abilities physical processes
gymnastics) • Self-control • …
• Collective social behavior • Meta-cognition
• … • …
1900 Behaviorism
Psychology should focus on studying observable information (stimulus,
behavior), not on unobservable construct assumed to exist in the mind.

Cognitive psychology
1950
Study of our mind and behavior can be greatly supported by establishing
cognitive constructs and system.

1970 Cognitive neuroscience

Our mind and behavior can be understood through the working mechanisms
of neural systems.
 Cognitive approach to understanding mind and behavior
• Information processing system

Bottom-up
process
Top-down
process

An early, simplistic model

 Cognitive approach to understanding mind and behavior
• Information processing system
Serial processing
input A B C output

Parallel processing
Bottom-up A
process
input B output
Top-down
process C

Cascade processing

input A

C output
An early, simplistic model
Perception
• Sensory information is processed into perceptual experience
• Different from sensation
• Sensation: passively receiving information input by
sensors
• Perception: awareness of the received signals
Basic neural pathways and processes before perception
• A simplified version
Cortex
Hearing

Vision Balance
Photoreceptor

Hair cell

Smell Olfactory receptors

Thalamus
Nociceptor Pain

Taste bud Mechanoreceptor

Touch
Taste Thermoreceptor

Proprioception Temperature
Basic neural pathways and processes before perception
• A simplified version

External stimulus Receptors Neural signal Neural signal

(electrical) in the (electrical) in the
peripheral brain (mainly
Energy nervous system cortices)
transduction

…
Sensation Perception

Bottom-up

This bottom-up model has some limitations:

• It cannot explain the active control of the variation of
perceived content.
• It cannot explain illusions.
• It cannot explain experience-dependent perception
 Central
processing
Top-down
feedback
Existing
Memory

Bottom-up
signal

Sensation
Top-down control explains:
• Active variation of perceived Central
processing
content.
• Illusions. Top-down
• Experience-dependent perception feedback
Existing
(Expertise-dependent perception) Memory
• Individual differences in
perception. Bottom-up
signal

Sensation
• Active variation of perceived content.
• Illusion
Perception that deviates from reality
• Illusions due to top-down process.
• Experience-dependent perception (Expertise-dependent perception)

Demo here by handwriting aesthetics

• Individual differences in perception.

• Different individuals may have (very) different perception to

identical stimulus
Theories emphasizing top-down processes in object recognition

Bar et al.’s model

Bar, M., Kassam, K. S., Ghuman, A. S., Boshyan, J., Schmid, A. M., Dale, A. M., ... & Halgren,
E. (2006). Top-down facilitation of visual recognition. Proceedings of the national academy of
sciences, 103(2), 449-454.
Theories emphasizing top-down processes in object
recognition

Interactive-iterative framework

Baruch, O., Kimchi, R., & Goldsmith, M. (2018). Attention to distinguishing features in
object recognition: An interactive-iterative framework. Cognition, 170, 228-244.
Face recognition

• Face recognition v.s. general object recognition

• They are different
• Face inversion effect: faces are much harder to identify
when presented inverted or upside-down rather than
upright
• Face recognition requires specific neural region
• Fusiform face area (FFA)
• Face recognition is strongly relying on holistic processing
• Part-whole effect – it is easier to recognize a face part when
presented within a whole face rather than in isolation.
• Prosopagnosia
Face recognition

• Prosopagnosia
• Face blindness
• Inability to efficiently process face information

Busigny, Thomas, et al. "Holistic perception of the individual face is specific and
necessary: evidence from an extensive case study of acquired
prosopagnosia." Neuropsychologia 48.14 (2010): 4057-4092.
Face recognition

• Fusiform face area

Weiner, Kevin S., and Kalanit Grill-Spector. "The improbable simplicity of the
fusiform face area." Trends in cognitive sciences 16.5 (2012): 251-254.
Face recognition

• Expertise hypothesis
1) Holistic or configural processing is not unique to faces but should be found for any objects of
expertise.
2) The fusiform face area should be highly activated when observers recognise the members of
any category for which they possess expertise.
3) If the processing of faces and of objects of expertise involves similar processes, then
processing objects of expertise should interfere with face processing.
Face processing model

Bruce, V., & Young, A. (1986). Understanding face

recognition. British journal of psychology, 77(3), 305-327.
Mental processes related to perception

• Mental imagery
• Perceiving things without their actual presentation
• Different from actual perception in
• Level of details
• Precision
• Stability
• Hallucination (to be differentiated from illusion)
• Experience of real perception when the actual perceived
contents do not exist
• Dreaming
Memory
Major stages of memory-related activity from an information-
processing point of view:
reinforcing

Encoding Storage Retrieval

forgetting
Memory limited

Unlimited?
Sensory memory

Short-term memory
Tongue Twister
Working memory She sells seashells by the seashore,
The shells she sells are seashells, I'm sure.
So if she sells seashells on the seashore,
Long-term memory Then I'm sure she sells seashore shells.

Non-declarative memory (implicit)

Declarative memory Semantic memory

Episodic memory
Sensory memory

• Temporary maintenance of sensory

perception after the stimulus has been
withdrawn.

• Visual: up to a few hundreds of

milliseconds

• Auditory: up to a few seconds

Short-term memory

Sensory Short-term Long-term

Input
stores store Store

Multi-store model, Atkinson and Shiffrin (1968)

• Temporary memory
• How long is short-term?
• Typically referring to the duration from a few seconds to hours
• Limited capacity
• The magic seven (George Miller, 1956)
• Highly dynamic and fluid
• Constantly updating and forgetting

Atkinson, R. C., & Shiffrin, R. M. (1968). Human memory: A

proposed system and its control processes. In Psychology of
learning and motivation (Vol. 2, pp. 89-195). Academic Press.
The magic number seven (George Miller, 1956)

• The number of items that can be held in short-term memory is about seven
• Differs from people to people
• Examples of violation:
• 1111111
• 1234567/ABCDEFG/KEYWORD
• So, here items refers to “chunk” of information George Armitage Miller
• A chunk is a memory unit that consists of several components that are (February 3, 1920 – July 22, 2012)

strongly associated with one another

• Chunks are largely independent from each other
• So the following two series contains similar amount of information (both
are roughly 7 chunks:
• 3-9-2-1-8-4-0
• 33-99-22-11-88-44-00
• Similar to compressibility of an image.
Working memory
• A system, or a set of processes, holding mental representations
temporarily available for use in thought and action

• Short-term memory focuses on storage, working memory

focuses on the operation of information Alan Baddeley and Graham Hitch

The woman at the park was at least 40

The woman at the table in the park next to the lake in the countryside was at least 40
 Working memory

Central
executive
Alan Baddeley and Graham Hitch
Shape Object Kinesthetic Tactile Music
Smell Taste
and
Visual Spatial Haptic Speech Lip-reading sound Central executive: central information
and attentional control
Episodic buffer Phonological loop: processing and
Visual-spatial Phonological

Artic
sketch pad loop storing information briefly in a
phonological (speech-based) form.
Visuo-spatial sketchpad: temporary
storage for spatial and visual info.
Episodic buffer: temporary storage for
Visual semantics ßà Episodic long-term memory ßà Language integrated information coming from
the visuo-spatial sketchpad and
phonological loop
Working memory capacity (WMC)
• The ability to hold and manipulate information in a temporary active state
• Example of measurement:
• n-back task: press button when the current symbol is the same with the n-th previous one
d h c i b o h m d h q d o n b l k s d k w x (n=3)
• reading span task: read sentences aloud and recall the final word of each sentence
• backward digit recall (4532 -> 2354)
• dot matrix recall

• Working memory capacity correlates positively with intelligence, particularly, fluid intelligence
• Crystallised intelligence: knowledge, skills and experience
• Fluid intelligence: the ability to solve novel problem with minimal reliance on knowledge

Fluid intelligence r = 0.8

Intelligence WMC
Crystallised intelligence low

• Variability in individual’s working-memory capacity is strongly predictive of individual’s academic performance.

Long-term memory
• Declarative memory
• Conscious recollection of events and facts
• Episodic memory (EM): personal experiences of events and their contexts
• Semantic memory (SM): knowledge about the world, concepts, language
• Difference between EM and SM:
• Associated with different neural anatomical systems (EM and SM can be separately damaged)
• EM: one-time experience; SM: re-visitable experiences

• Non-declarative memory
• Does not involve conscious recollection
• Measure by observing changes in behavior
• Occupies a very significant part in our life learning, especially reflected in skill learning
• Driving, musical instrument playing, language (particularly spoken), walking(!)
• Put the video here
• Implicit learning, implicit memory
Attention
Attention is . . . the taking into possession of the mind, in clear and vivid
form, of one out of what seem several simultaneously possible objects or
trains of thought. Focalisation, concentration, of consciousness are of its
essence.
---- William James

Selectivity of processing

William James
(January 11, 1842 – August 26, 1910)

à selective attention test

Attention
According to voluntariness
• Passive attention
• Active attention

According to selectivity
• Focused attention
• Divided attention

According to target of information

• External attention
• Internal attention

According to eye-movement
• Overt attention
• Covert attention
Attention
• Passive attention: controlled in a
bottom-up way by external stimuli
- e.g., loud sound, saliency, novelty,
unexpectedness, attractiveness

• Active attention: controlled in a top-

down way by the individual’s goals or
expectations
- competition between active attention
and passive attention
Attention
• Focused attention (or selective attention):
focus only on a specific stimulus input while ignoring others.
• Divided attention:
simultaneously focus on two or more stimulus inputs
Attention
• External attention:
the selection and modulation of sensory information
e.g., Stroop task

• Internal attention:
the selection, modulation, and maintenance of internally generated
information
Try to silently count from 1 with a step of 1 and switch between two rules:
1) find number that is multiples of 7;
2) find number that starts with a vowel (e.g., eighteen)
Whenever either rule is satisfied, report the number and switch to another rule.
 Attention
• Overt attention:
Paying attention to a specific position with eyes being directed to and focus
on the position.
• Covert attention:
Paying attention to a specific position without fixating your eyes on it.

Posner paradigm (1980)

Theory of attention
• Spotlight model
The coverage of attention resembles a spotlight pattern: the
center of spotlight receives more attention/processing resource,
while the peripheral areas receive less/no attention.
• An extended version: zoom lens theory
The spotlight area is adjustable in size
 Theory of attention
• Split attention
Attended areas can be composed of discontinuous areas
(Note that split attention is conceptually different from divided attention mentioned earlier)

Experimental design
and evidence:

Awh, E., & Pashler, H. (2000). Evidence for split

attentional foci. Journal of Experimental Psychology:
Human Perception and Performance, 26(2), 834.
Retinal structure account of visual attention
• Retina hosts two types of photoreceptors:
• Rods and cones
• 5 million cones and 92 million rods
• Rods are more sensitive to light (for night vision), but color blind
• Cones are separated into three types sensitive to three different colors

Bear, M., Connors, B., & Paradiso, M. A. (2020). Neuroscience: Exploring the
Brain, Enhanced Edition: Exploring the Brain. Jones & Bartlett Learning.
Retinal structure account of visual attention
• Retina hosts two types of photoreceptors:
• The majority of cones are distributed on the fovea (a small pit at the center)

Bear, M., Connors, B., & Paradiso, M. A. (2020). Neuroscience: Exploring the
Brain, Enhanced Edition: Exploring the Brain. Jones & Bartlett Learning.
Attention enhances brain responses

Brain response

Stimulus
input

Luck, S. J., Woodman, G. F., & Vogel, E. K. (2000). Event-related potential

studies of attention. Trends in cognitive sciences, 4(11), 432-440.
Major attention networks in the brain
• Attention is a complex function that requires widespread brain
resource, i.e., large-scale neural networks

• Posner (1980) proposed two attention networks

Endogenous system: goal-directed attention network, it is
controlled by the individual’s intentions and is used when
central cues are presented
Exogenous system: stimulus-driven attention network, it
automatically shifts attention and is involved when
uninformative peripheral cues are presented. Stimuli that are
salient or different from other stimuli (e.g., in colour) are most
likely to be attended to using this system.

Posner, M. I. (1980). Orienting of attention. Quarterly journal of experimental

psychology, 32(1), 3-25.
Major attention networks in the brain
• Similarly, Corbetta and Shulman’s (2002) proposed two attention networks in in basic aspects of visual processing:
• A goal-directed/top-down system
• Dorsal attention network consists of a fronto-parietal network including the intraparietal sulcus.
• Influenced by expectations, knowledge and current goals
• A stimulus-driven/bottom-up attention system
• Ventral attention network, consists primarily of a right-hemisphere ventral fronto-parietal network.
• Used when an unexpected and potentially important stimulus (e.g., flames appearing under the door)
occurs.
• it has a “circuit-breaking” function, meaning visual attention is redirected from its current focus.

*No need to memorize this figure Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-
driven attention in the brain. Nature reviews neuroscience, 3(3), 201-215.
What is learning?
From education perspective:
Acquiring new knowledge, skills, experiences, etc.
From cognitive psychology perspective:
Acquiring new memory (declarative and non-declarative)
From neuroscience perspective:
Updating neural architecture
From evolution perspective:
Adaptation
Neural foundation of learning
• Neuron
• The basic unit for information processing in the nervous
system
• There are around 1000,000,000,000 neurons in the brain,
forming a highly complex network
• Neural synapses
• The basic structure of information transmission in the
nervous system
• Neural plasticity
• The properties of neural synapse are changeable in a way
that explains learning
• How?
An abstraction of any kind of information transmission and processing system:

info Node 1 Node 2 Node 3 … Node 5 output

We can find similar architecture in the nervous system:

sensor

output
Neural foundation of learning
Learning is all about the improving the following two
outcomes:

1. Speed
2. Accuracy (decision making)

How does the neural system support these two achievements?

Neural foundation of learning
• One of the main manifestations of learning is that the same
cognitive task is performed faster.
• Try to practice pronouncing this word:
sesquipedalianism
• Other examples?
• Faster performance means faster information transmission
in the neural synapses
• How?
• Neural synaptic plasticity
• Short-term plasticity
• Long-term plasticity
A brief version of neural synapse
General procedure of a synaptic
transmission:
1. An impulse signal arrives at the axon
terminal and activates voltage-gated calcium
channels in the cell membrane.
2. Ca2+ rushes into the cell.
3. The synaptic vesicles move to the
membrane, releasing neurotransmitter into
the synaptic cleft.
4. The neurotransmitter binds to receptor
proteins on the postsynaptic cell.
5. The neurotransmitter activates or
deactivates the postsynaptic cell, depending
on the type of transmitter and receptor.
Short-term synaptic plasticity
• Short-Term Depression (STD)
• caused by depletion of neurotransmitters
consumed during the synaptic signaling
process at the axon terminal of a pre-
synaptic neuron
• Real-life example: smell, touch, etc
• Short-Term Facilitation (STF)
• caused by influx of calcium into the axon
terminal after spike generation, which
increases the release probability of
neurotransmitters.
• Real-life example: “in the process of
something”
Long-term synaptic plasticity
Neurons fire together wire together!
Rules for synaptic modification (Hebb’s rule):
1. When the presynaptic axon is active and, at the same time, the
postsynaptic neuron is strongly activated under the influence of other 1904-1985
inputs, then the synapse formed by the presynaptic axon is
strengthened. This is another way of stating Hebb’s hypothesis: neurons
that fire together wire together.

2. When the presynaptic axon is active and, at the same time, the
postsynaptic neuron is weakly activated by other inputs, then the
synapse formed by the presynaptic axon is weakened. In other words,
neurons that fire out of sync lose their link.
Long-term synaptic plasticity
Neurons fire together wire together!

Long-term plasticity based on Hebb’s rule explains skill learning:

• A complex cognitive procedure presumably goes through the same neural pathway. 1904-1985
• Thus, adjacent neurons in the pathway always fire together.
• The connections along the pathway are enhanced.
• The performance of the cognitive procedure become faster.

Neuron 1 Neuron 2 Neuron 3 Neuron 1 Neuron 2 Neuron 3

Neuron 4 Neuron 5 Neuron 6 Neuron 4 Neuron 5 Neuron 6

Neuron 7 Neuron 8 Neuron 9 Neuron 7 Neuron 8 Neuron 9

Long-term synaptic plasticity

We have now addressed the question of how neural plasticity

enables faster response (one aspect of learning), but how does it
support accuracy, or decision making?
 Untrained network
Long-term synaptic plasticity 1 1×1 + 0×1 = 1

How connectivity pattern enables decision making

0 1×1 + 0×1 = 1

0 0×1 + 1×1 = 1

1 0×1 + 1×1 = 1
×1

×2 Trained network
Input neurons Output neurons
(sensor/receptor) (decision maker) 1 1×2 + 0×1 = 2

0 1×1 + 0×2 = 1
• Assuming the sensor neuron responds to black
pixel (1), not to white pixel (0)
• A neuron network with homogeneous
0 0×2 + 1×1 = 1
connectivity cannot make decision.
• Changing the connectivity can help us build a
decision maker! 1 0×1 + 1×2 = 2
Long-term synaptic plasticity
……..

cat dog dog cat dog dog cat dog dog dog cat dog ….

feed for training

Deep learning AI is
using the same .. ..
.
.
principle. ..
.
..
. output
input layer
layer hidden hidden
layer layer
Can have many hidden layers after training

cat

dog
P(cat) = 0.954
P(dog) = 0.006
.. ..
.
. ..
..
. . output P(cat) = 0.079
input
layer hidden hidden
layer P(dog) = 0.921
layer layer
Can have many hidden layers
Main effects of learning (training) on the cognitive system
• Learning reduces cognitive load for performing a task
• Cognitive load: the amount of brain activity engaged in a task.
• Easier task, lower cognitive load, vice versa.
• Higher skill, lower cognitive load, vice versa.
• Cognitive load is limited (frequently equalized to working memory capacity)
• Extensive learning and practice can induce qualitative change, turning task performance activity from
controlled processes into automatic processes.
• Examples: typing, juggling, speech, walking, musical instrument playing, juggling rubik's cubes while
solving them (basically, all kind of learning)
Automatic processes
• Cognitive activity that are highly efficient, automatic
• Requires minimal cognitive resources, effort, and attention
• Inflexible
• Goal-less
• Examples?
Controlled processes
• Cognitive activity that are goal-directed, under controlled
• Effortful Shiffrin and Schneider
• Requires much cognitive resources, effort, and attention (1977)’s duel process theory
Schneider, W., & Shiffrin, R. M. (1977).
• Flexible Controlled and automatic human information
• Examples? processing: I. Detection, search, and
attention. Psychological review, 84(1), 1.