Advanced Neural Implants

and Control

Daryl R. Kipke

Associate Professor

Department of Bioengineering

Arizona State University

Tempe, AZ 85287

kipke@asu.edu

Approved for Public Release, Distribution Unlimited: 01-S-1097

The Underlying Premise…

The ability to engineer reliable,

high-capacity direct interfaces to the
brain and then integrate these into a
host of new technologies will cause
the world of tomorrow to be much
different than that of today.
 However…

� There are some serious scientific barriers between

where we stand today and where we can stand in
the future.
• How do we establish permanent and reliable interfaces to
selected areas of the central nervous system?
• How do we use these interfaces to directly and reliably
communicate at high rates with the brain?
 Applied Neural Implants and Control

Systems Science & Visualization &

Signal Processing Modeling
He (BME) Farin (CSE)
Hoppensteadt (Math & Nelson (CSE)
EE) Razdan (CSE)
Kipke (BME) Smith (Math) INFO
Si (EE)
Project Director Neural & Tissue Tissue Culture &
Kipke (BME) Engineering Analysis
Kipke (BME) Capco (Bio)
Advisory Massia (BME) Massia (BME)
Committee Panitch (BME) Pauken (Bio) BIO
Raupp, Rousche (BME)
Hoppensteadt,
Farin
Materials Synthesis MEMS
& Bioactive Shen (EE)
Coatings Pivin (EE)
Ehestraimi (BME) Li (EE) MICRO
Massia (BME)
Panitch (BME)
Raupp (ChemE)
 Primary Goals of the

BIO:INFO:MICRO Project

� Develop new neural implant

technologies to establish
reliable, high-capacity, and long-
term information channels
between the brain and external
world. VizMod
SysSci
TisClt

NeuEng MEMS
� Develop real-time signal Mat'lSyn

processors and system

controllers to optimize
information transmission
between the brain and the
external world.
 Systems-level Approach…

Feedback control signals

Subject
Neural system
(global)
local
External Adaptive Neural
World Controller Implant
Controlled
neural
plasticity

Objective 2: Optimize Objective 1: Optimize neural

Adaptive Controller interface
 Topics

� Project overview
� Towards the Development of Next-
Generation Neural Implants (BIO, MICRO,
and INFO)
� Bioactive Coatings to Control the Tissue

Responses to Implanted Microdevices

� Modeling the Device-Tissue Interface

� Direct Cortical Control of an Actuator
� Neural Control of Auditory Perception
� Wrap-up
 Focus on Next-Generation

Neural Implants

Feedback signals: local

host response
Subject
Neural system
(global)
local
External Neural Neural
World Controller Implant
Controlled
Info. Signals: electrical neural
& chemical plasticity

Objective 2: Optimize Objective 1: Optimize neural

Adaptive Controller interface to achieve reliable, two-way,
high-capacity information channels.
…and “self-diagnostic”
 Fundamental Problem of Implantable

Microelectrode Arrays

� Brain often encapsulates the device with scar tissue

� Normal brain movement may cause micro-motion at the tissue-
electrode interface
� Proteins adsorb onto device surface
� Useful neural recordings are eventually lost

Electrode 1

Electrode N

Implant Failure

Month N

Implant Month 1
 3rd-Generation Neural Implants

Technology
Spectrum

1st-generation 2nd-generation 3rd-generation

Microwires Silicon arrays Neural Implants
Desired Properties
• Very high channel count
(<1000)
• Bioactive coatings
• Flexible
• Engineered surfaces
• Controlled biological
response
• Integrated electronics
“Brain-centered” Design of Neural Implants

Initial conceptual designs

recording site through hole

Standard Perforated Probe

Simple Bioactive Probe

Differential Bioactive Probe bioactive gel

e.g. corticosteroid flexible

NGF e.g. GABA
polyimide
substrate

cross-section (A-A) cross-section (B-B)

A A B B
A bioactive gel
B

A B

through hole connecting recording site bond pads

channel
 Polymer-substrate Neural Implants

• 2-D planar devices can be bent into 3-D structures

• Increases insertion complexity

Holes to
promote
integration 90 degree
with neuropil angles
Recordings From Polymer-substrate

Neural Implants

One Day Post-op

Chan. 9

Chan. 10

Lost most unit activity

after 7 days – Most likely
due to failure to properly
close dural opening.
 Flexible Neural Implants Present

Surgical Challenges

� While the “micro-motion” hypothesis suggests that flexible

neural implants should be more stable, the same flexibility
presents significant new surgical challenges.

“Difficult” insertion “Easy” insertion

Rdr2, 9-00 Rdr3, 9-00

 Using Dissolvable Coatings to

Stiffen the Neural Implant

� Dip-coat microdevice with polyethylene glycol (PEG)

• Provides mechanical stiffening prior to implant
• Quickly dissolves when in contact with tissue
First insertion of coated microdevice into Second insertion of coated microdevice
gelatin -- Device easily penetrates into gelatin – The device is too flexible to
material penetrate material because the PEG has
dissolved.
 Micromachined Surgical Devices

Silicon Knife/Inserter

PEG
Vacuum nozzle

Insertion aid

Flexible probe

Vacuum Actuated Knife/Inserter

 Exploratory Functionality

Passive Other Active Devices

Polymer Substrate
Surface Engineering (Thermal, Magnetic, Strain, etc.)

• Magnetic/thermal
stimulation
• Drug delivery channels Bioactive Component
Electrical Active Fluid Microchannels
Storage Structures
Recording/Stimulating FET Devices,
Surfaces ChemFETs
• Active micro-
manipulation of probes

Mechanical Signal
Transfer Structures Processing Termination

Currently...
Internal Review
Feasibility Studies
Insertion Aids
Multiple Dimensions and Forms
 Implant Coatings and Surface Modifications

Parylene-N,C Photo-crosslinked
Cl Polyimides
O O
Cl
C C
N N O
C C
O O n

smooth porous

Surface Plasma Treatments NH2 NH2 NH2 NH2

(NH3 - Amination)
 Advanced Neuro-Device Interfaces

Passive

NH2 NH2 NH2 NH2

Chemical/Electronic
Amplification

ion beam

metal

modified region
site or interdigits
Active
polymer (PI/P-C)
release layer
or substrate

Silicon FETs?

 Topics

� Project overview
� Towards the Development of 3rd-Generation
Neural Implants (BIO, MICRO, and INFO)
� Bioactive Coatings for Controlled
Biological Response (BIO, MICRO, and INFO)
� Modeling the Device-Tissue Interface
� Direct Cortical Control of an Actuator
� Neural Control of Auditory Perception
� Wrap-up
 Approach

Engineer the neural implant surface in order to control

both the material response and the host response.

Advanced biomaterials and

micro-devices for long-term
implants (BIO, MICRO, INFO)

Models and 3-D visualization Cellular and biochemical

of device-tissue dynamics response characterization
(BIO, INFO) (BIO, MICRO)
 Factors Limiting Chronic

Soft Tissue Implants

� Inability to control cellular interactions at

biomaterial-tissue interface
� Initial adsorption of biological proteins
• Non-selective cellular adhesion
� Unavoidable “generic” foreign body reactions
• Inflammation
• Fibrous capsule formation
 Potential Solution

� Engineer surface for minimal protein adsorption

and selective cell adhesion
• Cell-resistant polymer coatings
• Synthetic: Polyethylene Glycol, Polyvinyl Alcohol
• Natural: Polysaccharides, Phospholipids
• Surface immobilization of biologically active
molecules
• Mimic biochemical signals of extracellular matrix
• Cell binding domains for integrin receptors
 Biomimetic Surface Modification

O O O
O O O O O O
O OH OH OH O
HO HO HO OH HO OH HO
OH
HO NH
N OH HO NH
2 N OH
2

NTF NTF

Material Surface

 Recombinant NGF Fusion Protein

Active or inactive plasmin­

Degraded plasmin­

Factor IIIa
degradable substrate
substrate

substrate

Plasmin

Human b-NGF cleavage

Human b-NGF
Fibrin

plasmin
Fibrin
 Bioactive Functionality

Methods 6-hour diffusion in rat cortex

Fluorescence Intensity Profile

250

200

NeuroTrace� DiI tissue-labeling paste,

Pixel Value

150

inverted fluorescent microscope with

100

5 0

FITC/rhodamine filter cube 0

0 2 0 4 0 6 0 8 0

Distance (microns)
100 120 140 160

 Topics

� Project overview
� Towards the Development of 3rd-Generation
Neural Implants (BIO, MICRO, and INFO)

� Bioactive Coatings to Control the Tissue

Responses to Implanted Microdevices (BIO, MICRO,
and INFO)

� Modeling the Device-Tissue Interface

(BIO, MICRO, and INFO)
� Direct Cortical Control of a Motor Prosthesis
� Neural Control of Auditory Perception
� Wrap-up
The Device-Tissue Interface

Neural Interface:
Micro-device, Neurons, Glia, Extracellular Space
The Goal is to Characterize, Predict, and Control

the Device-Tissue Interface

Biophysical
Tissue State Device Function
Model of the
(e.g., encapsulation, Device-Tissue (e.g., impedance
excitability) spectrum)
Interface

• Integrate bioelectrical, histological and biochemical data

• Optimize electrode specifications
Visualization of the Chronic Device-Tissue

Interface With Confocal Microscopy

A B

C D
In vivo Visualization of the Chronic

Device-Tissue Interface

Multi-Domain Continuum Model

• Tissue is two (or more) coupled

volume-conducting media
• Electrode is boundary condition

r
At each "point" r in space:
r
volume fraction fe / i ( r )
r
potential Fe / i ( r , t )
r
conductivity tensor Ge / i ( r )
membrane parameters
a, C, gL , etc.
 Equations for a Multi-Domain

Continuum Model

Volume conductor equations (conservation of current)

- fe� � (Ge�Fe ) = +� I memi + I app

i

- fi �� ( Gi �Fi ) = -I memi i = index over intracellular domains

Membrane potential(s) and membrane current(s)

� ¶Vi �
Vi = F i - F e I memi = ai � Ci + Iioni �
Ł ¶t ł
Fe / i = potential (mV) a i = surface to volume ratio (cm -1 ) Vi = membrane potential (mV)
Ge / i = conductivity (mS/cm) I memi = membrane current (mA/cm ) 3
Ci = membrane capacitance (mF/cm 2 )
f e / i = volume fraction I app = applied current (m A/cm3 ) I ioni = membrane current (mA/cm 2 )
 Levels of Modeling

Numerical
Analytical

Multiple intracellular domains

A single intracellular domain

Voltage-dependent conductances
Passive membrane conductance

I ioni = � g ij � q ijk (Vi - E j )

I ion = g L (V - E L )
j k

¶qijk q - q (Vi )
¥

= - ijk ijk

¶t t ijk (Vi )

Complex electrode geometry Simple electrode geometry

Tissue inhomogeneous and Tissue assumed homogenous and
anisotropic isotropic
under construction much progress
 Bi-domain Model for the

Microcapillary Bioreactor

Write BCs and assume: j = j1eiwt � Fe / i ( x, t) = F1e / i ( x;w )eiwt

100 Hz
Fe
Calculate profiles
Fi
F 1
e/ i ( x;w) V
EL

in bioreactor

...and predict Z (w )
...and impedance...
as tissue parameters
Z (w ) =
Z fe / i , Ge / i ,a , C, g L , EL
F ( L;w) - F ( 0;w )
1
e
1
e are experimentally
j1 manipulated
w
 Recap

� Focused & integrated effort

• BioMEMS…Neural
Engineering…Materials…
Computational BIO
Neuroscience…Cellular
Biology…Visualization
� Why are we so excited?
INFO
• We have the very real
potential of characterizing MICRO
the biological responses to
neural implants and then
engineering new classes
of microdevices to provide
a permanent high-capacity
interface to the brain
 Why the BIO, INFO, and

MICRO Program?

� Wide-open Challenges
• Characterizing and modeling the biological (cellular and
chemical) responses around a neural implant
• Controlling the dynamic biological responses around a neural
implant.
• Designing, fabricating, and using “advanced” neural implants
� Collaboration Possibilities
• Additional functionalities for implantable microdevices of the
class that we are working on.
• Exploring fundamentally new types of tissue-device interfaces.
• Complementary studies of the neural interface (experimental
and analytical)
• Confocal microscopy of the neural interface
• Sharing technologies, procedures, insights, etc…
• New emergent ideas…
 Systems-level Analysis of Advanced

Neuroprosthetic Systems

Feedback control signals

Subject
Neural system
(global)
local
External Adaptive Neural
World Controller Implant
Controlled
neural
plasticity

Objective 2: Optimize Objective 1: Optimize neural

Adaptive Controller interface
 Systems-level Approach for Advanced

Neuroprosthetic Systems

Feedback control signals

Subject
Neural system
(global)
local

External Adaptive Neural

World Controller Implant
Controlled
neural
plasticity

Objective 2: Develop Objective 1: Optimize neural

adaptive controller to interface

optimize system

performance.

 Advanced Neuroprosthetic Systems

External World

Neuroprosthetic System
Sensory
Transduction &
Pre-processing Motor Movement
Commands
High-Level
Sensory
Neural
Integration
Computation
Perception,
Decision,
� Underlying System Principles Detection
•Two-way communication with targeted neural systems

•Harness neural plasticity to our advantage

•Appropriately balanced “wet-side” and “dry-side” computation

 Approach

� Four Project Areas

�Direct neural control of actuators
�Detection of novel sensory stimuli through
monitoring neural activity
�Neural control of behavior
�Investigate signal transformations from
ensembles of single neurons to local field
potentials to EEG.
 Topics

� Project overview
� Towards the Development of 3rd-Generation Neural
Implants (BIO, MICRO, and INFO)
� Bioactive Coatings to Control the Tissue Responses to
Implanted Microdevices (BIO, MICRO, and INFO)
� Modeling the Device-Tissue Interface (BIO, MICRO, and
INFO)
� Direct Cortical Control of a Motor Prosthesis

(BIO, MICRO, and INFO)

� Neural Control of Auditory Perception
� Wrap-up
 Direct Cortical Control of Actuators

External World

Neuroprosthetic
System External Actuator
Goal: Control Robotic Arm or
arm-related Virtual Reality
Sensory actuator
Transduction &
Pre-processing Motor Movement
Commands
Sensory High-Level
Integration Neural
Computation
Perception,
Decision,
Detection
 Fundamental Questions

� What are “optimal” real-time signal processing

strategies for precise 3-D control of external, arm-
related actuators in the presence of sensory
distractions and/or physical perturbations to the
arm?
� To what extent can we use composite neural
signals [neuronal (unit) recordings, local field
potentials, and brain-surface recordings] for control
signals?
� How do we take advantage of inherent or
controlled neural plasticity in order to optimize
system performance?
 Experimental Preparation

• Train monkeys to perform tracking and/or reaching tasks.

• Record cortical responses with multichannel neural
implants.
• Measure arm movement in 3-D space.
 Chronic Neural Recordings
� Multi-channel neural implants in motor and sensorimotor cortical areas.
� Eventually: Sub-dural electrodes for local potentials
Perievent Histograms Target 1, reference = C_rel, bin = 20 ms
dsp009b dsp034a dsp046a

40
10 100
20
0 0 0

Extracellular recordings 40
-0.2 0 0.2
dsp012a
0.4 0.6

150
-0.2 0 0.2
dsp037a
0.4 0.6

15
10
-0.2 0 0.2
dsp051a
0.4 0.6

Offline
100
20 5
50
0 0 0
-0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6
dsp018a dsp040a dsp057a
20 15

Analysis
40
10
10 20
5
0 0 0
-0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6
dsp024a dsp042a dsp058a
20 80

Neural
40
20 10 40
0 0 0
-0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6
dsp025a dsp042b

Recording
40

20 20

0 0
-0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6
dsp030a dsp045a

System 10

0
30
20
10
0
-0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6
Time (sec) Time (sec)

Real-time Actuator
Signal Control
Processing
Direct Cortical Control of Movement

Green ball: Target Yellow ball: Actual hand position, or

hand position estimated from cortical
responses

m0602pa

 Topics

� Project overview
� Towards the Development of 3rd-Generation Neural
Implants (BIO, MICRO, and INFO)
� Bioactive Coatings to Control the Tissue Responses to
Implanted Microdevices (BIO, MICRO, and INFO)
� Modeling the Device-Tissue Interface (BIO, MICRO, and
INFO)
� Direct Cortical Control of a Motor Prosthesis (BIO, MICRO,
and INFO)
� Neural Control of Auditory Perception(BIO,
MICRO, and INFO)
� Wrap-up
 Neural Control of Auditory Perception

External World

Neuroprosthetic
System
Goal: Control
auditory perception
Sensory
Transduction &
Pre-processing Motor Movement
Commands
Sensory High-Level
Integration Neural
Computation
Perception,
Decision,
Detection
 Fundamental Questions

� To what extent can we control auditory-mediated behavior using

intra-cortical microstimulation (ICMS) through the neural interface?

Source Received
Signal Transmitter Channel Receiver Signal
Stimulator Neural Auditory
Interface Cortex
� What are the information transmission characteristics of the
multichannel neural implant in high-level cortical areas using
ICMS?
� Channel capacity (bits per second)
� Channel reliability
� Channel resolution
� How can we optimize information transmission
� Implant designs, Neural implant locations, Signal encoding strategies,
Controlled neural plasticity
 Chronic Neural Recordings

� Multi-channel neural implants in primary auditory cortex

Extracellular recordings
in auditory cortex
Estimation of
Neural Neuronal
Offline Response
Recording
Analysis Properties
System

Algorithm Selection

Sounds
Electrical Signal
Stimulation Encoder
to Aud. Ctx.

Behavioral performance to both sounds and

cortical electrical stimulation
 Auditory Behavior

• Lever-press sound or ICMS discrimination task

• Center paddle hit starts trial, 2-tone pair presented
• Reward obtained by signaling the correct stimulus
sequence

left center right

rat
 Frequency Selectivity in Auditory Cortex

Frequency 80
dsp002a 6.
80
dsp002b 11.
80
dsp010b 24.

response areas 60
3.
60
5.5
60
12.
40 40 40

1 2 5 10 20 30 1 2 5 10 20 30 1 2 5 10 20 30

dsp012a 42. dsp018b 21. dsp018c 44.

80 80 80

60 60 60
21. 10.5 22.
40 40 40

1 2 5 10 20 30 1 2 5 10 20 30 1 2 5 10 20 30

dsp018d 22. dsp020a 20. dsp024a 20.

80 80 80

60 60 60
11. 10. 10.
40 40 40

1 2 5 10 20 30 1 2 5 10 20 30 1 2 5 10 20 30

dsp024b 56.
80

Sound 60
28.

Level 40

1 2 5 10 20 30

Freq.
 Signal Encoding Algorithm:

Frequency Selectivity

ICMS pattern is based u5b 8

solely on frequency 80 6

Spikes
60
selectivity of neurons dB

40
4

recorded on an electrode 2

1 5 10 30 0
u32a
8
kHz
80
6
dB 60
4
Spikes

40

1 5 10 30 0

kHz
 Behavioral Performance

Ricms6

 Rat Behavioral Performance

RICMS 6

100

90

80

Percent Correct

70

60

50

40

30

20

10

00

00

00

00

00

00
6/

6/

6/

6/

6/

6/
/0

/1

/2

/0

/1

/2
09

09

09

10

10

10
Training day
Implanted
Cortical
Electrodes
 Expected Results to ICMS Stimuli

Begin ICMS

100

D % due

%
to ICMS

Trial #

Auditory trial =

ICMS Algorithm1 =

ICMS Algorithm2 =

 Behavioral Curve

RICMS 6 10/25 (Only Session)

100

80

Percentage

60
audPercent,
40
icmsPercent,

20

0 100 200

Trial
 Alternative Signal Encoding Algorithm:

Cortical Activation Pattern

For a given electrode, the unit firing pattern is used as a

template for ICMS delivery
Auditory
Sound on
Stimulus

Response
Raster

Matching ICMS
‘ pattern’

***Procedure is simultaneously duplicated on each active electrode

 Recap

� Focused & integrated effort

• Neural Engineering…Signal
Processing…Systems
Neurophysiology…Visualization BIO

� Why are we so excited?

INFO
• We have the very real
potential of developing new MICRO
classes of neuroprosthetic
systems to explore our ability
to interact directly with the
brain.
 BIO, INFO, and MICRO…

� Wide-open Challenges
• Appropriate mathematical constructs for describing neural
encoding and decoding.
• Advanced data visualization techniques for understanding this
new class of neural data.
• Understanding signal transformations as a function of the
spatial and temporal scale of the neural data.

� Collaboration Possibilities
• Exploring new signal encoding and decoding strategies for
particular neuroprosthetic applications.
• Sharing technologies, procedures, insights, etc…
• New emergent ideas…
 Topics

� Project overview
� Towards the Development of 3rd-Generation Neural
Implants (BIO, MICRO, and INFO)
� Bioactive Coatings to Control the Tissue Responses to
Implanted Microdevices (BIO, MICRO, and INFO)
� Modeling the Device-Tissue Interface (BIO, MICRO, and
INFO)
� Direct Cortical Control of a Motor Prosthesis (BIO, MICRO,
and INFO)
� Neural Control of Auditory Perception(BIO, MICRO, and
INFO)

� Wrap-up

 Project Challenges

� Scientific
• Overcoming engineering and scientific hurdles.
• Identifying and fostering strategic alliances with appropriate
external groups.
• Crossing disciplines

� Management
• Strategic planning
• Resource allocation
• Open and effective communication among the diverse project
team
• Team-building: Maintaining enthusiasm, energy, and focus
after the initial “honeymoon” period
 “Insanely Intense

Interdisciplinary” Research

“pieces of a puzzle” “easy synergism”

INFO
BIO
•Hard work
•Open minds
INFO •Honesty Breakthrough
MICRO •Top-notch research Science

MICRO BIO
 What Does the Future Hold?

“Perhaps within 25 years there will be some new ways to put

information directly into our brains. With the implant technology that
will be available by about 2025, doctors will be able to put something
like a chip in your brain to prevent a stroke, stop a blood clot, detect
an aneurysm, help your memory or treat a mental condition. You
may be able to stream (digital) information through your eyes to the
brain. New drugs may enhance your memory and fire up your
neurons.” -- Dr. Arthur Caplan,
Director of the Center of Bioethics,
University of Pennsylvania
Arizona Republic, Dec 27, 1998.

 Acknowledgments

� ASU Colleagues
• 13 co-PI’s, 5 research faculty, numerous graduate
and undergraduate students.
� Arizona State University administration
• Seed funding from Department, College, and
University
• Significant cost-share on this project
� DARPA Program Managers
• Eric Eisenstadt, Abe Lee, and Gary Strong