You are on page 1of 29

James Mnatzaganian & Ross Reinhardt

Neuromorphic Computing Overview Theory of Neuromorphic Architectures Advantages and Challenges of Neuromorphic Architectures Neuromorphic Architectural Attempts Memristor Overview Memristor Functionality Memristive-based Neuromorphic Architectures


Neuromorphic Architectures


Modeled from neural circuitry

Artificial neural networks

Abandons Von Neumann architecture

Integrates processor with memory

High degree of parallel processing High efficiency


Biological Model
Neuron: Processes received information Axon: Sends outgoing signal to other neurons Synapse: Connection between neurons Remembers previous state and weight of path Dendrites: Receives incoming signals from other neurons

Information Flow

Neurons triggered at a voltage threshold Operate on voltage spikes rather than constant V Input through dendrites, output through axon Learning achieved by synapse memory Synapses adjust weight based on previous states



Hardware Implementation

o Goal: Emulate biological neural networks



High Speed Parallel Processing Low Power Adaptive Architecture

Recognition of patterns & matching tasks Integrates processor and memory

o Eliminates Von Neumann bottleneck o Difficult for Von Neumann architectures

o Can learn and operate on incomplete data

High densities achievable using memristorbased technologies

11/13/2012 7

Interconnection with existing architectures Complex design and management Thermal management Each neuron is independent Distributed processing and storage Simulated Neuromorphic Architectures
o Topology o Error checking

o Not true neural networks o High simulation overhead on current technology = low performance o Cannot be run in real-time on Von Neumann architectures o Can only simulate small portions of a neural network o Must be run on supercomputers = high power consumption



Systems of Neuromorphic Adaptive Plastic Scalable Electronics

o IBM attempt to build scalable architecture similar to the mammalian brain o DARPA funded 2009 o Failed to meet goals

[3] 11/13/2012 9


o One 2x3mm core o 1024 Axons o 256 Neurons

o 262,144 programmable or 65,356 learning synapses

o CMOS Technology

45 nm technology node ~500 Transistors / synapse High transistor count & layout area Not Scalable to biological levels

[4] 11/13/2012 10

Von Neumann bottleneck

Separated memory from CPU Single order execution Minimal parallelizability

Dawn of a new era The memristor

Low power High parallelizability Biologically-based






Theorized by Leon Chua in 1971 Demonstrated by HP Labs in 2008 Missing element Nonlinear twoterminal device Resistor with a memory element





Resistance is a function of applied voltage & time Resistance is based off oxygen vacancies




Nonlinearity is a result of M and i dependency on q Memristors state is dependent upon past and present events

[14] 11/13/2012 15

Synapses are used by neurons to pass signals to cells STDP Spike Timing-Dependent Plasticity Synaptic weight Strength of neuron connection Excitatory synapses increase membrane voltage Inhibitory synapses decrease membrane voltage Changing of weights allows biological systems to learn
Spiking from pre- to post-synaptic neuron Determines synaptic weight

[15] 11/13/2012 16

Memristance Synaptic weight Crossbar array

Memristors connected at each crosspoint connecting CMOS pre- and post-neurons Up to 1010 synapses / cm2 (100 nm pitch)




Memristors + Neuromorphic Computing



Memristor-Based Synapse STDP-Based Neuromorphic Architecture Memristors + Spin Devices mrFPGA



Biological model approach Memristor Training Circuit

Multi-synapse training scheme Self-training mode

Dynamic write time

Self-adaptive to environment Real-time synaptic weight adjustment

[12] 11/13/2012 20

STDP Spike Timing Dependent Plasticity

Biological process that adjusts the strength of connections between neurons

Change in weights is a function of excitatory and inhibitory synapses Analog architecture utilizing the crossbar structure

[13] 11/13/2012 21

Theoretical Design
o o o o o


Lateral spin valves & Memristors Proposed by Intel, 2012 May theoretically decrease power consumption by 15-300x compared to CMOS Terminal voltages in mV = lower power Still much less efficient than real neurons

o Magnetic Tunnel Junction o Two magnetic poles separated by thin insulator o Poles shift due to external magnetic fields
Spin = neuronal spiking Low power consumption Nanomagnets




Memristor-based FPGA Proposed 2011 Replaces SRAM interconnects

Help close the performance gap between FPGA and ASIC Greater performance with lower development costs

o From 6 transistors to 1 memristor o Current interconnects are 90% of area, 80% delay, 85% of power consumption


[16] 11/13/2012 23

CMOS scaling will eventually halt

Memristive-based neuromorphic architectures could be the key Active research by numerous companies and universities Its only a matter of time... A.I. ???

Hardware-based improvements needed



Memristors act like synapses A biological model consists of neurons connected via synapses Neuromorphic architectures are designed to mimic the brain Memristor-based neuromorphic architectures prove to be the most realizable option



[1] Kandel E.R., Schwartz, J.H., Jessell, T.M. 2000. Principles of Neural Science, 4th ed., McGraw-Hill, New York. [2] [3] Mosher, Dave. New Chip Borrows Brains Computing Tricks. <>. 2011. [4] John V. Arthur, Paul A. Merolla, Filipp Akopyan, Rodrigo Alvarez-Icaza, Andrew Cassidy, Shyamal Chandra, Steven K. Esser, Nabil Imam, William Risk, Daniel Rubin, Rajit Manohar, and Dharmendra S. Modha, Building Block of a Programmable Neuromorphic Substrate: A Digital Neurosynaptic Core, International Joint Conference on Neural Networks, 2012. Available: [5] M. Sharad, C. Augustine, G. Panagopolous and K. Roy, " Proposal for Neuromorphic Hardware using Spin Devices", arXiv:1206.3227v4 [6] (Grollier, 2012) [7] Rinky B P. Innovative Blood. Memristor - The Missing Circuit Element . <>. 2012. [8] Williams, Stanley. How We Found the Missing Memristor. IEEE Spectrum. 2008. [9] Kvatinsky, Shahar. Logic Design With Memristors. Technion Israel Institute of Technology. 2012. [10] Borhetii, Julien, et. al. Memeristive switches enable stateful logic operations via material implication. nature vol 464. doi:10.1038/nature08940. 2010. <>. [11] Hyun Jo, Sung, et. al. Nanoscale Memristor Device as Synapse in Neuromorphic Systems. Nano letters doi: 10.1021/nl904092h. 2010. <>. [12] Hui Wang; Hai Li; Pino, R.E.; , "Memristor-based synapse design and training scheme for neuromorphic computing architecture," Neural Networks (IJCNN), The 2012 International Joint Conference on , vol., no., pp.1-5, 10-15 June 2012. doi: 10.1109/IJCNN.2012.6252577. URL: [13] Ebong, I.; Deshpande, D.; Yilmaz, Y.; Mazumder, P.; , "Multi-purpose neuro-architecture with memristors," Nanotechnology (IEEE-NANO), 2011 11th IEEE Conference on , vol., no., pp.431-435, 15-18 Aug. 2011. doi: 10.1109/NANO.2011.6144522. URL: [14] Massimiliano Verscae. (2011, August 9). Fuzzy logic and memristive hardware, neurdo. Available: [15] Robert Stufflebeam. "Neurons, Synapses, Action Potentials, and Neurotranmission." Consoritum on Congnitive Science Instruction. <>. 2008. [16] Cong, J.; Xiao, B.;, "mrFPGA: A Novel FPGA Architecture with Memristor-Based Reconfiguration," Nanotechnology (IEEE-NANO), 2011 IEEE International

Symposium on Nanoscale Architectures






Synaptic Plasticity

o Excitatory & Inhibitory synapses

o Weight of synapse changes in response to inputs forming new pathways o Responsible for network reorganization and learning

o Action potential (spike) at an excitatory synapse strengthens a connection, while spikes at inhibitory synapses lower the probability that a post-synaptic cell will spike




Hybrid Memristors + transistors Computational only Fast for Boolean logic CMOS compatibility

IMPLY pIMPq == (NOTp)ORq MAGIC Memristor Aided LoGIC Combinatorial logic Logic inside memory Performs & stores logic Separate input and output Reduced area memristors

Many types of memristor-based logic Fast computational memory Logic + memory Linear / nonlinear / somewhere in-between
11/13/2012 29