Deep-learning Cognitive Memory Architecture for Vision Networks

Aigerim Tankimanova (201103451), Irina Fedorova (201101178),Olga Krestinskaya (201101957)

Supervisor: Dr. Alex Pappachen James, Nazarbayev University, Astana, Kazakhstan

INTRODUCTION

PRELIMINARY RESULTS

DESIGN METHODOLOGY

PROBLEM STATEMENT:
Limitations in scalability of CMOS devices combined with the veracity, volume and
velocity of real-time imaging data demands development of newer memory
architectures to improve efficiency and processing speed.

The proposed design consists of a imaging sensor, HTM for feature extraction, and a memory model
for feature selection, when combined to be useful to create object templates useful for
neuromorphic storage and recognition application. Various tools used in each of the stages are
listed in Table 1.
Table 1. Experiments relevant to the design and required tools

OBJECTIVE:

Design experiments

Design a new CMOS-memristor hybrid neural systems inspired memory architecture

for real-time image processing and recognition application.

Tools

Modified HTM and feature selection algorithm

Design of memristor-based pixel sensor

Matlab, Verilog A
LTSpice, VLSI

Hardware implementation of HTM

The proposed memory array architecture includes deep-learning principles,

hierarchical temporal memory (HTM) component and a feature ranking algorithm
that is based on the principles of attention, repetition and required number of
details. While the sensory pixel detection and processing part is based on a
memristive pixel sensor integrated with on-pixel hybrid CMOS/memristive analog-todigital converter. The major activities of the project (Fig 1) and responsibilities are
distributed as follows:
1. Memory array architecture - Olga Krestinskaya
2. HTM hardware implementation Irina Fedorova
3. Memristor based pixel sensor design Aigerim Tankimanova

MEMORY ARCHITECTURE:
The main components of self-learning cognitive memory array are pixel sensor, HTM with feature
processing function and memory space (Fig.2). The steps involved in pixel processing through the
proposed architecture are the following:
1) The input images are received by memristive pixel sensor which transmits obtained pixels to
HTM.
2) HTM is a part where the data is processed and transferred to the suitable form for feature
selection.
3) Feature selection and ranking is a part is inspired by human memory processing where main
criteria are feature intensity, repetition factor, number of details and attention factor.
4) Unnecessary pixels are expelled, whereas the most important data pixels are saved.
5) Irrelevant data is removed for the memory after certain period of time considering the
aforementioned criteria.
Real-time input

Memristive pixel sensor

HTM

Remembering process

Attention
Details
Repetition

Memory
Forget unimportant
features

Forget after specified time period

Fig. 2. Deep-learning cognitive memory array architecture

Fig. 1. Gantt chart for the project

HTM hardware implementation

The major principles involved in the development of proposed system draw

inspirations from the ideas of human memory model, hierarchical temporal memory,
content addressable memory architectures and neuron like memristors.

Conventional CAM. CAM is a type of computer memory used for high-speed

application [2]. CAM consists of cells where bits of data are stored. The main
functions of cell are to store and compare bits. The main disadvantages of CAM
based on CMOS transistors are large power consumption, limited capacities and the
occurrence of data loss and lack of scalability [3].
Hierarchical temporal memory (HTM). HTM is one of the algorithms that model
behavior of cortical regions of human brain [4]. HTM constitutes spatial and
temporal pooler components. The making of predictions and learning of the input
sequences is established in the temporal pooler. HTM has a variety of applications
from image processing to audio and behavior recognition [5].
Memristor. The fourth fundamental circuit element which has two states: on and off
[6]. The ability of the memristor to update its state, considering its current state and
a new input, is proved to be essential for the imitation of biological synapses
components. The distinguishing feature of the memristor is a compact on-chip area,
elimination of problems associated with leakage current which leads to the
possibility of applying memristor-based circuits for memory applications [7].

Photodiode

Fig.6. a) Memristor-based pixel; b) not amplified output.

(a)

(b)

Fig.7. (a) Circuit for threshold and averaging operation of HTM; (b)Simulation of the
threshold/averaging circuit.

HTM and feature selection

Fig. 8. shows the features selected from the
HTM feature maps. Here, image pixels are
processed through HTM and then feature
selection algorithm is applied. It can be
observed that after the selection process only
most important features from several images
are saved.

Feature
selection

Standard
deviation
filter

Fig. 8. Matlab simulation of feature

selection process.

Cost and efficiency

The use of memristors would allow reduction of on-chip area in comparison to
the CMOS only counterparts. This in turn would result in decreased amount of silicon
area required for manufacturing and, as a consequence, the costs for bringing the
product to the market would be reduced. In addition, the propriety of having no reverse
leakage current alleviates the problem connected with the power losses and improve
the overall efficiency.

BACKGROUND

Memory Model. Human memory is understood to be comprising of sensory register,

short term and long-term memory. The memorized information is influenced by
number of repetition, personal experience, strength of stimulus and attention [1].
These basic principles of human memory model is assumed to be the basis for the
proposed memory architecture.

(b)

(a)

Fig.7 (a) presents the circuit for threshold and averaging operation for one image block
that was illustrated in Fig. 4. The results of simulation, assuming only 4 inputs, could be
seen on Fig.7 (b).

DISCUSSION

PRACTICAL SOLUTION:
Memristor-based pixel sensor

Fig.6 (a) shows the

proposed circuit design for
memristor-based
pixel.
Fig.6 (b) shows the
preliminary pixel output
voltage.

HTM implementation

Tanner EDA, LTSpice, Python

Feature selection and

ranking

Memristor-based pixel circuit

Fig. 4. Hardware implementation of HTM

Fig. 3. Block diagram of the proposed pixel design

Fig 3. shows the overall structure of the

CMOS-memristor pixel sensor.
Pixel read block: The advantages of memristorbased pixel sensor design is the reduced chip
area and elimination of leakage power loss.
Amplification: The amplification of the level of
the signal will help to perform better digitizing
of input .
Threshold logic based ADC: The pixel sensor
should distinguish different levels of intensities
of the input signal. Therefore, a threshold logic
based A/D conversion will be applied to the
input analog signal.

Fig. 4 shows the block diagram for the

proposed design of HTM on hardware, where
is set of binary inputs; N= number of image
blocks; X=0,1,2,3 M; M=binary pixel values
of block.
Algorithm for HTM implementation:
1) Find the threshold for each block by
averaging of particular block
2) Use the threshold to compare it with
3) Compute the average value for each block
4) Compare the average values of different
blocks and select the winner
5) Update the threshold of the winning block

Feature selection
Eq. (1) is the proposed model for feature selection taking into account design principles of human
memory. The model defines the threshold to which the decision to accept or reject particular
pixels is made. 1

= [w 0 0 + w k ( 1
2

0

, 1 , 2 , 3

+ 0.01 2 + 3 )]

(1)

Real time application.

The developed architecture is expected to be useful for real-time application, especially
in the cases connected with biometric cyber-security and investigation, where the
recognition of the image patterns and overall object as well as
remembering of certain features is of great importance.

CONCLUSION
In this project a novel memory architecture is proposed for image processing and
recognition application. The advantages of the design are associated with the use of
neuromorphic memristors that lead to the improved memory speed, small on-chip area
and reduced power consumption. The preliminary results were promising; however for
the future work adverse effects, such as noise, parasitic capacitance and inductance
should be further investigated. This should be followed by comprehensive analysis of the
aforementioned issues and their elimination.

REFERENCES
[1] K. A. Ericsson and W. Kintsch, Long-Term Working Memory, Psychological Review, 1995.
[2] L.M. Lonescu, A.G. Mazare, G. Serban, V. Barbu, and A. Constantin, FPGA implementation of an associative content addressable memory, Applied
Electronics (AE), 2011 International Conference, 2011.
[3] Kumar, S.D and Sk, N.M, A novel binary content addressable memory design using reversible logic, Computer and Communications Technologies
(ICCCT), 2014 International Conference.
[4] HIERARCHICAL TEMPORAL MEMORY including HTM Cortical Learning Algorithms, Numenta, 2011
[5] X. Chen, W. Wang and W. Li, An Overview of Hierarchical Temporal Memory: A New Neocortex Algorithm, Modelling, Identification & Control (ICMIC),
Proceedings of International Conference, 2012.
[6] M. Das, S. Singhal, Memristor-Capacitor Based Startup Circuit for Voltage Reference Generators, International Conference on Advances in Computing,
Communications and Informatics. IEEE, 2014.
[7] Kozma, Robert, Robinson E Pino, and Giovanni E Pazienza. Advances In Neuromorphic Memristor Science And Applications. Dordrecht: Springer, 2012.
Print.