See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/323193463

Comparative analysis of image processing algorithms for visual prosthesis

Conference Paper · April 2017

DOI: 10.1109/ICCSP.2017.8286551

CITATIONS READS
0 289

3 authors, including:

Sharmilli Nukapeyi Ramakrishna Gandi

Gayatri Vidya Parishad College of Engineering Annamacharya Institute of Technology & Sciences
6 PUBLICATIONS   11 CITATIONS    13 PUBLICATIONS   33 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Smart Object Picker View project

Professor Appraisal Data mining View project

All content following this page was uploaded by Ramakrishna Gandi on 29 September 2018.

The user has requested enhancement of the downloaded file.

 International Conference on Communication and Signal Processing, April 6-8, 2017, India
Comparative Analysis of Image Processing
Algorithms for Visual Prosthesis
N.Sharmili, N.Swapna and G.Ramakrishna
 Abstract—Visual prosthesis is one of the emerging technology
in biomedical implants which helps to restore useful vision for
visually impaired people suffering from retinal degenerative
diseases. The visual prosthesis consists of two parts. One is
external and the other is internal (surgically implanted). The
external part includes a camera, image processor and a
transmitter. The image processor captures the video using a
camera, encodes the captured information to data forms that can
be recognized by implanted stimulator and transmits the coded
data through transmitter. The internal implanted part includes a
receiver and electrode array. The receiver converts the received
bit stream into electrical signals and stimulates the survival parts
of the retina using electrical signals applied through the electrode
array. The survival parts then sends a signal to the brain where it
is interpreted as image/video. The researchers suggested that an
array of 600-1000 electrodes are required to do some basic
forms of vision such as reading large letters, recognizing
faces..etc. The image processor is the crucial part of visual
prosthesis where it captures high resolution image data from
camera, and resizes it into 1024 resolution image. To extract 1024
pixel vital information, the related image processing algorithms
include RGB to Gray conversion, Gamma encoding, Edge
Detection and down sampling are reported in this paper. The
comparative analysis of image processing algorithms provided
here will drive an image processing strategy with less
computational complexity of achieving better performance that is
suitable to implement on MIPS based microcontroller.
Index Terms—BWIP, Image Processing, Edge Detection,
Epiretinal prosthesis, RIRS, Vision prosthesis.
I. INTRODUCTION
V ISUAL prosthesis is currently being developed world
wide ameliorate more than 40 million blind individuals in
the world whose plight would be due to incurable retinal
degenerative diseases. The retinal neuro degenerative diseases
such as Age-related Macular Degeneration (AMD) and
Retinitis Pigmentosa (RP) are the predominant causes of
human blindness due to progressive degeneration of
photoreceptors in the retina but internal layers of the eye are
still healthy. Visual prosthesis is an electronic device designed
to bypass the damaged parts of the retina by electrically
stimulating the surviving healthy parts of the retina i.e.
N.Sharmili is with Dept. of Computer Science and Engg., GVPCEW,
Visakhapatnam, India (e-mail: logintosharmi@gmail.com).
N.Swapna is with Dept. of Computer Science and Engg., KIET, Korangi,
India (e-mail: swapna_b7@yahoo.com).
G.Ramakrishna is with Dept. of Computer Science and Engg., VIIT,
Visakhapatnam, India (e-mail: gandiramakrishna@gmail.com).
978-1-5090-3799-5/17/$31.00 ©2017 IEEE
ganglion cells using electrical pulses applied through an
electrode array. Ganglion cells generate a nerve impulse and
transmits to the optic nerve, which carries it to the brain where
the electrical signals are interpreted as a visual image [1-3].
Visual prosthesis consists of an external part and an internal
implant, which are connected by a transcutaneous inductive
link. The external image processing unit processes the camera-
captured image and transmits the processed information
serially to internal implant through wireless link. The internal
implant takes the serial image information, transforms it into
electrical pulses and delivers them to the electrode array. The
electrodes stimulate the healthy neurons in eye, and visual
sensations are perceived.
Epiretinal Prosthesis (EP) mentioned in this paper is one of
the dominant approach in visual prosthesis where majority of
researchers or working. The commercially available Epi-
Retinal Implant (ARGUSII at $115,000) was unable to restore
useful vision to patients due to its 60 electrodes resolution.
Present day retinal prosthesis projects are mainly aimed to
develop an implant with several hundreds of electrodes (even
up to 1024 due to limited space to implant on the surface of
retina), so that the blind can read letters, navigate a room and
recognize faces [4][5][7]. Research in this paper is focused on
image-processing in external part in addition to implant design
for better visual perception [3]-[11].
The captured image from camera is having large number of
pixels (640*480 image having 307200 number of pixels).
These pixels have to be reduced to 1024 number of pixels i.e.
the number of electrodes implanted in internal part of EP. Best
image processing strategy is needed to extract useful
information from 32x32 resolution images for improving
visual perception and ability to conduct visual tasks daily. This
paper presents a best image processing strategy by analyzing
and implementing different image processing algorithms such
as gray conversion, smoothing and edge detection and down
sampling are studied and implemented for object recognition
and feature extraction... etc. However the internal implant
design with real electrodes for electrical stimulation is beyond
the scope of this paper and it can be developed by standard
approach as described in research articles[7].
The paper is organized as follows. Section II summarizes
the system architecture of the EP. Section III describes the
Image processing strategy for EP using different image
processing techniques. Section IV represents the comparative
analysis and results of EP. Finally, Section V concludes the
paper.
 EXTERNAL BODY WORN INTERNAL RECEIVER STIMULATOR
IMAGE PROCESSOR
Image /
video RF Stimuli
Signal Camera Signal
Image Processor ASK demodulator
Simulated LED
Data Encoder Decoder Electrode Array
RF Transmitter Electrode
Stimulator

Fig. 1. Block diagram of Epiretinal prosthesis system

II. EPIRETINAL PROSTHESIS SYSTEM ARCHITECTURE
Epiretinal Prosthesis System shown in Fig. 1. consists of
Body Worn Image Processor (BWIP) as external part that is
body worn, Internal Receiver Stimulator (IRS) as internal that
is surgically implanted under the retina and a wireless
telemetry which links external and internal parts. External part
comprises a video camera, Image processor and a RF
power/data transmitter. External part captures the image/video
using the camera, process them using image processor to
data required to display one full frame and support the update
rate 20 - 30 fps for viewing a good motion picture and refresh
rate 50 - 60 fps for flicker-free vision. To meet these real
time requirements of vision processing for natural vision, the
image processing algorithms with less computational time are
required. Figure 2 shows image processing strategy for better
visual perception.
Smoothing
RGB to Gray Edge Detection
Conversion

match the channel number and color depth of the stimulation
electrode array, then encodes into data frame format and
transmits the processed data frames serially to ASK modulated
RF transmitter for wireless transmission. The internal portion
consists of a RF power/data receiver circuit which receives
power/ data and transmits the received data to Stimulation
controller. Stimulation controller delivers electric pulses
corresponding to received data and route those pulses to
electrode array which stimulates the targeted tissue cells.
This paper represents a newer image processing strategy for
better image prediction in developing higher density image
processor for next generation retinal prosthesis. The camera
and image processor modules can be integrated and
implemented as standalone app in a manner similar to mobile
based Android App.
III. IMAGE PROCESSING STRATEGY FOR EPIRETINAL
PROSTHESIS
Majority of researchers are working on 1024 electrode
epiretinal prosthesis [6][13-16] to do basic vision tasks.
Image processor is an essential part of visual prosthesis which
captures image using camera and implements a specific image
processing strategy to transform image to data forms of size
(i.e. 1024 electrodes implanted inside the human eye) that can
be accepted by implanted stimulator. With this restriction,
the image processor has to provide satisfying image
processing ability and also has to extract useful information
from the captured image that provides an even better view of
the world. Various image processing algorithms like RGB to
Gray conversion, Smoothening, Edge detection and image
compression are analyzed and implemented to help blind for
better visual perception. [12- 14]. The real time requirements
of natural vision processing in BWIP is that it should hold the
Captured RGB
image
Output data to Image Image
internal implant Encoding Compression
010111000…
Fig. 2. Block diagram for Image processing in External Image processor
unit
In the external part, the web camera captures the 640x480
image in JPEG format and transmits it to the image processor.
After studying various image processing algorithms in
MatLab, the image processing strategy i.e. RGB to Gray
conversion, Smoothing, Edge Detection, Down sampling is
preferred [6][13]. To implement these image processing
algorithms in hardware, it would be first implemented in Java
using built in functions which can be converted to machine
code. The processed image is transformed into encoded data
for transferring to the internal implant part.
A. RGB to GRAY conversion
In external part, the captured RGB image using a camera is
given to image processor which converts it into gray format to
reduce computational complexity in processing high volume of
data (volume of RGB image is 3 times larger than gray image).
For RGB to Gray image conversion the three algorithms
Lightness method, Average method and Luminosity method
are implemented.
Lightness Method
The lightness method takes average to the most prominent
and least prominent colors for RGB to Gray image conversion.
Grey color pixel value = (max(R, G, B) + min(R, G, B)) / 2
Average Method
The average method averages the pixel color values for
RGB to Gray image conversion.
 Grey color pixel value = (R + G + B) / 3
Luminosity Method
The luminosity method or weighted method is an advanced
version of the average method which also averages the color
values, but it results a weighted average to account for human
perception. Human eye is more sensitive to green color than
other colors, so green is heavily weighted. In this method, the
formula 0.21 R + 0.72 G + 0.07 B is used for RGB to Gray
image conversion.
(a) (b) (c) (d)
Fig. 3. (a) RGB Image, (b) Lightness Image, (c) Average Image, (d)
Luminosity Image
Among the three algorithms, the luminosity method works best
and computationally less expensive for RGB to grayscale
image conversion.
B. Image enhancement
The converted Gray Image is given as input to the Image
Enhancement Method. Image Enhancement technique
improves the quality of the image for a specific application.
The three types of functions used frequently for image
enhancement are Negative Transformation, Logarithmic
Functions and Gamma Correction or Power-law Functions.
Negative transformation:
Negative transformation is a linear transformation in which
output intensities are identical to input intensities. The
expression for Negative Transformation is s = 256 – r . The
Negative transformation results inverting the input pixel values
(i.e. gray level intensities of the image) there by producing a
negative like image.
Log Transformation:-
Log transformation compresses the dynamic range of
images with large variations in pixel values and maps a narrow
range of low gray-level values in the input image into a wider
range of the output levels. The general form of Log
transformation:
s =c log (1+r) Where c is a constant, r is the gray pixel value and r ≥
0 Because of Sobel operator’s smoothing effect (Gaussian
This transformation is used to expand the values of dark
pixels in an image while compressing the higher level values.
Gamma Correction:
Gamma correction is a form of nonlinear operation that
encodes and decodes luminance values of video or still image
systems. Gamma correction is defined by the following power-
law expression: Gamma= A* Grayγ or Vout = A Vinγ Where A
is a constant, the input and output values are non-negative real
values. For A = 1, input and output values are in the range 0–
1. For gamma value γ < 1 is called as encoding gamma and the
process of encoding with this compressive power-law is called
gamma compression. A gamma value γ > 1 is called as
decoding gamma and the process of the expansive power-law
is called gamma expansion. By observing the results of above
three techniques of image enhancement, Figure 4(d) is best
compared to others visually
(a) (b) (c) (d)
Fig. 4. (a) Input Gray Image, (b) Negative Image, (c) Logarithmic function
Image, (d) Gamma corrected Image
C. Edge detection
The converted enhanced image is given as input to the
image edge detection method. Edges are sharp local changes
of intensity in an image and occur on the boundary between
two different regions in an image. The process of finding
abrupt changes in pixel intensity which characterize
boundaries of objects in a scene or image is called edge
detection. The main goal of edge detection is to extract the
important features from the edges of an image (e.g., corners,
lines, curves) for basic computer vision tasks such as
recognition and tracking. The common and efficient edge
detection algorithms are Sobel Edge Detection and Canny Edge
Detection.
Sobel Edge Detection:
The Sobel edge detection is a discrete differential edge
detection which uses two 3x3 kernels: one calculates the
gradient in the x-direction (i.e. Gx), while the other one
calculates the gradient in the y-direction (i.e. Gy) are shown in
Fig. 5. The image is convolved with both kernels to
approximate the derivatives in horizontal and vertical change.
At each given point, magnitude of the gradient can be
calculated with G = .To compute the gradient
magnitude faster with G = |Gx| + |Gy|
Gx Gy
1 2 1 -1 0 1
0 0 0 -2 0 2
-1 -2 -1 -1 0 1
(a) Horizontal Mask (b) Vertical Mask
Fig. 5. Masks used by Sobel Operator
smoothing), the detected image is less sensitive to noise. Sobel
method does not generate high accurate image for edge
detection due to its smoothing effect, but its quality is adequate
enough to be used in numerous applications. The steps we are
following in Sobel edge detection algorithm are
Step 1: Smoothing – Blurring the image to reduce the noise.
Step 2: Finding gradients – mark the edges where the
gradients of the image have large magnitude.
Canny Edge Detection:
Canny edge detection algorithm is a sophisticated
algorithm to detect a wide range of edges in a gray image. The
canny edge detection is implemented in two steps as follows:
Step 1: Smoothing the image to remove the noise. The
smoothing filter used here is Gaussian filter.
Step 2: Finding gradients mark the edges where the gradients
of the image have large magnitude. The formula for gradients
of the image in X and Y directions where the kernel size is 2×2
array matrix i.e. Pixel [i][j].
[i][j] = pixel[i+1][j-1] + 2×pixel[i+1][j]+pixel[i+1][j+1] - pixel[i-1][j-1] –
2 × pixel[i-1][j] - pixel[i-1][j+1]
[i][j] = pixel[i-1][j+1] + 2×pixel[i][j+1] +pixel[i+1][j+1] - pixel[i-1][j-1]
– 2 × pixel[i][j-1] - pixel[i+1][j-1]
Step 3: Non maximum suppression removes pixels not related
to an edge and local maxima should be marked as edges.
Step 4: Double thresholding which takes only potential edges.
Step 5: Edge tracking by hysteresis in which final edges are
determined by suppressing all edges that are not strong edges.
Hysteresis uses upper and lower thresholds. If a pixel gradient
is above the upper threshold, then the pixel is marked as an
edge. If a pixel gradient is below the lower threshold, then that
pixel is discarded. Finally, if the pixel gradient is between the
two thresholds, then that pixel is connected above the upper
threshold is marked as an edge.
Although canny edge detection works for better detection of
edges especially in noisy state, but it consumes more time due
to its complex computation. The results of Edge detection is
shown in figure 6. To achieve real time requirements of natural
vision the simplest sobel edge detection is preferred.
(a) (b) (c) (d)
Fig. 6. (a) Input Enhanced Image, (b) Gaussion Image, (c) Sobel Edge
Detection Image, (d) Canny Edge Detection Image
D. Image compression
The converted edge detection image is given as input to the
image compression method. For the process of image
compression, two types of image compression algorithms haar
and Daubeshies wavelets are analysed.
TABLE I
COMPARATIVE ANALYSIS OF IMAGE PROCESSING ALGORITHMS
Captured RGB image(640×480)
Haar Wavelets:
The Haar wavelet transform is simple, memory efficient,
exactly reversible without the edge effects characteristic of
other wavelets and computationally cheap. Haar wavelet
reflects only the changes between adjacent pixel pairs and
doesn’t having overlapping windows. The transform estimates
two coefficients namely, scaling and wavelet function
coefficients which are mainly calculated from pair wise
averages and differences. For an input of 2n numbers, the Haar
wavelet transform calculating and storing the difference and
passing the sum from pair-wise input values. This process is
repeated recursively by pairing up the sums to generate the
next scale: finally resulting in 2n-1 differences and one final
sum. The pair-wise average of numbers is taken in this down
sampling for EPS and this process is repeated until the down
sampled image size is 32x32 which is shown in figure 7.
Daubeshis Wavelet:
The most dominant discrete wavelet transforms was
Daubechies wavelet transform which is used for texture feature
analysis due to its orthogonal and compact support abilities.
The Daubechies wavelet transforms are smoother than Haar
wavelet, but they are differing in computing the scaling signals
and wavelets.Daubechies wavelet transform calculating
running averages and differences (via scalar products with
scaling signals and wavelets) using few more values of the
signal to produce averages and differences.
This difference will make Daubechies more efficient for
performing the signal processing tasks such as compression
and noise removal for image enhancement and signal
recognition.

Input for the

image processor

Algorithm - 1 Algorithm - 2 Algorithm - 3 Proposed Algorithm

Luminosity Method
RGB to Gray Of time complexity 65
conversion
Lightness Method Average Method(70) Luminosity Method (65)
(80)(640×480) (640×480) (640×480)
Gamma correction
Image algorithm of time
Enhancement complexity 230
Gamma correction (230) Negative Transform (75) (640×480) Log Transform (90)
(640×480) (640×480)
Sobel edge detection of
Image Edge time complexity 160
detection
Sobel edge detection Canny edge detection (180)
(160)(640×480) (640×480)
Haar wavelets transform
Image Down of time complexity 70
sampling
Haar wavelets(80) (32×32) Daubeshis wavelets(70)(32×32)
 There are many types in Daubechies transforms. In this
section the simplest Daubechies 4 wavelet transform is
discussed which uses overlapping windows to reflect changes
between all pixel intensities.
The Daubechies D4 transform consists of four wavelet and
scaling coefficients. The scaling coefficients sum is also one,
thus the calculation is averaging over four adjacent pixels. In
Daubechies D4 wavelet algorithm, the coefficients for the
scaling function are represented as hi (high) and the wavelet
coefficients are represented as gi (low). One step of the
forward transform can be represented as infinite matrix of
wavelet coefficients shown below, multiplied by the infinite
signal vector (s).
ai =…h0, h1, h2, h3,0,0,0,0,0,0,0,0… si
ci =… l0, l1, l2, l3, 0,0,0,0,0,0,0,0… si+1
ai+1 =…0,0,h0, h1, h2, h3,0,0,0,0,0,0… si+2
ci+1 =…0,0,l0, l1, l2, l3,0,0,0,0,0,0,… si+3
ai+2 =…0,0,0,0,h0, h1, h2, h3,0,0,0,0,… si+4
ci+2 =…0,0,0,0,l0, l1, l2, l3,0,0,0,0,0,… si+5
ai+3 =…0,0,0,0,0,0,h0, h1, h2, h3,0,0,… si+6
ci+3 =…0,0,0,0,0,0,l0, l1, l2, l3,0,0,… si+7
The dot product (inner product) of the infinite vector and a
row of the matrix produces either a smoother version of the
signal (ai) or a wavelet coefficient (ci). In an ordered wavelet
transform, the smoothed (ai) are stored in the first half of an n
element array region. The wavelet coefficients (ci) are stored
in the second half the n element region. The algorithm is
recursive. The smoothed values become the input to the next
step. The wavelet coefficient values are moved along the
signal vector and a four element dot product is calculated.
Expressed in terms of arrays, for the forward transform this
would be
ai = s[i] _ h0 + s[i + 1] _ h1 + s[i + 2] _ h2 + s[i + 3] _ h3
ci = s[i] _ g0 + s[i + 1] _ g1 + s[i + 2] _ g2 + s[i + 3] _ g3
The resultant output of Daubeshis is shown in figure 7.
(a) (c)
(b)
Fig. 7. (a) Edge Detection Image (b) Haar Compression Image (c)
Daubeshis Compression Image
IV. COMPARATIVE ANALYSIS AND RESULTS
By applying different combinations of the above image
processing strategies, the best image processing methodology is
evaluated for better visual perception. The outputs of process flow of
image processing methodology using various combinations of image
processing algorithms is shown in table I. The presented results are
satisfactory for visual perception to do daily tasks.
V. CONCLUSION
After analysis of the all the above image processing algorithms
we conclude that some are the best algorithms to get the better
Visual Perception. To achieve the real time requirements of
vision processing in smart eye for better vision, the fast and
efficient Image processing system which includes normalized
View publication stats
luminosity, Gamma correction, Sobel edge detection and Haar
wavelets algorithms is used. The processed image is
transmitted to the internal part of the visual prosthesis system
or smart eye. The basic vision tasks such as reading large text,
walking without assistance and recognition of an object are
achieved by using the above algorithms in the image processor
in the visual prosthesis system. The presented results would
encourage researchers to use this image processing approach
of visual prosthesis for their future work in portable image
processor.
REFERENCES
[1] World Health Organization, ―Magnitude and causes of visual
impairment,‖ Fact Sheet no. 282, November 2004, http://www.who.int/
mediacentre/factsheets/fs282/en/.
[2] R. W. Massof and D. L. Rickman, ― Obstacles encountered in the
development of the low vision enhancement system,‖ Optometry and Vision
Science, vol. 69, no. 1, pp. 32–41, 1992.
[3] E. Peli, L. E. Arend, and G. T. Timberlake, ―Computerized image
enhancement for visually impaired people: new tech- nology, new
possibilities,‖ Journal of Visual Impairment & Blindness, vol. 80, no. 7,
pp. 849–854, 1986.
[4] E. Peli, R. B. Goldstein, G. M. Young, C. L. Trempe, and S. M. Buzney,
―Image enhancement for the visually impaired: simulations and
experimental results,‖ Investigative Ophthalmology & Visual Science, vol.
32, no. 8, pp.2337 - 2350, 1991.
[6] N.Sharmili, P.S. Ramaiah, and G.Swamynadhan, ―Image
Compression and Resizing for Retinal Implant in Bionic Eye‖,
International Journal of Computer Science &Engineering Survey, vol.2,
no.2, pp. 30-37, 2011.
[7] Banarji, A., V. S. Gurunadh, S. Patyal, T. S. Ahluwalia, D. P. Vats, and
M. Bhadauria, "Visual prosthesis: Artificial vision," Medical Journal
Armed Forces India 65, no. 4, pp. 348-352, 2009.
[8] G.Wang, P.Wang, Y. Tang, and W. Liu, ―Analysis of dual band power
and data telemetry for biomedical implants,‖ IEEE Trans. Biomed.
Circuits Syst., vol. 6, no. 3, pp. 208–215, Jun. 2012..
[9] Matteucci, B. Paul , Philip Byrnes-Preston, Spencer C. Chen, H. Lovell,
and Gregg J. Suaning. "ARM-based visual processing system for
prosthetic vision." In Engineering in Medicine and Biology Society,
EMBC, 2011 Annual International Conference of the IEEE, pp. 3921-
3924. IEEE, 2011.
[10] Fink, Wolfgang, Cindy X. You, and Mark A. Tarbell. "Microcomputer-
based artificial vision support system for real-time image processing for
camera-driven visual prostheses."Journal of biomedical optics 15, no.
1,pp. 016013-016013, 2010.
[11] E. Noorsal, K. Sooksood, H. Xu, R. Hornig, J. Becker, and M.
Ortmanns, ―A neural stimulator frontend with high-voltage compliance
and programmable pulse shape for epiretinal implants,‖ IEEE J. Solid-
State Circuits, vol. 47, no. 1, pp. 244–256, 2012.
[12] Zhou, D. David ., Jessy D. Dorn, and Robert J. Greenberg. "The
Argus® II retinal prosthesis system: An overview." In Multimedia and
Expo Workshops (ICMEW), 2013 IEEE International Conference on,
pp. 1-6. IEEE, 2013.
[13] N.Sharmili, P.S. Ramaiah, ― A Real- Time Research Platform for
Extraocular Image Processing for Epiretinal Prosthesis‖, International
Journal of Computer Science and Information Technologies, vol.6,
no.2,pp. 1134-1140,2011.
[14] W. Liu and M. S. Humayun, ―Retinal prosthesis,‖ in Proc. IEEE
Int.Solid-State Circuits Conf., Dig. Tech. Papers, pp. 218–219, 2004,
[15] S. H. Greenwald, A. Horsager, M. S. Humayun, R. J. Greenberg, M. J.
McMahon, and I. Fine, ―Brightness as a function of current amplitude
in human retinal electrical stimulation,‖ Invest. Ophthalmol. Visual
Sci., vol. 50, no. 11, pp. 5017–5025, Nov. 2009.
[16] Zou, Yuexian, Guangyi Shi, Yufeng Jin, and YaliZheng. "Extraocular
image processing for retinal prosthesis based on DSP." In Nano/Micro
Engineered and Molecular Systems, 2009. NEMS 2009. 4th IEEE
International Conference on, pp. 563-566. IEEE, 2009.