A Novel Approach For Colorization of A Grayscale Image Using Soft Computing Techniques

International Journal of Multimedia Data Engineering and Management
Volume 8 • Issue 4 • October-December 2017
A Novel Approach for Colorization

of a Grayscale Image using Soft
Computing Techniques
Abul Hasnat, Government College of Engineering & Textile Technology, Berhampore, India
Santanu Halder, Government College of Engineering & Leather Technology, Kolkata, India
Debotosh Bhattacharjee, Jadavpur University, Department of Computer Science and Engineering, Kolkata, India
Mita Nasipuri, Jadavpur University, Department of Computer Science and Engineering, Kolkata, India
ABSTRACT
Colorization of grayscale image is a process to convert a grayscale image into a color one. Few research
works reported in literature on this but there is hardly any generalized method that successfully
colorizes all types of grayscale image. This study proposes a novel grayscale image colorization method
using a reference color image. It takes the grayscale image and the type of the query image as input.
First, it selects reference image from color image database using histogram index of the query image
and histogram index of luminance channel of color images of respective type. Once the reference image
is selected, four features are extracted for each pixel of the luminance channel of the reference image.
These extracted features as input and chrominance blue(Cb) value as target value forms the training
dataset for Cb channel. Similarly training dataset for chrominance red(Cr) channel is also formed.
These extracted features of the reference image and associated chrominance values are used to train
two artificial neural network(ANN)- one for Cb and one for Cr channel. Then, for each pixel of the
of query image, same four features are extracted and used as input to the trained ANN to predict the
chrominance values of the query image. Thus predicted chrominance values along with the original
luminance values of the query image are used to construct the colorized image. The experiment has
been conducted on images collected from different standard image database i.e. FRAV2D, UCID.
v2 and images captured using standard digital camera etc. These images are initially converted into
grayscale images and then the colorization method was applied. For performance evaluation, PSNR
between the original color image and newly colorized image is calculated. PSNR shows that the
proposed method better colorizes than the recently reported methods in the literature. Beside this,
“Colorization Turing test” was conducted asking human subject to choose the image (closer to the
original color image) among the colorized images using proposed algorithm and recently reported
methods. In 80% of cases colorized images using the proposed method got selected.
Keywords
Adjacency Ratio, Artificial Neural Network, Clustering, Color Quantization, Colorization, Neighborhood
Statistic, Particle Swarm Optimization, PSNR, YCbCr
DOI: 10.4018/IJMDEM.2017100102

Copyright © 2017, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

19
INTRODUCTION
Color is a key attribute of an image due to its extensive use in various important fields like medical
image processing, face image processing, video processing, entertainment etc. (Gonzalez et al.,
2010; Umbaugh, 1998; Gonzalez & Woods, 2010). Colorization is a process that converts a gray
scale image to a color one. It is an active and challenging area of research with a lot of interest in
the image editing and compression community (Zhang et al., 2016; Larsson et al., 2016; Haldankar
et al., 2007; Reinhard et al., 2001; Welsh et al., 2002; Blasi et al., 2003; Levin et al., 2004; Yatziv
& Sapiro, 2006; Kang & March, 2007; Sathik & Parveen, 2010; Bugeau et al., 2014; Hasnat et al.,
2017). A few grayscale image colorization methods work successfully on different types of images.
Mainly, two types of colorization methods are found in the literature. In the first category, the user
manually colorizes some sample area in the grayscale image which is later used as source of color for
colorization of the entire grayscale image. But this process is time consuming and tedious. Also, it
requires very careful sample color selection from the user. The second category of algorithms uses a
reference color image which presumably contains semantically similar color for the grayscale query
image. This process is semi-automatic and sometimes requires user intervention in the process and
has produced some impressive colorization using user input. But sufficiently complex images still
may require many user interactions.
Colorization process has no unique solution because for a same luminance value in two different
positions in the grayscale image, there may be different chrominance value in the color image.
Although some works found in the literature, colorization still remains a challenging area. Haldankar
et al. (2007) proposed a system in 2007 which modifies a gray scale image into a color one by the
luminance effect of the reference image but the time required for colorization is huge for a large size
image and hence the method is less effective in real time system. In 2001, Reinhard et al. (2001)
proposed a method for a general color correction that takes one image’s color characteristics from
another using statistical analysis to make a synthetic image which takes on another image’s look. But
this method is not actually a method of grayscale image colorization. A semi-automatic neighborhood
statistics based colorization approach was proposed by Welsh et al. (2002). Their method, effectively
colorizes a grayscale image where they choose to transfer only chromatic information and retain the
original luminance values of the query image. Further rectangular swatch based colorization is done
for enhancing the colorized areas where the user is not satisfied with the color information. In this
method, colors are transferred between the corresponding swatches. But Welsh et. al (2002) finally
concluded that their method won’t work on images where color change is gradual such as face image.
Moreover, all these existing methods take reasonable amount of time for colorization.
In 2003, Blasi et al. (2003) proposed a fast colorization method for homogeneous images based
on a new data structure, called anti-pole tree. But it is not a method which is applicable for all
types of images. In 2005, Levin et al. (2004) presented a method based on the neighboring pixels
in space-time that have similar intensities and similar colors. This method is formalized by using a
quadratic cost function and generalized to an optimization problem which could be solved using a
standard optimization technique. In 2006, Liron Yatziv and Guillermo Sapiro proposed a method
where a graph is formed considering each pixel as node and intrinsic distance is used to measure the
similarity of colors where users manually colorizes some regions (called scribble) of the grayscale
image which are used as sample color to colorize the whole image. Sung Ha Kang and Riccardo March
proposed (Kang & March, 2007) different variational models using chromaticity color components
and colorizes the gray scale image by minimizing the total variation where sample colors should
be given manually by the user. In 2010, M. S. Sathik and N. R. S. Parveen proposed a method to
convert a gray scale X-ray image into a RGB space to diagnose the fracture properly which works for
only certain types of images (Sathik & Parveen, 2010). In 2014, Aurélie Bugeau et al. proposed an
automatic colorization process where color source pixel is selected based on the best color for each
pixel from a set of candidate pixel using pixel energy minimization method (Bugeau et al., 2014).
20
In their method, several suitable color values are computed using different features and associated
metrics and then finally best color is chosen by solving a variational model which allows the automatic
selection (Bugeau et al., 2014). But again, their method gives an unsatisfactory result for face image
colorization. Hasnat et al., 2017, proposed a patch matching image colorization but that works on
face images only. Recently in 2016, Zhang et al. (2016) proposed a generalized Convolution Neural
Network based grayscale image colorization system. In their model, initially the model is trained
using huge number of images. Once the model is trained, it may be applied to colorize any image.
Also in 2016, Larsson et al (2016) proposed a grayscale image colorization method using a deep
convolution architecture and take spatially localized multilayer slices (hyper-columns), as per-pixel
descriptors. They train the architecture and the trained model predicts the hue and chroma distributions
for each pixel for the query gray image. Still there is scope to develop a generalized grayscale image
colorization method that may work on all types of images.
This article proposes a new grayscale image colorization method where a new pixel descriptor is
developed using adjacency ratio and neighborhood statistics. This method is applicable in luminance-
chrominance color model.
The method requires building the reference color image database containing many images of
different types such as face image, tree image, sky image, river image etc. Initially each reference
image is color quantized (Celal et al., 2014; Mahamed et al., 2005; Barman et al., 2017) using Particle
Swarm Optimization and the color quantized image along with its type are stored in the database.
Then the training dataset is created using a new pixel descriptor as input and chrominance value as
target output for each color channel of color reference image.
During colorization, a grayscale query image and the type of the grayscale image are taken as
input. A reference color image is retrieved from database by comparing histogram index of luminance
channel of reference images in the database of respective type and the histogram index of query
grayscale image. Two separate Artificial Neural Network(ANN)- one for each of chrominance blue(Cb)
and chrominance red(Cr) channel are trained. Next this trained ANN predicts the chrominance values
for query grayscale image. This method is applied on various images collected from standard image
database i.e. FRAV2D color face database (FRAV2D Database, 2004), many natural images of ucidv2
(Schaefer & Stich, 2004) and images captured using standard digital camera. Initially these color
images were converted into grayscale one and the proposed colorization method were applied on them.
Evaluation of the quality of the colorized images is notoriously difficult (Zhang et al., 2016;
Larsson et al., 2016). For performance evaluation, initially color images were converted into a
grayscale images and then those images were colorized. Perceptual evaluation of the colorized images
is done using “colorization Turing test,” asking human subject to choose the image (which is closer
to the original color image) between colorized image using proposed algorithm and colorized images
using existing algorithms (Zhang et al., 2016). In 80% cases, colorized image using the proposed
algorithm got selected. Experimental result asserts the effectiveness of the proposed colorization
process obtained over a collection of images. Assessment of the quality of the colorized images was
also measured using Peak Signal to Noise Ratio (PSNR) (Blasi et al., 2003) between the original
color image and colorized image using the proposed method and two recently reported method in
literature (Zhang et al., 2016; Larsson et al., 2016). The proposed method outperforms the existing
methods in terms of PSNR also.
This paper is organized as follows: Section II explains reference color image database formation,
Section III explains the steps of the proposed colorization process. Section IV depicts the details
for interactive colorization module and Section V shows the experimental results. Lastly section VI
concludes and remarks about some of the aspects analyzed in this article.
21
BUILDING REFERENCE COLOR IMAGE DATABASE
The proposed colorization method requires a color image database of different types. All the stored
images are categorized into different types such as face images, tree images, sky images, river images
etc. Table 1 shows some images of the database along with their type. Number of images and types of
images in the database may vary according to requirement of the user. In this work, reference image
database is formed with more than 250 images.
Original color image is not stored. Color image is color quantized taking 64 clusters. Quantized
values for Cb and Cr channel along with original luminance values are stored in the database. Image
type associated with each image is also stored. Color quantization (Celal et al., 2014; Mahamed et
al., 2005; Barman et al., 2016) is a pre-processing method used to reduce the number of colors in
Table 1. A few images in reference color image database(partial) with respective type
22
images with minimum distortion such that the reproduced image should be very close to the original
image visually. In general, color quantization is performed in two steps. The first step is pallete
design- selection of appropriate number of colors (generally 8-256). The second step is pixel mapping-
replacing each pixel color with the color in the palette. In this work, Color Image Quantization using
Particle Swarm Optimization(CIQ-PSO) is applied (Mahamed et al., 2005; Barman et al., 2016).
A Color Image Quantization using PSO Algorithm (CIQ-PSO)

Color image quantization using PSO is reported to reduce number of colors in a color image (Mahamed
et al., 2005; Barman et al., 2016). Assume that our search space is d-dimensional, and the ith particle
of the swarm in tth is x i (t ) and velocity of the particle is vi (t ) . The best visited position for the
particle is pbest and the best position explored by all so far called global best is pgbest . So the particle
velocity of a particle of (t+1)th iteration, vi (t + 1) is updated using Equation 1.
vi (t + 1) = w * vi (t ) + c1 * r1 * (pbest (t ) − x i (t )) + c2 * r2 * (pgbest (t ) − x i (t )) (1)
Where w , c1 , c2 , vi (t ) are positive constants called inertia weight, cognitive learning rate,
social learning rate, velocity of particle x i (t ) on tth iteration respectively, and r1 , r2 are random value
generated in the range [0, 1] .
And the particle position of (t+1)th iteration, x i (t + 1) is updated using Equation 2.
x i (t + 1) = x i (t ) + vi (t + 1) (2)
An upper bound, Vmax is placed on the velocity in all dimensions.

For color image quantization, each particle is represented as x i = (mi,1,....., mi,k ,......, mi,k )
where mi,k refers to the kth cluster centroid vector of the ith particle. It is a K × 3 dimensional vector
where K is number of cluster for color quantization and 3 represents values for Y, Cb & Cr. Therefore,
a swarm represents a number of candidate color maps. The fitness, f (x i ) of each particle is expressed
using Equation 3.
∑ ∑ (z p − mk )2
k =1 ∀z p ∈C K
f (x i ) = (3)
NP
Where N p is number of pixel, K denotes number of cluster, z p is the Y, Cb & Cr values of the
pixel p and mk is the centroid of kth cluster. For the first population, each particle is randomly
initialized from the color image to contain K centroids (i.e. color triplets). Colormap is represented
by the set of K color triplets. The K-means clustering algorithm is then applied to each particle at a
user-specified probability pkmeans . Each pixel is then assigned to the cluster with the nearest centroid.
The fitness of each particle is calculated using Equation 3. The velocity and particle position are
updated using equation 1 and 2 respectively. The procedure is repeated till maximum number of
iterations, t max . The global best particle after tmax iterations is considered to be the final colormap.
23
CIQ-PSO Algorithm
• Step 1: Randomly initialize each particle- K color triplets from the image for first generation.
• Step 2: For t = 1 to tmax
◦◦ For each particle i.
▪▪ Apply K-means for a few iterations with a probability pkmeans .
▪▪ For each pixel z p
• Compute d 2 (z p − mi,k ) for all cluster C i,k
• Assign z p to C i,kk where
◦◦ d 2 (z p − mi,kk ) = min{ d 2 (z p − mi, ) }
k
∀k =1,..K
◦◦ Fitness f (x i ) is calculated using Equation 3.

◦◦ Update the global best solution gbest using PSO.
◦◦ Update the centroids using Equation 1 and Equation 2.
Figure 1(a), 1(c) and Figure 1(b), 1(d) show one original reference color image and color quantized
image (no of cluster=64) respectively.
Thus, in the color quantized image, chrominance values of Cb and Cr are 64, 64 respectively.
This quantized image along with its type is stored in the database. In this way, number of target color
value is reduced.
For training dataset formation, the number of target chrominance value becomes large if original
image is used. This results in large time for colorization process. For example, a standard core-i3,
2GB RAM machine requires nearly 12 minutes for colorization of grayscale image of size 800*600.
After application of CIQ-PSO on reference color image, training time is reduced to approximately
60-70seconds for the same image. Therefore CIQ-PSO increases colorization speed significantly.
The reference image database building is a onetime process. New image and type of new image
may be added in the database latter also. Once the reference color image database is formed, the user
may proceed for colorization step.
Figure 1. Color quantization a) & c) original image b) & d) Color quantized image where number of cluster=64
24
PROPOSED COLORIZATION METHOD
The proposed colorization method takes grayscale query image and its type as input. In the next step
the system retrieves a suitable color reference image from the user specified type (of the query image)
from reference color image database using histogram index. This selected reference image is used as
source of color information for the query grayscale image during colorization. The flowchart of the
colorization process is shown in Figure 2.
User gives the query grayscale image and selects the type of the query image for colorization.
First the reference color image is selected from the collection of images of respective type from the
Figure 2. Flowchart of the proposed colorization process
25
database. Reference image is selected by comparing histograms index of query image and histogram
index of luminance channel of the images from database of respective type. Once the reference image
is selected feature extraction is done in next step.
Feature Extraction
Let I S (a, b, c) be the reference color image in YCbCr color space and I S (a, b) is its grayscale image.
Here a single pixel is represented by a feature descriptor of four features- three statistical feature and
adjacency ratio. As a single luminance value of grayscale image may represent entirely different parts
of an image, therefore the statistical features within the pixel’s neighborhood are used to guide the
color search process. Pixel wise features are extracted from both reference image and query grayscale
image. Extracted features along with target chromaticity value of reference image forms the training
dataset. Extracted features of grayscale query image forms the test dataset.
Four features are extracted for each pixel: (i) Luminance (ii) Standard deviation of luminance of
5x5 patch around the pixel (iii) Mean of luminance of 5x5 patch around the pixel (iv) Adjacency ratio.
1. Luminance
The Y value of the YCbCr reference image is considered as first feature.
2. Standard Deviation
The mean value of a 5x5 sub-window around each pixel is considered as the second feature.
3. Mean
Again, for a pixel, keeping it in the middle, a 5x5 pixel window is considered values corresponding
to each pixel and mean is calculated which is taken as the third feature.
4. Adjacency Ratio
For a pixel x c with its neighbours x i where i varies from 1 to p is considered to compute
Adjacency ratio (Ullah et al., 2012). In this case 3x3 sub-window around a pixel x c is
considered where p=8. First, the pixel intensity difference between x c and x i is calculated
using Equation 4.
∆I i = I i − I c (4)
Where Ii and Ic are the intensity values of pixels x i and x c respectively. Next, the sum of the
intensity difference ratio of pixel x c is computed using Equation 5.
p −1 
∆I 
fratio = ∑  i  (5)
 I c 
i =0 
Here fratio is used as the fourth feature in the present work. These features of a reference color
image are used to create the training dataset.
Table 2 shows the sample training data set(partial) extracted from the reference color image
shown in Figure 3.
Training of Artificial Neural Network

In the proposed colorization process, initially ANN is trained with the extracted pixel information
(for each pixel a four-dimensional feature vector) of reference image.
26
Figure 3. Reference color image for features extraction
Table 2. Sample extracted features of the image in Figure 2(a)
SL Luminance Standard Deviation Mean Adjacency ratio

1 122 17.4139 136.1944 0.0163
2 121 17.4221 136.1944 0.0743
3 122 17.4172 136.1944 0.0245
4 122 17.3804 136.2500 0.0327
5 122 17.8249 135.6111 0.0081
6 122 17.8588 135.5555 0
7 121 17.9164 135.4722 0.0330
8 121 17.9734 135.3888 0.0165
9 121 18.0548 135.2777 0.0247
10 121 18.0935 135.2222 0.0165
11 121 18.1535 135.1388 0.0247
12 121 18.1916 135.0833 0.0165
27
The four features along with chromaticity value of the reference color image for each pixel is
considered as one training data point. Thus, these features of all the pixels of the reference color
image forms the training dataset for ANN. The fifth is the target value which is the chromaticity value
of the current pixel of the reference image. ANN (Haykin, 2001; Alpaydin, 2012; Han et al., 2011;
Jain et al., 1988; Gan et al., 2007), a machine learning approach, is a massive parallel distributed
processor made up of simple processing elements (PE) called neuron, which has a natural propensity
for storing experimental knowledge and making it available for use. An Artificial Neural Network
is specified by: a) Neuron model: the information processing unit of the NN, b) An architecture: a
set of neurons and links (Each link has a weight) connecting neurons and c) A learning algorithm:
used during training of ANN for modifying the weights in order to model a particular learning task
correctly on the training examples. In this work, three-layer feed-forward back propagation neural
networks have been used for color prediction.
Two ANN are trained for two separate chrominance channel namely Cb and Cr color channels.
A schematic diagram of the network used in the present work is shown in Figure 4. The three layers
of the ANN are- a) Input layer of four nodes b) Hidden layer consisting six nodes c) Output layer
containing single node.
• Input Layer: Color prediction is based on four input features so input layer contains four fanout
(these nodes forward the input to all hidden nodes) nodes.
• Hidden Layer: Hidden layer contains six hidden nodes. Each hidden node takes four weighted
input features and one bias input, sums up and finally hyperbolic tangent sigmoid transfer function
is applied. The output of hidden layer is forwarded to the output node. In the present work, ANN
with six hidden node in hidden layer is applied for colorization.
• Output Layer: Output layer contains a single neuron which takes input from six hidden nodes
and one bias value. It sums up all input and applies pure linear transfer function for producing
the output.
The network uses Levenberg-Marquardt optimization for training function that updates weight and
bias values. Gradient descent with momentum weight and bias learning function is used as learning
method (Haykin, 2001; Alpaydin, 2012; Han et al., 2011).
Increase in number of nodes in the hidden layer result in better colorization but the training time
of the ANN increases resulting slower colorization process. So in this work, the three layer ANN
model having six nodes only in the hidden layer of the network is used.
Color Prediction using Artificial Neural Network

Once two ANN (one for Cb channel and one for Cr channel) are trained using pixel features of
reference image, then each ANN is ready to predict the chromaticity values for the grayscale query
image. Now, for Cb channel the trained ANN takes a four dimensional feature vector for a pixel of
grayscale query image, it predicts the chromaticity values. This process is repeated for all pixels of
query image. Thus we get the whole Cb channel. Same process is applied to predict Cr channel also.
Now, the predicted chromaticity channels along with the original luminance channel constructs
the final colorized image. In other words, assume query image is IT (a, b) and IT (a, b, c) is the
generated color image of IT (a, b) . The luminance channel IT (a, b) are copied directly into the
luminance channel of IT (a, b, c) . The chrominance values for each pixel of the query image are
predicted values using ANN.
28
Figure 4. Architecture of the ANN for color prediction
INTERACTIVE COLORIZATION MODULE
This section presents the Interactive Colorization Module(ICM) for recolorization of a region in the
newly colorised image. This module is useful when user is not satisfied with some regions of the
colorized image. In this phase, the user can select a sample area as a sub-window IT (a, b, c ) on the
newly generated color image where recolorization is required and another sub-window I (a, b, c) on S
reference color image as a source of color for re-colorization process. Next the colorization process
is applied on the user selected regions and adjacent pixels of the user selected region which are
semantically similar. The re-colorization process is same as discussed in the previous section. Figure
5 shows the flowchart for the Interactive Colorization Module.
Suppose the user wants to recolorize the region shown in red rectangle in the colorized image
shown in Figure 6(a). So the system initially finds all the adjacent pixels similar to selected region
in terms of luminance values. All the detected pixels similar in luminance values and adjacent to
user selected region is shown in Figure 6(b). The user also selects the region which is used as color
source for recolorization process in the reference color image as shown in Figure 6(c).
Now the color values for the selected region is predicted using ANN again and the chromaticity
values are overwritten for the selected pixels in the colorized image which is shown in Figure 6(d).
EXPERIMENTAL RESULT
The proposed algorithm four grayscale image colorization has been applied on several images collected
from different standard image database i.e. UCID.V2 (Schaefer & Stich, 2004) of standard natural
images, FRAV2D (FRAV2D Database, 2004) face image database and natural images captured using
29
Figure 5. Flowchart of the Interactive colorization process
standard digital camera. For performance analysis, initially a color image is converted into gray scale
image and then it is colorized using proposed method and two recently reported methods found in
the literature (Zhang et al., 2016; Larsson et al., 2016). PSNR between the original color image and
colorized image is calculated and experimental result shows the proposed method performs better
compared to the recent methods.
Figure 7, Figure 8, Figure 9, Figure 10, Figure 11, Figure 12, Figure 13, Figure 14, Figure 15,
Figure 16, Figure 17, Figure 18, Figure 19, Figure 20, Figure 21, and Figure 22 show samples of the
colorized images using the proposed method, Zhang et. al (2016) method and Larsson et. al (2016)
method. In the below figures, each image set contains six images where the first image is the original
color image, second image is the converted grayscale image, third image is the reference color image,
30
Figure 6. Recolorization process (a) Colorized image where user selects sample area for needs recolorization (b) Selected pixels
with similar luminance values adjacent to user sample area (c) User selected area shown in rectangle in the reference color image
which is used as color source for recolorization (d) Colorized image where chromaticity values are overwritten using new values
for the pixels shown in Figure 6(c).
fourth image is the colorized image using proposed method, fifth image is the colorized image using
Zhang et. al (2016) and the last image is colorized image using the Larsson et. al (2016) method. It
is shown as follows: (a) original color image (b) grayscale query image (c) reference color image (d)
colorized image using proposed method (e) colorized image using Zhang et. al (2016) method (f)
colorized image using Larsson et. al (2016) method.
Performance evaluation of colorization method is a difficult one and Zhang et. al (2016) proposed
perceptual evaluation of the colorized images using survey called “colorization Turing test”. They
asked human subject to select better colorized image using different methods. In this work, nearly
200 volunteers (age group 20-50 years) have participated in our performance evaluation survey. Each
of the volunteers was initially shown the original color image and then they were shown colorized
images using different colorization methods. Each one of the volunteers was asked to choose a
colorized image that is closer to the original color image. In 80% cases the volunteers had chosen
the colorized images using proposed algorithm.
Peak signal-to-noise ratio (PSNR) is computed between the original color image and generated
colorized image (Blasi et al., 2003). While comparing performance of colorization process, the higher
31
Figure 7. Sample results of grayscale image colorization: Image Set 1
PSNR generally indicates better the colorization. PSNR is defined by Mean Squared Error (MSE)
using Equation 6.
 MAX 
 f 
PSNR = 20 log10   (6)
 MSE 
Where MSE is defined using Equation 7.
m n 2
1
MSE =
mn
∑ ∑ I (i, j ) − I (i, j )
c
(7)
1 1
Where I, Ic represents the matrix data of original image and the colorized image respectively.
m, n represents number of rows of and number of columns respectively in the images. MAXf is the
maximum signal value that exists in original image.
32
The PSNR of the original color image and the colorized image (which are initially converted into
grayscale image then colorized using different methods) is calculated. The PSNR for all the images
shown in Figures 7 to 22 is given in Table 3.
33
34
35
36
37
38
39
CONCLUSION
This article proposes a novel grayscale image colorization method to convert a grayscale image into a
color one with the help of a reference color image. This method is applicable for all types of images.
The experiment has been conducted on images collected from different standard image database i.e.
FRAV2D, UCID.v2 and images captured using standard camera, etc. For performance evaluation,
“Colorization Turing test” was conducted asking human subject to choose the image (close to the
original color image) between colorized image using proposed algorithm and recent methods in the
literature. In 80% of cases colorized images using the proposed method got selected. Also, the PSNR
between the original color image and newly colorized images were calculated. Also, proposed methods
perform better in terms of PSNR metric. Experimental result asserts the effectiveness of the proposed
colorization process obtained over a collection of images. However, solution to the colorization
problem is subjective rather than objective and it is not a unique one. The proposed method has a
huge application in the area of archaeology, entertainment, law enforcement etc.
40
Table 3. PSNR Ratio of chrominance channel of Original Color image and Colorized image
Image Set of Figures 7 to 22 Proposed method Zhang et al (2016) Larsson et al (2016)

Cb Cr Cb Cr Cb Cr
1 33.1600 36.2900 23.3600 28.0800 24.7400 25.7000
2 33.9300 35.0800 20.2600 13.6000 28.9100 25.3600
3 20.8000 31.8300 24.7800 28.3900 24.6900 27.5100
4 45.9900 46.8200 19.8100 21.9600 29.5400 31.8000
5 36.3300 39.8200 16.0300 15.1000 18.7200 17.3500
6 30.2700 31.9800 21.9500 25.6700 23.7700 23.9400
7 27.9900 33.6700 26.0200 33.0800 28.1200 34.7200
8 37.3800 41.0600 26.0400 33.6000 29.9600 35.9900
9 28.6100 37.2500 18.3000 25.9200 17.1400 28.8500
10 38.7800 32.2700 28.7500 25.8800 23.5200 22.6300
11 36.2100 34.1800 27.5100 26.5200 22.5800 22.6100
12 38.8100 40.6600 25.3200 26.1100 29.3400 29.8400
13 26.0700 31.4800 24.8500 21.5300 24.6200 25.1400
14 27.9900 34.0100 29.2500 33.4400 30.4300 32.9500
15 28.6900 35.1700 27.2700 33.9200 29.8900 35.6200
16 28.9900 32.8400 26.7000 32.4400 25.3000 33.9700
41
REFERENCES
Omran, M. G., Engelbrecht, A. P., & Salman, A. (2005). A Color Image Quantization Algorithm Based on
Particle Swarm Optimization. Informatica, 29, 261–269.
Alpaydin, E. (2012). Introduction to machine learning (2nd ed.). New Delhi: PHI Learning Private Limited.
Barman, D., Hasnat, A., Sarkar, S., & Rahman, A. (2016). Color Image Quantization using Gaussian Particle
Swarm Optimization (CIQ-GPSO). Proc. of International Conference on Inventive Computation Technologies,
Coimbatore, India. doi:10.1109/INVENTIVE.2016.7823295
Blasi, G. D., & Recupero, D. R. (2003). Fast Colorization of Gray Images.
Bugeau, A., Ta, V., & Papadakis, N. (2014). Variational Exemplar-Based Image Colorization. IEEE Transactions
on Image Processing, 23(1), 298–307. doi:10.1109/TIP.2013.2288929 PMID:24235307
Celal, O., Emrah, H., & Dervis, K. (2014). Color Image Quantization: A short Review and an Application with
Artificial Bee Colony Algorithm. INFORMATICA, 25(3), 485–503. doi:10.15388/Informatica.2014.25
FRAV2D Database. (2004). Retrieved from http://www.frav.es/databases/frav2d/
Gan, G., Ma, C., & Wu, J. (2007). Data Clustering Theory, Algorithms and Applications. SIAM.
doi:10.1137/1.9780898718348
Gonzalez, R. C., & Woods, R. E. (2010). Digital Image Processing. Boston, MA, USA: Addison Wesley.
Gonzalez, R. C., Woods, R. E., & Eddins, S. L. (2010). Digital Image processing using MATLB. New Delhi,
India. Tata: Mc-Graw Hill.
Haldankar, G., Tikare, A., & Patil, J. (2007). Converting Gray Scale Image to Color Image. Proceedings of the
SPIT-IEEE Colloquium and International Conference (Vol. 1, pp. 189-192).
Han, J., Kamber, M., & Pei, J. (2011). Data mining Concepts and Techniques (3rd ed.). Waltham, USA: Morgan
Kaufmann Publishers An imprint of Elsevier.
Hasnat, A., Halder, S., Bhattacharjee, D., & Nasipuri, M. (2017). A Proposed Grayscale Face Image Colorization
System using Particle Swarm Optimization. International Journal of Virtual and Augmented Reality, 1(1),
72–89. doi:10.4018/IJVAR.2017010106
Haykin, S. (2001). Neural Networks A comprehensive foundation (2nd ed.). New Delhi: Pearson Education.
Jain, A. K., & Dubes, R. C. (1988). Algorithms for Clustering Data. Upper Saddle River, NJ, USA: Prentice-
Hall, Inc.
Kang, S. H., & March, R. (2007). Variational Models for Image Colorization via Chromaticity and Brightness
Decomposition. IEEE Transactions on Image Processing, 16(9), 2251–2261. doi:10.1109/TIP.2007.903257
PMID:17784599
Larsson, G., Maire, M., & Shakhnarovich, G. (2016). Learning Representations for Automatic Colorization.
Proceedings of the European Conference on Computer Vision (pp. 577-593). Springer International Publishing.
Levin, A., Lischinski, D., & Weiss, Y. (2004). Colorization using Optimization. ACM Transactions on Graphics,
23(3), 689–694. doi:10.1145/1015706.1015780
Reinhard, E., Ashikhmin, M., Gooch, B., & Shirley, P. (2001). Color Transfer between Images. IEEE Computer
Graphics and Applications, 21(5), 34–41. doi:10.1109/38.946629
Sathik, M. M., & Parveen, N. R. S. (2010). Feature Extraction on Colored X-Ray images by Bit-Plane Slicing
Technique. International Journal of Engineering Science and Technology, 2(7), 2820–2824.
Schaefer, G., & Stich, M. (2004). UCID - An Uncompressed Colour Image Database. Proc. of SPIE, Storage
and Retrieval Methods and Applications for Multimedia, San Jose (pp. 472-480),
Ullah, I., Hussain, M., Muhammad, G., Aboalsamh, H., Bebis, G., & Mirza, A. M. (2012). Gender recognition
from face images with local wld descriptor (pp. 417–420). IWSSIP.
42
Umbaugh, S. E. (1998). Computer vision and image Processing. NJ, USA: Prentice Hall.
Welsh, T., Michael, A., & Klaus, M. (2002). Transferring Color to Greyscale Images. ACM Transactions on
Graphics, 21(3), 277–280. doi:10.1145/566654.566576
Yatziv, L., & Sapiro, G. (2006). Fast Image and Video Colorization Using Chrominance Blending. IEEE
Transactions on Image Processing, 15(5), 1120–1129. doi:10.1109/TIP.2005.864231 PMID:16671293
Zhang, R., Isola, P., & Efros, A. A. (2016). Colorful Image Colorization. Proceedings of the European Conference
on Computer Vision (pp. 649-666). Springer International Publishing.
43

A Novel Approach For Colorization of A Grayscale Image Using Soft Computing Techniques

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Novel Approach For Colorization of A Grayscale Image Using Soft Computing Techniques

Uploaded by

Copyright:

Available Formats

International Journal of Multimedia Data Engineering and Management

Volume 8 • Issue 4 • October-December 2017

A Novel Approach for Colorization

BUILDING REFERENCE COLOR IMAGE DATABASE

A Color Image Quantization using PSO Algorithm (CIQ-PSO)

vi (t + 1) = w * vi (t ) + c1 * r1 * (pbest (t ) − x i (t )) + c2 * r2 * (pgbest (t ) − x i (t )) (1)

An upper bound, Vmax is placed on the velocity in all dimensions.

◦◦ Fitness f (x i ) is calculated using Equation 3.

PROPOSED COLORIZATION METHOD

Figure 2. Flowchart of the proposed colorization process

Training of Artificial Neural Network

Figure 3. Reference color image for features extraction

Table 2. Sample extracted features of the image in Figure 2(a)

SL Luminance Standard Deviation Mean Adjacency ratio

Color Prediction using Artificial Neural Network

Figure 4. Architecture of the ANN for color prediction

INTERACTIVE COLORIZATION MODULE

Figure 5. Flowchart of the Interactive colorization process

Figure 7. Sample results of grayscale image colorization: Image Set 1

Where MSE is defined using Equation 7.

Figure 8. Sample results of grayscale image colorization: Image Set 2

Figure 9. Sample results of grayscale image colorization: Image Set 3

Figure 10. Sample results of grayscale image colorization: Image Set 4

Figure 11. Sample results of grayscale image colorization: Image Set 5

Figure 12. Sample results of grayscale image colorization: Image Set 6

Figure 13. Sample results of grayscale image colorization: Image Set 7

Figure 14. Sample results of grayscale image colorization: Image Set 8

Figure 15. Sample results of grayscale image colorization: Image Set 9

Figure 16. Sample results of grayscale image colorization: Image Set 10

Figure 17. Sample results of grayscale image colorization: Image Set 11

Figure 18. Sample results of grayscale image colorization: Image Set 12

Figure 19. Sample results of grayscale image colorization: Image Set 13

Figure 20. Sample results of grayscale image colorization: Image Set 14

Figure 21. Sample results of grayscale image colorization: Image Set 15

Figure 22. Sample results of grayscale image colorization: Image Set 16

Image Set of Figures 7 to 22 Proposed method Zhang et al (2016) Larsson et al (2016)

You might also like