minerals

Article
An Enhanced Rock Mineral Recognition Method
Integrating a Deep Learning Model
and Clustering Algorithm
Chengzhao Liu 1 , Mingchao Li 1, * , Ye Zhang 1 , Shuai Han 1 and Yueqin Zhu 2
1 State Key Laboratory of Hydraulic Engineering Simulation and Safety, Tianjin University,
Tianjin 300354, China
2 Development Research Center of China Geological Survey, Beijing 100037, China
* Correspondence: lmc@tju.edu.cn




Received: 26 May 2019; Accepted: 23 August 2019; Published: 26 August 2019


Abstract: Rock mineral recognition is a costly and time-consuming task when using traditional
methods, during which physical and chemical properties are tested at micro- and macro-scale in
the laboratory. As a solution, a comprehensive recognition model of 12 kinds of rock minerals
can be utilized, based upon the deep learning and transfer learning algorithms. In the process,
the texture features of images are extracted and a color model for rock mineral identification can also
be established by the K-means algorithm. Finally, a comprehensive identification model is made by
combining the deep learning model and color model. The test results of the comprehensive model
reveal that color and texture are important features in rock mineral identification, and that deep
learning methods can effectively improve identification accuracy. To prove that the comprehensive
model could extract effective features of mineral images, we also established a support vector machine
(SVM) model and a random forest (RF) model based on Histogram of Oriented Gradient (HOG)
features. The comparison indicates that the comprehensive model has the best performance of all.

Keywords: rock mineral; classification recognition; deep learning model; transfer learning model;
clustering algorithm

1. Introduction
The classification and identification of rock minerals are indispensable in geological works.
The traditional recognition methods are based on the physical and chemical properties of rock minerals,
which are used to identify rock minerals at macro- and micro-scales [1,2]. With the development of
computer science and artificial intelligence, the recognition model can be established using machine
learning methods [3–7]. Traditional recognition methods have definite physical meaning; while machine
learning methods are driven by data [8,9]. The two methods have their own strengths and weaknesses.
The significant strength of traditional identification model is that the model can be explained.
Vassilev and Vassileva [10,11] analyzed the optical properties of low-temperature ash (LTA) and
high-temperature ash (HTA) of the coal samples to get the weight of each mineral in the crystalline
matter basis. Then they classified the inorganic matter in coal using the composition from the HTA
and got a good result. But the experiment environment and operating procedures of this method
are difficult to implement. Zaini [12] used wavelength position, linear spectral un-mixing and a
spectral angle mapper to explore carbonate mineral compositions and distribution on the rock mineral
surface and got a good result. Moreover, Zaini’s method needs SisuCHEMA SWIR sensor, which is
integrated with a computer workstation to collect the data. Adep [13] developed an expert system
for hyperspectral data classification with neural network technology. Their system was also used to

Minerals 2019, 9, 516; doi:10.3390/min9090516 www.mdpi.com/journal/minerals

Minerals 2019, 9, 516 2 of 17

calculate the depth and distribution of iron ore in a specific engineering project. The scholars chose the
key factors to build the model. However, the professional knowledge and equipment are necessary for
the process. It is not an efficient and economical way to conduct traditional model sometimes.
In recent years, machine learning methods have been applied in image recognition, including
sensing images [14,15] and mineral images [16]. Multi-source and multi-type mineral datasets are
suitable for machine learning methods [17,18]. Moreover, the application of machine learning in the
identification of rock images has been proved useful and efficient [19–23]. Aligholi et al. [24] classified
45 slices of 15 kinds of rock minerals automatically based on the color of the rock minerals and achieved
high accuracy. Li et al. [25] applied feature selection and transfer learning to classify microscopic
images of sandstone. The feature engineering is significant in machine learning model establishment.
However, it is easy to be influenced by subjective factors. As a result, it is necessary to develop an
end-to-end mineral identification model.
At present, deep learning algorithms have been developed and studied widely. Google introduced
the design and application of a deep learning framework, which was flexible to build deep learning
models [26]. Moreover, deep learning methods have achieved a remarkable prominence in recognition
of rock mineral images. Zhang et al. [27] retrained Inception-v3 to identify granite, phyllite and breccia,
and thereby proved that the Inception-v3 model was suitable for rock identification. The neuro-adaptive
learning algorithm was also used to classify the iron ore real-timely [28]. Lots of researchers studied
the application of deep learning in geochemical mapping [29], mineral prospectivity mapping [30] and
micro-scale images of quartz and resin [31]. It proves that deep learning models are outstanding in
complex image data processing and analysis. While, it is hard to improve deep learning model, because
the deep learning architecture is fixed. The model ensemble can be adopted to solve the problem.
In this paper, a comprehensive model for rock mineral identification was developed by combining
the Inception-v3 model and a color model. First, as shown in Section 3.1, the texture features of
rock minerals in the image were extracted based on the color and brightness of rock minerals. Then,
the Inception-v3 model was retrained with these preprocessed images. Moreover, a color model was
proposed (as shown in Section 3.2) to recognize the mineral images based on the result of Inception-v3
model. Finally, the comprehensive identification model based on the deep learning model and color
model can be established (as shown in Section 4). We also constructed two models using SVM
and RF-based on Histogram of Oriented Gradient (HOG) features. The comparison of the three
models indicates that the comprehensive model can achieve significant performance. In this research,
the elaborate mineral images are adopted, whose features are obvious. We want to study whether the
comprehensive model can extract the obvious features, such as cleavage and luster. If effective features
can be extracted, the model can be used in a larger domain.

2. Methodology
This research is based on the deep learning method, transfer learning and clustering algorithm.
The overall of the method is shown in Figure 1. The process can be divided into two parallel ones. One
of them cut the raw images (images without any pre-process) into pieces. Then T1 is calculated, and the
color model is established using these image slices. T1 is broadcasted to the other process at the same
time. The other process will calculate and outline the binary points in the raw images once received T1 .
Then the processed images are going to be trained using the transfer learning method based on deep
learning method. Finally, a comprehensive model is established, combining the Inception-v3 model
and color model. T and T1 are thresholds defined in Section 3.1.
Minerals 2019, 9, 516 3 of 17
Minerals 2019, 9, x FOR PEER REVIEW 3 of 16

Raw image data

Mineral image
slices

Calcula te the gray Calcula te the color K-mea ns

value of each point value of each point calculation
T1

If the value changes If the value changes

more than T, set the more than T1, set the
point as binary point point as binary point

Preprocesse d
image data

Transfe r learning

Comprehensive model

Inception-v3 Color
model model

Results from
Comprehensive
Image data Inception-v3
result
Identific ation by model
Inception-v3 model Identific ation based
on color model and
results from
Inception-V3 model

Figure 1.
Figure The training
1. The training process
process and
and identification
identification process
process of a comprehensive model.

2.1. Deep Learning Algorithm

2.1. Deep Learning Algorithm
Deep learning was proposed by Hinton and Salakhutdinov [32] and has been widely applied
Deep learning was proposed by Hinton and Salakhutdinov [32] and has been widely applied in
in semantic recognition [33], image recognition [34], object detection [35], and the medical field [36].
semantic recognition [33], image recognition [34], object detection [35], and the medical field [36]. In
In image recognition, the end-to-end training mode is adopted in the training process. In each iteration
image recognition, the end-to-end training mode is adopted in the training process. In each iteration
of a training process, each layer can adjust its parameters according to the feedback from training data
of a training process, each layer can adjust its parameters according to the feedback from training
to improve the accuracy of the model. Moreover, the nonlinear function mapping ability is added to
data to improve the accuracy of the model. Moreover, the nonlinear function mapping ability is
the deep learning model. As a consequence, it reduces the computational time for extracting features
added to the deep learning model. As a consequence, it reduces the computational time for
from images and improves the accuracy of the deep learning model.
extracting features from images and improves the accuracy of the deep learning model.
In this paper, the Inception-v3 model [37] was set as a pre-trained model. In the Inception-v3
In this paper, the Inception-v3 model [37] was set as a pre-trained model. In the Inception-v3
model, there are five convolutional layers and two max-pooling layers in front of the net structure.
model, there are five convolutional layers and two max-pooling layers in front of the net structure.
To reduce the computation cost, the convolution kernel is converted from one 5 × 5 matrix to two
To reduce the computation cost, the convolution kernel is converted from one 5 × 5 matrix to two 3 ×
3 × 3 matrixes. Then, 11 mixed layers follow. These mixed layers could be divided into three categories,
3 matrixes. Then, 11 mixed layers follow. These mixed layers could be divided into three categories,
named block module 1, block module 2 and block module 3. Block module 1 has three repetitive mixed
named block module 1, block module 2 and block module 3. Block module 1 has three repetitive
layers, and each layer is connected by a contact layer. The input size of first mixed layer is (35,35,192),
mixed layers, and each layer is connected by a contact layer. The input size of first mixed layer
and output size is (35,35,256). The rest mixed layers’ input, and output size are (35,35,288) in the block
is(35,35,192), and output size is (35,35,256). The rest mixed layers’ input, and output size are
module 1. Batch size is the number of training samples in each step. In the front of block module 2,
(35,35,288) in the block module 1. Batch size is the number of training samples in each step. In the
there is a small mixed layer, whose input size is (35,35,288) and the output size is (17,17,768). The rest
front of block module 2, there is a small mixed layer, whose input size is (35,35,288) and the output
of the mixed layers’ input and output size are (17,17,768) in the block module 2. In block module 3,
size is (17,17,768). The rest of the mixed layers’ input and output size are (17,17,768) in the block
the input size is (17,17,768) and output size is (8,8,2048). After the mixed layer, a convolutional layer,
module 2. In block module 3, the input size is (17,17,768) and output size is (8,8,2048). After the
mean pooling layer and dropout layer is followed to extracted image features. Finally, the softmax
mixed layer, a convolutional layer, mean pooling layer and dropout layer is followed to extracted
classifier is trained at the end of the net.
image features. Finally, the softmax classifier is trained at the end of the net.
In the process of convolution, a color image is converted into a 3D matrix including all RGB
values; then, each pixel is evaluated using the kernel and pooling during each iteration. The average
pooling layer and max-pooling layer could get the average value and max value of a matrix,
Minerals 2019, 9, 516 4 of 17

Minerals 2019, 9, x FOR PEER REVIEW 4 of 16

In the process of convolution, a color image is converted into a 3D matrix including all RGB values;
then, each pixel
respectively. is evaluated
Finally, using the
the common kernelcan
features andbepooling
extractedduring
from each iteration.
a large numberTheof average
images.pooling
In the
layer
mixed and max-pooling
layer, convolutionlayer could
and get the
pooling areaverage value andwhich
synchronized, max value
is ableofto
a matrix,
achieverespectively. Finally,
better performance
the common
in feature features can be extracted from a large number of images. In the mixed layer, convolution
extraction.
and pooling are synchronized, which is able to achieve better performance in feature extraction.
2.2. Transfer Leaning Method
2.2. Transfer Leaning Method
According to transfer learning, the parameters in trained models from clustering domains can
According
be utilized to transfer
to establish learning,
a new modelthe parameters
efficiently. In in trained
a new taskmodels
of imagesfromrecognition,
clustering domains can be
the parameters
utilized to establish
of a pre-trained a new
model aremodel
set asefficiently.
the initialInparameters.
a new task Transfer
of imageslearning
recognition, the parameters
can help of a
scholars avoid
pre-trained model are set as the initial parameters. Transfer learning can help
building a model from scratch and make full use of the data, which reduces computational cost andscholars avoid building a
model from scratch and
dependence on big datasets. make full use of the data, which reduces computational cost and dependence
on bigIndatasets.
this research, the Inception-v3 model was selected as the pre-trained model for rock mineral
imageIn recognition.
this research,Thethe pre-trained
Inception-v3deep model was selected
learning model as the pre-trained
is able model for rockand
to reduce computational mineral
time
image
cost when a new model is trained. Therefore, transfer learning based on a deep learning modelcost
recognition. The pre-trained deep learning model is able to reduce computational and time has
when a new model
been widely is trained.
used [38,39]. TheTherefore,
original data transfer
set oflearning based onmodel
the Inception-v3 a deepcontains
learningabout
model1.2
has been
million
widely
imagesusedfrom[38,39]. The 1000
more than original data set The
categories. of the Inception-v3
model involvesmodel
about contains
25 millionabout 1.2 million
parameters. images
However,
from
with more than 1000
the transfer categories.
learning method, Thethe
model involves
training about
process 25 million
based on thisparameters.
model can be However,
quickly with the
finished
transfer learning method, the training process based on this model can be quickly
even though the computer has no GPU in general. Figure 2 shows the process of transfer learning finished even though
the computer has no GPU in general. Figure 2 shows the process of transfer learning method.
method.

Images of Convolution layers Labels of

Softmax
ImageNet and pooling layers ImageNet

Transfer learning

Rock mineral Feature Features of rock Result of rock mineral

Softmax
images extraction mineral images image classification

Figure2.
Figure 2. Transfer
Transferlearning
learningwith
withInception-v3.
Inception-v3.

2.3.
2.3. Clustering
Clustering Algorithm
Algorithm
The
The clustering
clustering algorithm
algorithm has
has aa mass
mass application
application in
in pattern
pattern recognition
recognition (PR)
(PR) and
and is
is based
based on
on the
the
Euclidian
Euclidian Distance. Absolute distance is the main point of Euclidian Distance, which measures the
Distance. Absolute distance is the main point of Euclidian Distance, which measures the
clustering
clusteringofoflots of of
lots datasets by absolute
datasets by absolutedistance. Reasonably,
distance. a rankawill
Reasonably, rankbe will
proposed by the distance
be proposed by the
size of thesize
distance dataset.
of the dataset.
In
Inthis
thisresearch,
research,thethe
color model
color is established
model based on
is established the K-means
based algorithm.algorithm.
on the K-means In the algorithm,
In the
the absolute distances of objects are calculated, and the objects that share the closest absolute
algorithm, the absolute distances of objects are calculated, and the objects that share the closest distances
are in the distances
absolute same category.
are inFinally, a silhouette
the same category.coefficient
Finally, a is used to evaluate
silhouette theisclustering
coefficient effect: the
used to evaluate
clustering effect:
b𝑏i −−a𝑎i
Si = 𝑖 𝑖 . (1)
𝑆𝑖 = max(ai , bi ) . (1)
𝑚𝑎𝑥( 𝑎𝑖 , 𝑏𝑖 )
In Equation
In Equation (1),
(1), aai is
is the
the average
average distance
distance from
from the the ith
ith point
point to
to other
other points
points inin the
the same
same kind,
kind,
i
which is called intra-cluster dissimilarity; bi is the average distance from the ith point to the points in
which is called intra-cluster dissimilarity; bi is the average distance from the ith point to the points
any
in kind,
any which
kind, which is called
is called inter-cluster dissimilarity.
inter-cluster dissimilarity. WhenWhen the the
number
numberof kinds is more
of kinds thanthan
is more 3, the
3,
average value is set as a silhouette coefficient. The silhouette coefficient is in [−1, 1]. In
the average value is set as a silhouette coefficient. The silhouette coefficient is in [−1, 1]. In the training the training
colormodel,
color model,thethemodel
modelreceiving
receivingthe
themaximum
maximumsilhouette
silhouettecoefficient
coefficientisisgoing
goingtotobe
beestablished.
established.
Minerals 2019, 9, 516 5 of 17

2.4. Support Vector Machine

Support Vector Machine (SVM) is a supervised learning method. When given a binary classification
dataset xn , yn N
 D  N
n=1 with xn ∈ R and yn ∈ [−1, 1], for each (xn , yn ) belongs to xn , yn n=1 . An SVM
model can be established by the equation:

f (x) = ωT x + b. (2)

The optimal weight vector wT and bias b are obtained by the Hinge Loss Function [40]. In this
research, the Histogram of Oriented Gradient (HOG) algorithm was used to extract the feature of rock
mineral images and then an SVM model was trained with these features.

2.5. Random Forest

Random Forest (RF) is proposed by Leo Breiman. The steps of establishing an RF model are
as following:
Step 1: Input T as training dataset size and M as feature numbers.
Step 2: Input the feature number of each node (m, m << M) to get the decision result at a node in
the tree.
Step 3: Select K samples with playback from a dataset N randomly. Then, K taxonomic trees are
constructed by these selected samples. The rest of the dataset is used to test the model.
Step 4: Randomly select m features at each node and find the best splitting mode by the
selected features.
Step 5: Each tree grows completely without pruning and may be adopted after a normal tree
classifier is built.

3. Algorithm Implementation
Rock minerals have unique physical properties, such as color, streak, transparency, aggregate
shape and luster [41]. By analyzing their properties, rock minerals can be identified more accurately.
Table 1 shows the properties of 12 kinds of rock minerals.

Table 1. Twelve kinds of rock mineral’s physical properties.

Minerals Color Streak Transparency Aggregate Shape Luster

Cinnabar Red Red Translucent Granule, massive form Adamantine luster
From metallic luster to
Hematite Red Cherry red Opaque Multi-form
soil state luster
Granule, massive form,
Colorless or
Calcite White Translucent threadiness, stalactitic Glassy luster
white
form, soil state
From Emulsions, massive,
Waxy luster, glassy
Malachite Green Pale green translucent to incrusting, concretion
luster, soil state luster
opaque forms or threadiness
Granule, stalactitic,
Azurite Navy blue Pale blue Opaque Glassy luster
incrusting, soil state
Determine by
Aquamarine White Transparent Cluster form Glassy luster
its component
Pale green, Granule, radial pattern,
Augite Black Opaque Glassy luster
black massive
Magnetite Black Black Opaque Granule, massive form Metallic luster
Clintheriform, scaly
Molybdenite Gray Light gray Opaque Metallic luster
form or lobate form
Massive, granule or
Stibnite Gray Dark gray Opaque Strong metallic luster
radial pattern
Crineus, yellow, White, pale From opaque to Adamantine luster, sub
Cassiterite Irregular granule
black brown transparent adamantine luster
Transparent, Clintheriform, massive, Glassy luster, nacreous
Gyp White, colorless White
translucent threadiness luster
Minerals 2019, 9,
Minerals 2019, 9, 516
x FOR PEER REVIEW 66 of
of 17
16

In Table 1, color is a key feature of rock minerals in rock mineral image recognition, because
In Table 1, color is a key feature of rock minerals in rock mineral image recognition, because
different compositions of rock minerals lead to different colors. Also, the composition of different
different compositions of rock minerals lead to different colors. Also, the composition of different rock
rock minerals usually changes, and the colors vary widely. Some rock minerals with a single
minerals usually changes, and the colors vary widely. Some rock minerals with a single composition
composition have a pure color. For example, cinnabar is usually red, malachite is green, and calcite is
have a pure color. For example, cinnabar is usually red, malachite is green, and calcite is white.
white. A rock mineral often shows a mixed color on the surface because of mixed composition,
A rock mineral often shows a mixed color on the surface because of mixed composition, which is often
which is often indicated by binomial nomenclature, such as lime-green. However, whether a
indicated by binomial nomenclature, such as lime-green. However, whether a particular type of rock
particular type of rock mineral has a pure or mixed composition, those in that classification always
mineral has a pure or mixed composition, those in that classification always share some certain kinds
share some certain kinds of colors. Therefore, color is an important classification criterion.
of colors. Therefore, color is an important classification criterion.
The texture is also an important feature for rock mineral image recognition. The main
The texture is also an important feature for rock mineral image recognition. The main components
components of rock mineral texture are aggregate shape and cleavage. Different rock minerals have
of rock mineral texture are aggregate shape and cleavage. Different rock minerals have different
different distinctive textures. For example, cinnabar’s aggregate is granular, with perfect cleavage
distinctive textures. For example, cinnabar’s aggregate is granular, with perfect cleavage and uneven
and uneven to sub-conchoidal fracture; aquamarine’s aggregate is cluster form, with imperfect
to sub-conchoidal fracture; aquamarine’s aggregate is cluster form, with imperfect cleavage and
cleavage and conchoidal to irregular fracture; malachite’s aggregate is multi-form, as shown in
conchoidal to irregular fracture; malachite’s aggregate is multi-form, as shown in Figure 3. The texture
Figure 3. The texture is also helpful in recognizing rock minerals. Therefore, the combination of color
is also helpful in recognizing rock minerals. Therefore, the combination of color and texture can
and texture can improve the accuracy of the rock mineral recognition model.
improve the accuracy of the rock mineral recognition model.

(a) (b) (c)

Figure
Figure3.3.Samples ofof
Samples rock mineral
rock characteristics
mineral [42].
characteristics (a)(a)
[42]. Aquamarine; (b)(b)
Aquamarine; cinnabar; (c)(c)
cinnabar; hematite.
hematite.

3.1.
3.1. Mineral
Mineral Texture
Texture Feature
Feature Extraction
Extraction
The
The visual
visual features
features ofof rock
rock minerals
minerals also
also include
include the
the texture
texture ofof rock
rock mineral
mineral aggregates
aggregates andand
cleavage. In light conditions, the reflection of different cleavages is different. Meanwhile,
cleavage. In light conditions, the reflection of different cleavages is different. Meanwhile, the the reflection
of a given of
reflection rock mineral
a given follows
rock a pattern
mineral followsbecause
a pattern ofbecause
its own ofcharacteristics. Therefore, Therefore,
its own characteristics. the texturethe
of
rock minerals can be extracted based on the brightness and color variation of images.
texture of rock minerals can be extracted based on the brightness and color variation of images.
In
In an
an area
area where
where the
the brightness
brightness changes
changes significantly
significantly inin the
the image,
image, the
the rock
rock mineral
mineral texture
texture can
can
be
be outlined. According to the RGB color system, color images can be converted to grayscale to
outlined. According to the RGB color system, color images can be converted to grayscale to show
show
brightness
brightness variations
variations more
more effectively.
effectively.The
Theconversion
conversionrule rule[43]
[43]isisas
asfollows:
follows:
𝑔𝑟𝑎𝑦 = 0.299 × 𝑅 + 0.587 × 𝐺 + 0.114 × 𝐵, (3)
gray = 0.299 × R + 0.587 × G + 0.114 × B, (3)
where, gray is the grayscale value of a pixel, and R, G and B are the three primary color values of the
pixel. The
where, graygray
is thevalue of thevalue
grayscale image ofcan be calculated
a pixel, and R, G and by Equation
B are the(3). In primary
three the lightcolor
condition,
valuestheof
brightness
the in the
pixel. The grayopaque
value ofareatheofimage
a rockcanmineral image by
be calculated is distinctly different
Equation (3). In thefrom
lightthat of the
condition,
surrounding
the brightnesspixels.
in theIf opaque
the rockarea
mineral’s luster
of a rock is brilliant
mineral imagein the image, some
is distinctly pixels from
different achieve a higher
that of the
brightness value
surrounding than
pixels. thatrock
If the of their surrounding
mineral’s luster isarea, while
brilliant inthe
the rest of the
image, somepixels do achieve
pixels not. If the luster
a higher
brightness value than that of their surrounding area, while the rest of the pixels do not. If the luster ofa
of a rock mineral is dull in the image, which indicates that it has a low reflection, pixels achieve
alower
rock brightness value
mineral is dull inthan their surrounding
the image, pixels,
which indicates thatasitshown in Figure
has a low 4. pixels achieve a lower
reflection,
brightness value than their surrounding pixels, as shown in Figure 4.
 Minerals 2019, 9, 516 7 of 17
Minerals 2019, 9, x FOR PEER REVIEW 7 of 16

(a) (b)

(c) (d)
Figure
Figure4.4.The
Thebrightness
brightnessdistribution
distribution of
of rock
rock mineral images.
images. (a)
(a) Calcite;
Calcite;(b)
(b)brightness
brightnessdistribution
distributionof
ofcalcite
calciteimage;
image;(c)
(c)cinnabar;
cinnabar;(d)
(d)brightness
brightnessdistribution
distributionofofcinnabar
cinnabarimage.
image.

InInthe
the same
same light
lightcondition,
condition,the brightness
the brightness value fluctuates
value around
fluctuates a fixed
around a value, and theand
fixed value, changed
the
value of the fluctuation is set as
changed value of the fluctuation is set |∆Z |.
i as |Δ𝑍𝑖 |. The brightness value dramatically changes if it the
The brightness value dramatically changes if it is beside is
texture and the value is far more
beside the texture and the value is farthan |∆Zmore
i |, namely |Δ𝑍𝑖>>
than |∆Z| |∆Zi |. The
|, namely variation
|Δ𝑍| >> |Δ𝑍𝑖 |.forThe
a pixel is calculated:
variation for a
pixel is calculated:
∆Zi = grayi − gray, (4)
Δ𝑍𝑖 = 𝑔𝑟𝑎𝑦𝑖 − 𝑔𝑟𝑎𝑦, (4)
where,𝛥𝑍
where, ∆Z𝑖 iisisthe
thevariation
variationofofbrightness
brightnessfor
foreach
eachpixel
pixeland
andthe
thegrayscale
grayscalevalue
valuefor forpixel
pixelpoints
pointsisis
determined by Equation (3). Moreover, gray is the mean value of the grayscale of the
determined by Equation (3). Moreover, gray is the mean value of the grayscale of the central pixelcentral pixel
and
and 8 pixels around the central pixel. In order to eliminate edge influence, the maximum
8 pixels around the central pixel. In order to eliminate edge influence, the maximum and minimum are and
minimum
removed,are removed,
as shown as shown (5)
in Equations in Equations
and (6). (5) and (6).
n 9
gray  ( grayi  graymax  graymin ) / 7 , (5)
n =9
Xi 1
gray = ( grayi − graymax − gray7min )/7, (5)
i=1| 1
|Δ𝑍𝑖 |′ = ||Δ𝑍 𝑖 − |Δ𝑍|| = ||Δ𝑍𝑖 | − ∑|Δ𝑍𝑘 ||, (6)
7
 𝑘=1
7

where, the grayi is the value|∆Z ofi |0the

ith i |pixel’s
  1 X 𝑔𝑟𝑎𝑦 
and 𝑔𝑟𝑎𝑦𝑚𝑖𝑛 are the
= |∆Z − |∆Z| grayscale
= |∆Zi | − value,|∆Z k |, 𝑚𝑎𝑥 (6)
7 calculated, |𝛥𝑍 ′
maximum and minimum grayscale values of the pixels 𝑖 | is the variation of
 being k =1

the ith grayscale
where, the grayi pixel.
is the T is selected
value of the ith as pixel’s
the threshold
grayscalevalue.value,After
gray experimentation, we found that a
max and graymin are the maximum
′
value of T = 15 was suitable. If the point satisfies
and minimum grayscale values of the pixels being calculated, |Δ𝑍 𝑖 | > T, it will be
0 set as a feature point.
|∆Zi | is the variation of the ith grayscale
In Taddition,
pixel. theastexture
is selected featuresvalue.
the threshold couldAfter be extracted from color
experimentation, wechanges
found that at the interface
a value of T =of15rock
was
minerals. Therefore, the color change 0 at the interface of a
suitable. If the point satisfies |∆Zi | > T, it will be set as a feature point. rock mineral could be measured by the
variation of RGB proportion. Three linearly independent indexes, namely C1, C2 and C3 were
selected, which were calculated in Equation (7). Moreover, n was set as 0.01 to avoid a denominator
of zero.
𝐶1 = 𝑅/(𝐺 + 𝑛), 𝐶2 = 𝐺/(𝐵 + 𝑛), 𝐶3 = 𝐵/(𝑅 + 𝑛), (7)
Minerals 2019, 9, 516 8 of 17

In addition, the texture features could be extracted from color changes at the interface of rock
minerals. Therefore, the color change at the interface of a rock mineral could be measured by the
variation of RGB proportion. Three linearly independent indexes, namely C1 , C2 and C3 were selected,
which were calculated in Equation (7). Moreover, n was set as 0.01 to avoid a denominator of zero.

C1 = R/(G + n), C2 = G/(B + n), C3 = B/(R + n), (7)

where, C was calculated to evaluate the variation of C1 , C2 and C3 comprehensively, as shown in

Equation (7).
p
C = K C1 2 + C2 2 + C3 2 , (8)

where, K is the expansion coefficient, and in this research, K was set to 1. C will change sharply when
any one of C1 , C2 and C3 changes. ∆C was set as the maximum difference in coefficients:

∆C = max{∆Ci }, (9)

where, ∆Ci wasMinerals

the 2019,
difference
9, x FOR PEERbetween
REVIEW the pixel and the surrounding pixels, and 8 ofT16
1 was used as a
threshold value to determine whether the point was a texture boundary point; and T’ was a matrix,
where, C was calculated to evaluate the variation of C1, C2 and C3 comprehensively, as shown in
which was used (7). the maximum distances within each class. T 1 indicated the max distance
to store
Equation
between the 12 rock minerals, which was 𝐶 = calculated
𝐾√𝐶1 2 + 𝐶2 2 + by
𝐶3 2 , K-means. First, a matrix which (8) contains the
RGB values of every
where, Ktype of rock mineral
is the expansion image
coefficient, and in thiswas calculated
research, K was set toby
1. CK-means algorithm.
will change sharply when Then, a result
any one ofthe
matrix that contained C1, Cmax
2 and C3 changes. Δ𝐶 was set as the maximum difference in coefficients:
distance for these 12 kinds of rock mineral was assigned to T’:
Δ𝐶 = 𝑚𝑎𝑥{ 𝛥 𝐶𝑖 }, (9)
where, Δ𝐶𝑖 was the difference between the pixel and the surrounding pixels, and T1 was used as a
T0 = {D1 ,point
threshold value to determine whether the
. . . ,aD
D2 , was i , . . . boundary
texture
, Dm }, point; and T’ was a matrix, (10)
which was used to store the maximum distances within each class. T1 indicated the max distance
where, Di is thebetween the 12 rockin
max distance minerals, which was calculated
each category from any by K-means.
point’s First, a matrix
C value to which contains
the mean the
colors’ RGB values;
RGB values of every type of rock mineral image was calculated by K-means algorithm. Then, a result
and m is the number of mineral classes. Finally, the maximum value in T’ was assigned
matrix that contained the max distance for these 12 kinds of rock mineral was assigned to T’: to T 1 . When T 1
is smaller than ∆C, the pixel point will be𝑇 ′ marked = {𝐷1 ,D2 , . . . as
, 𝐷𝑖 , a
. . . texture
, 𝐷𝑚 }, boundary point; and (10) in the contrary
case, it is not going
where,toDibe marked.
is the Theintexture
max distance and from
each category cleavage any point’s canCbe outlined
value in the
to the mean image
colors’ RGB by combining
the brightness andvalues;color change.
and m is the number The recognition
of mineral accuracy
classes. Finally, the maximum of rock
valuemineral images
in T’ was assigned to Tcan
1. be increased
When T1 is smaller than 𝛥𝐶, the pixel point will be marked as a texture boundary point; and in the
with the extraction of texture and the cleavage of rock minerals.
contrary case, it is not going to be marked. The texture and cleavage can be outlined in the image by
In a word,combining
the feature of rock
the brightness and mineral
color change.images
The recognition can be extracted
accuracy by calculating
of rock mineral images can be the changes in
brightness values increased
and with
colorthe values.
extraction of
Intexture
this and the cleavage
research, texture of rock features
minerals. of rock minerals were extracted
In a word, the feature of rock mineral images can be extracted by calculating the changes in
to strengthen the texture
brightness features
values and colorof the Inoriginal
values. rock
this research, mineral
texture featuresimages by marking
of rock minerals and outlining the
were extracted
texture boundary to strengthen
points,theastexture
shown features of the original
in Figure rock mineral
5. After images by Inception-v3
extraction, marking and outlining
model the and the color
texture boundary points, as shown in Figure 5. After extraction, Inception-v3 model and the color
model will be trained.
model will be trained.

(a) (b)

(c) (d)

Figure 5. Cont.
Minerals 2019, 9, 516 9 of 17

Minerals 2019, 9, x FOR PEER REVIEW 9 of 16

(e) (f)
Figure 5. Samples of texture extraction for rock mineral images. (a) Cinnabar; (b) texture extraction of
Figure 5. Samples of texture extraction for rock mineral images. (a) Cinnabar; (b) texture extraction of
cinnabar; (c) calcite; (d) texture extraction of calcite; (e) malachite; (f) texture extraction of malachite.
cinnabar; (c) calcite; (d) texture extraction of calcite; (e) malachite; (f) texture extraction of malachite.
3.2. Color Model of Rock Mineral
3.2. Color Model of Rock Mineral
The colors of the 12 types of rock minerals considered here are shown in Table 1. A computer
generated the color through combining red, green, and blue to build RGB color space, in which red,
The colorsgreen,
of theand12bluetypes of rock
have values in [0, minerals
255]. As a result, considered here (226
there are 1,677,216 are ×shown
256 × 256 in Table 1. A computer
= 1,677,216)
colors in RGB color space. Any change of the value in
generated the color through combining red, green, and blue to build RGB color space, R, G or B will lead to color variation. It is in which red,
possible to implement recognition for rock mineral images based on the feature of color. The rock
green, and bluemineral
havelabel
values
can bein [0, 255].
determined by As a result,
the following there are 1,677,216 (226 × 256 × 256 = 1,677,216)
equation:
2 2 2
colors in RGB color space. Any change 𝑆ji = 𝜌ji 2 =of
(𝑅the
𝑖 - R ji value
) + (𝐺 𝑖 - in
G𝑗𝑖 )R, G𝑖 -B
+ (𝐵 or𝑗𝑖 )B , will lead to color (11) variation. It is
where, ρji are the Euclid Distance. Ri, Gi and Bi are the mean values of RGB value in rock mineral
possible to implement recognition for rock mineral images based on the feature of color. The rock
images. Rji, Gji and Bji are the mean values of the RGB value of the ith color in the jth rock mineral.
mineral label can be determined
According by the
to the color distance Sji,following
the rock mineral equation:
ranked top 6 are recognized, and shown in the
result. See Equations (11–12).
 2𝑚𝑖𝑛{𝑆𝑗𝑖 } 2  2
S ji = ρ ji 2 =𝑆cor𝑒
Ri ′𝑗−= 1R−ji∑𝑛 +𝑚𝑖𝑛{𝑆
Gi }−(𝑖 =G0,1, . . .+
ji , 𝑘), Bi − B ji , (12) (11)
𝑗=1 𝑗𝑖
𝑆cor𝑒 ′𝑗
𝑆cor𝑒𝑗 = 𝑛 . (13)
where, ρji are the Euclid Distance. Ri , Gi and B i are
∑ 𝑗=1 the ′mean
𝑆cor𝑒 𝑗 values of RGB value in rock mineral
Here a color model for rock mineral will be established based on the colors of rock mineral. The
images. Rji , Gji and Bji are the mean values of the RGB value of the ith color in the jth rock mineral.
color model will be used to assist inception-v3 model in identifying rock minerals. During
According to the identification of the typeSji
color distance of, athe rockthe
mineral, mineral ranked
inception-v3 model top 6 are
identifies therecognized, and
mineral first, then a shown in the
six-size set
result. See Equations of identification
(11) and (12).results is passed to the color model. The final result is given by the color
model after matching the color of the mineral with the results from inception-v3 model.
n o
4. Experiment and Results min S ji
Score0j = 1 − P n o (i = 0, 1, . . . , k), (12)
4.1. Image Processing n
j=1 min S ji
The dataset [42] consists of 4178 images of 12 kinds of elaborate rock minerals. The information
about rock mineral images in each category is shown in0 Table 2. Twenty images were selected
Score j
randomly in each category as a test dataset, and the remaining images were used to retrain the
Inception-v3 model.
Score j = Pn 0. (13)
j=1 Score j
Table 2. The types and numbers of rock mineral images.
Here a color model for rock mineral will be established based on the colors of rock mineral.
Rock Minerals Number Rock Minerals Number
The color model will be used to assist inception-v3
Cinnabar 327 model
Hematite in identifying
324 rock minerals. During
Aquamarine 380 Calcite 451
identification of the type of Molybdenite
a mineral, the inception-v3
316 model identifies
Stibnite the423mineral first, then a six-size
Malachite 335 Azurite 323
set of identification results is passed to the color
Cassiterite 378
model. Augite
The final result171 is given by the color model
after matching the color of the mineral with 335
Magnetite the results fromGyp Inception-v3 415model.

4. Experiment and Results

4.1. Image Processing

The dataset [42] consists of 4178 images of 12 kinds of elaborate rock minerals. The information
about rock mineral images in each category is shown in Table 2. Twenty images were selected randomly
in each category as a test dataset, and the remaining images were used to retrain the Inception-v3 model.
In establishing the model, numerous factors including the number and clarity of rock mineral
images, background noise and differences between features of minerals all influence its accuracy.
Generally, accuracy increases as the number of rock mineral images increases. However, the imbalance
among different types in quantity may lead to low accuracy. Therefore, we collected the images and
ensured that the total number of each category was at least 150. The proportion of rock minerals in an
image was then adjusted to at least 80%. Finally, the noise of the image, such as complex backgrounds
and labels, was removed to give the images the same size. In the process of training and the color
Minerals 2019, 9, 516 10 of 17

Minerals 2019,
model, the 9, x FOR PEER
texture REVIEW
features of all images were extracted and strengthened. All images were 10 of 16
also
translated into the RGB matrix. Then, the brightness values and a matrix containing the ∆C values of
In establishing the model, numerous factors including the number and clarity of rock mineral
the points of each image were calculated by Equation (3) and Equations (7)–(9), respectively. If they
images, background noise and differences between features of minerals all influence its accuracy.
were determined to be boundary points, the points were marked and outlined. Some pre-processed
Generally, accuracy increases as the number of rock mineral images increases. However, the
results are shown in Figure 5. Then, the Inception-v3 model was trained.
imbalance among different types in quantity may lead to low accuracy. Therefore, we collected the
images and ensured thatTable the total number
2. The of each
types and category
numbers of rockwas at least
mineral 150. The proportion of rock
images.
minerals in an image was then adjusted to at least 80%. Finally, the noise of the image, such as
complex backgrounds and labels, was removed Rock
Rock Minerals Number to give the images theNumbersame size. In the process of
Minerals
training and the color model, the texture features of all images were extracted and strengthened. All
images were also translatedCinnabarinto the RGB 327 matrix. Hematite
Then, the brightness324 values and a matrix
Aquamarine 380 Calcite 451
containing the △C values of the points of each image were calculated by Equation (3) and Equations
Molybdenite 316 Stibnite 423
(7)–(9), respectively. IfMalachite
they were determined 335 to be boundary
Azurite points, the points were marked and
323
outlined. Some pre-processed
Cassiteriteresults are 378
shown in Figure
Augite5. Then, the 171inception-v3 model was
trained. Magnetite 335 Gyp 415

4.2. Model
4.2. Model Establishment
Establishment

4.2.1. Model Training

images were
The processed images were used
used as
as the
the raw
raw data.
data. The
The input
input image
image size
size was
was set
setto
to299
299×× 299.
299. All
the input images were pre-processed for training. There were three input channels for length, width, width,
and color. The training steps were set to 20,000, and the learning rate was set to to 0.01.
0.01. The prediction
results were
results werecompared
comparedwithwiththe
the true
true label
label in in each
each step,
step, so that
so that the training
the training and validation
and validation accuracy
accuracy were
were calculated
both both calculated to update
to update the weights
the weights in the model.
in the model. The true The trueis label
label is the
the class classfrom
name namethefrom the
dataset.
dataset.
The The measures
primary primary measures used to training
used to evaluate evaluateeffectiveness
training effectiveness are the
are the training trainingvalidation
accuracy, accuracy,
validationand
accuracy accuracy and train cross-entropy,
train cross-entropy, as shown inasFigure
shown 6.in Figure 6.

Figure 6. Training process with Inception-v3.

The
The lighting
lighting conditions have an
conditions have an influence
influence on
on the
the RGB
RGB value
value of
of the
the images. To get
images. To get aa better
better color
color
model,
model, the
theself-adaption
self-adaptionK-means
K-meansalgorithm,
algorithm,which
whichis optimized by the
is optimized by silhouette coefficient,
the silhouette was used
coefficient, was
to train
used tothe model
train based on
the model 10 image
based on 10slices forslices
image each category
for each in differentinlighting
category condition
different lighting(with a size
condition
of about
(with 300of× about
a size 300). There wereThere
300 × 300). aboutwere
900,000 points
about in each
900,000 category.
points in eachThese points
category. were
These then were
points used
to train
then the color
used model
to train the by K-means.
color modelWhen the model
by K-means. was trained,
When the silhouette
the model coefficient
was trained, and the
the silhouette
maximum distance, which is the
coefficient and the maximum distance, D i in Equation
which is the Di in Equation (10), of each category from value
(10), of each category from point to the mean point
to the mean value were calculated. The training process ended until the silhouette coefficient was
maximum. The training results of the color model is shown in Table 3. Additionally, Figure 7 shows
samples of rock mineral images which were trained in color model.
Minerals 2019, 9, 516 11 of 17

were calculated. The training process ended until the silhouette coefficient was maximum. The training
results of the color model is shown in Table 3. Additionally, Figure 7 shows samples of rock mineral
images which were trained in color model.

Table 3. Training results of color model.

Colors (R, G, B)
Rock Minerals
Color1 Color2 Color3
Cinnabar (142,79,69) (105,36,31) (184,107,102)
Hematite (168,96,80) (127,157,138) -
Molybdenite (121,125,126) (93,96,98) -
Calcite (193,199,193) (175,177,167) -
Cassiterite (44,47,53) - -
Magnetite (36,33,31) - -
Malachite (88,168,129) (67,144,104) (50,108,75)
Azurite (0,19,151) (44,77,201) -
Aquamarine (0,95,56) (35,159,112) -
Augite (82,80,82) (122,120,124) -
Stibnite (92,104,115) - -
Gyp (161,161,163) (230,220,192) -
Minerals 2019, 9, x FOR PEER REVIEW 11 of 16

(a) (b) (c)

Figure7.7.Training
Figure Trainingsamples
samplesofofcolor
colormodel.
model.(a)
(a)Cinnabar;
Cinnabar;(b)
(b)calcite;
calcite;(c)(c)aquamarine.
aquamarine.

To eliminate the influence of different lightingresults

Table 3. Training condition, wemodel.
of color trained numerous images in different
lighting conditions. Due to the difference in the reflection of rock minerals, some mineral’s RGB values
Colors (R, G, B)
changedRock minerals while some do not, therefore, the number of RGB value kinds of these minerals
significantly, Color1 Color2 Color3
in the colorCinnabar
model is different. (142,79,69) (105,36,31) (184,107,102)
WhenHematite
training the SVM model and RF model, the features
(168,96,80) of raw images were extracted
(127,157,138) - by the
Molybdenite (121,125,126) (93,96,98)
HOG method. Then all features were resized to 256 × 256. The parameters of SVM and RF were -
Calcite (193,199,193) (175,177,167) -
determined by the grid search algorithm.
Cassiterite
Finally, we found that the
(44,47,53) -
penalty coefficient and -gamma of
SVM are 0.9 and 0.09. The number of(36,33,31)
Magnetite trees of RF is 180. The parameters- of RF are the default.
-
Malachite (88,168,129) (67,144,104) (50,108,75)
4.2.2. Model Test and Results
Azurite (0,19,151) (44,77,201) -
Aquamarine (0,95,56) (35,159,112) -
First, two Inception-v3 models were
Augite retrained from both raw
(82,80,82) images and texture feature-extraction
(122,120,124)
images. Then, the retrained model from
Stibnite the texture feature extraction
(92,104,115) - images was combined - with the
color model to create a comprehensive model. The comparison was made among the three -models to
Gyp (161,161,163) (230,220,192)

find one with the highest accuracy. To ensure the accuracy was more convincing, 440 images were
used toTotest
eliminate the influence
these models, of different
and the testing images lighting condition,
were randomly we from
selected trained
thenumerous
image datasetimages
of thein
different
rock lighting
mineral. Theseconditions. Due are
testing images to the difference
going in the
to be used reflection
as the of rock
input data for minerals, some mineral’s
a comprehensive model
RGB values changed significantly, while some do not, therefore, the number
and Inception-v3 model trained with or without feature extraction images. Meanwhile, the of RGB value kinds
images areof
these minerals in the color model is different.
going to be automatically processed when testing the comprehensive model and Inception-v3 model
trainedWhen
with training the SVM model
feature extraction imagesand RF model,
by the program. theThe
features of raw
test result is images
shown in were extracted
Table 4. by the
HOG method. Then all features were resized to 256 × 256. The parameters of SVM and RF were
determined by the grid search algorithm. Finally, we found that the penalty coefficient and gamma
of SVM are 0.9 and 0.09. The number of trees of RF is 180. The parameters of RF are the default.

4.2.2. Model Test and Results

Minerals 2019, 9, 516 12 of 17
Minerals 2019, 9, x FOR PEER REVIEW 12 of 16

Table 4. Results
Table4. Results of five models.
of five models.

Validation TestTest Accuracy

Accuracy
Models
Models Validation Accuracy
Accuracy Top-1 Top-1 Top-3Top-3
Model
Model basedbased
on rawonimages
raw images 73.1% 73.1% 64.1% 64.1% 96.0%
96.0%
Model based
Model on texture
based extraction
on texture imagesimages 77.4%
extraction 77.4% 67.5% 98.3%
67.5% 98.3%
Comprehensive modelmodel
Comprehensive 77.4% 77.4% 74.2% 99.0%
74.2% 99.0%
SVM-HOG 33.6%
SVM-HOG 33.6%
RF-HOG 30.4%
RF-HOG 30.4%

ToToensure
ensurethethemodels
modelsininTable
Table 44 are
are statistically
statistically feasible,
feasible, thethe precision
precisionrate, rate,recall
recallrate
rateand
and
F1-measure
F1-measure value
valueareare
used
usedto validate the the
to validate models. In Table
models. 4, the4,retrained
In Table the retrainedInception-v3 model
inception-v3 based
model
onbased on rawwas
raw images images was
able to able
reach to reach aaccuracy
a validation validation accuracy
of 73.1%, andofthe
73.1%,
top-1and andthetop-3top-1 and top-3
accuracies were
accuracies
64.1% were respectively.
and 96.0%, 64.1% and 96.0%, respectively.
The test accuraciesThe test accuracies
(precision rate) of(precision
SVM-HOG rate)
andofRF-HOG
SVM-HOG are and
32.8%
RF-HOG
and are 32.8% andThe
31.2%, respectively. 31.2%, respectively.
accuracy is too low,The accuracy
which is toothat
indicates low, which
the HOGindicates thatnot
features are theeffective.
HOG
features are not effective. On the other side, the comprehensive model has a significant
On the other side, the comprehensive model has a significant performance, which proves it can extract performance,
which
the proves
texture it can The
features. extract
testthe texture features.
accuracies The test accuracies
of each category (recall rate) of each
of these category
models (recall rate) ofin
are shown
these8,
Figure models
and the areF1-measure
shown in Figure
value8,ofand
thethe F1-measure
models is shown value of the5.models
in Table As weiscan shown
see, in
theTable 5. Asof
accuracy
we can see, the accuracy of each category of the comprehensive model is much
each category of the comprehensive model is much higher than that of the rest models, which indicates higher than that of
the rest models, which indicates that the comprehensive model is suitable
that the comprehensive model is suitable for rock mineral identification. The F1-measure value of the for rock mineral
identification. The F1-measure value of the comprehensive model indicates that the model is
comprehensive model indicates that the model is statistically feasible.
statistically feasible.

Accuracy of each category of test

Comprehensive model
70.8% HOG-SVM
Gyp 31.2%
23.0% HOG-RF
66.7%
Calcite 5.0%
7.2%
87.5%
Cassiterite 26.6%
23.8%
55.8%
Stibnite 9.1%
10.7%
64.2%
Molybdenite 46.6%
43.4%
59.7%
Magnetite 21.7%
38.9%
66.7%
Augite 18.6%
21.7%
75.0%
Aquamarine 23.9%
32.8%
91.7%
Azurite 38.4%
32.3%
91.7%
Malachite 39.1%
45.8%
76.7%
Hematite 43.4%
57.3%
87.5%
Cinnabar 43.9%
44.1%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Figure 8.
Figure 8. Test
Test accuracies
accuracies of
of each
eachcategory.
category.

Table 5. Validation parameters of models.

Models Precision Rate Recall Rate F1-measure Value

Comprehensive model 74.2% 77.5% 0.758
SVM-HOG 33.6% 28.5% 0.308
RF-HOG 30.4% 31.2% 0.308
 Table 5. Validation parameters of models.

Models Precision rate Recall rate F1-measure value

Comprehensive model 74.2% 77.5% 0.758
Minerals 2019,SVM-HOG
9, 516 33.6% 28.5% 0.308 13 of 17
RF-HOG 30.4% 31.2% 0.308

ItIt revealed
revealedthat thatthethe
accuracy
accuracy of theof Inception-v3
the inception-v3 model, with the
model, with texture feature feature
the texture extraction images,
extraction
is
images, is higher than that with the raw images in Table 4. The inception-v3 model with the feature
higher than that with the raw images in Table 4. The Inception-v3 model with the texture texture
extraction images reached a higher accuracy, which means the texture
feature extraction images reached a higher accuracy, which means the texture is a significant feature is a significant feature for rock
mineral identification. The texture extraction makes it easier to distinguish
for rock mineral identification. The texture extraction makes it easier to distinguish different rock different rock minerals,
which
minerals,improves
which the accuracy
improves theofaccuracy
the retrained
of themodel.
retrained model.
The
The comprehensive
comprehensive model model achieved
achieved the the highest
highest accuracy
accuracy in in the
the three
threemodels.
models. Color
Color properties
properties
could
could increase identification accuracy greatly. From the test accuracy of the comprehensive model,
increase identification accuracy greatly. From the test accuracy of the comprehensive model,
we can see that the top-1 accuracy is 74.2%, and the top-3 accuracy is
we can see that the top-1 accuracy is 74.2%, and the top-3 accuracy is 99.0%. By combining the 99.0%. By combining the retrained
Inception-v3 model andmodel
retrained Inception-v3 the color
andmodel,
the colora comprehensive
model, a comprehensive identification could be made
identification couldfor different
be made for
rock minerals. The texture and color features were used the most during
different rock minerals. The texture and color features were used the most during identification. identification.
Following
Following are are some
some recognition
recognition samplessamples in in Figure
Figure 9. 9. As
As you
you can can see,
see, these
these models
models sometimes
sometimes
achieved different results. When the texture features are similar, and the
achieved different results. When the texture features are similar, and the colors are different. colors are different. For example,For
magnetite’s cleavage is similar to the azurite’s, but their colors are not
example, magnetite’s cleavage is similar to the azurite’s, but their colors are not similar, the color similar, the color model could
identify
model could them successfully.
identify themWhen the minerals’
successfully. Whencolor is similar, color
the minerals’ but has is different
similar, but cleavage such as
has different
calcite and gym, the Inception-v3 model trained with feature extraction
cleavage such as calcite and gym, the Inception-v3 model trained with feature extraction images images could identify them
successfully.
could identify Therefore, the comprehensive
them successfully. Therefore, model
the has the advantages
comprehensive modelof both
has the
the color model and
advantages the
of both
Inception-v3 model trained with feature extraction images. Occasionally,
the color model and the Inception-v3 model trained with feature extraction images. Occasionally, however, the comprehensive
model
however, is going to be wrong when
the comprehensive model theisminerals’
going totexture
be wrong feature
when is vague, or the color
the minerals’ texture is abnormal.
feature is
This
vague, or the color is abnormal. This condition will confuse the comprehensive model, as shown in
condition will confuse the comprehensive model, as shown in Figure 10. It is magnetite in
Figure
Figure 10, 10. but
It isbecause
magnetite of the blue lighting
in Figure 10, butand vague
because oftexture
the blue feature,
lighting these
andmodels all received
vague texture an
feature,
incorrect
these models result.all received an incorrect result.

(a) True (b) True (c) True

(d) True (e) False (f) True

Figure 9. Cont.
Minerals 2019, 9, 516 14 of 17
Minerals
Minerals2019,
2019,9,9,x xFOR
FORPEER
PEERREVIEW
REVIEW 1414ofof1616

(g)
(g)True
True (h)
(h)False
False (j)(j)False
False
Figure
Figure9.9.Samples
Figure Samplesof
Samples ofrecognition
of recognitionresults.
recognition results.(a–c)
(a–c)are
(a–c) from
are
arefrom
fromthe comprehensive
the
thecomprehensive
comprehensive model,
model,
model,(d–f) are
(d–f) from
are
(d–f) from
are the
fromthe
Inception-v3
Inception-v3
the Inception-v3model
model
modeltrained
trainedwith
trainedwithfeature
with featureextraction
feature extraction
extraction images,
images,(g–j)
images, (g–j)are
arefrom
are fromthe
from theInception-v3
the Inception-v3model
model
without
without
without feature extraction.
feature
feature extraction.
extraction.

(a)
(a) (b)
(b) (c)(c)

Figure
Figure10.
10.Identification
Identificationresults
resultsfrom
fromthese
thesethree
threemodels.
models.(a) True;
(a)True;
True;(b)
(b) true;
(b) (c)(c)
true;
true; (c) true.
true.
true.

5. Conclusions
5.5.Conclusions
Conclusions
In
InInthis
thisresearch,
this research,three
research, threemodels
three modelsbased
models basedon
based ononInception-v3
Inception-v3model
Inception-v3 modelwere
model were
were established.
established.The
established. Thedeep
The deeplearning
deep learning
learning
model,
model, coupled
model, coupled with
coupled with color
with color model reached
color model
model reached top-1
reached top-1and top-3
top-1 and accuracies
and top-3 of 74.2%
top-3 accuracies and
accuracies ofof 74.2% 99.0%,
74.2% andrespectively.
and 99.0%,
99.0%,
The retrainedThe
respectively.
respectively. model
The usingmodel
retrained
retrained raw images
model using
usingraw reached
raw images
images top-1 and top-1
reached
reached top-3
top-1andaccuracies
andtop-3 of 64.1%ofof
top-3accuracies
accuracies and
64.1% 96.0%,
64.1% and
and
respectively.
96.0%, The
96.0%,respectively. retrained
respectively.The model
Theretrained using
retrainedmodel texture
modelusing extraction
usingtexture images
textureextraction reached
extractionimages top-1
imagesreached and top-3
reachedtop-1 accuracies
top-1and andtop-3top-3
of 67.5% and
accuracies
accuracies 98.3%,
ofof67.5%
67.5% respectively.
and
and98.3%, The comparison
98.3%,respectively.
respectively. The
Theofcomparison
the three models
comparison ofofthe indicates
the three
threemodelsthat the
models comprehensive
indicates
indicates that
thatthethe
model is
comprehensivethe best of
modelall. The
is theSVM-HOG
best of model
all. The achieved
SVM-HOG a validation
model
comprehensive model is the best of all. The SVM-HOG model achieved a validation accuracy of accuracy
achieved a of 32.8%.
validation The RF-HOG
accuracy of
model
32.8%.achieved
32.8%. TheTheRF-HOG a validation
RF-HOG modelaccuracy
model achieved
achieved ofa a31.2%.
validation The accuracy
validation results
accuracyindicate that The
ofof31.2%.
31.2%. the
Thedeep
resultslearning
results indicate
indicatemodel thatcan
that the
the
extract
deep effective
deeplearning
learningmodelimage
modelcan features.
canextract The comparison
extracteffective
effectiveimage between
imagefeatures. the
features.The traditional
Thecomparison models
comparisonbetween and the
betweenthe deep learning
thetraditional
traditional
model
models
models shows
and
andthethat
thedeepthe deep
deep learning learning
learning model
model models
shows
shows are
thatmuch
that the more
thedeep effective
deeplearning than
learningmodels
models theare
traditional
are much
muchmore ones.
more effective
effective
than The
thanthe deep learning
thetraditional
traditionalones. algorithm
ones. provides a new idea for rock mineral identification. The clustering
algorithm
The
Thedeepcould also improve
deeplearning
learning the performance
algorithm
algorithm provides
providesa anew of
newtheideadeep
idea for learning
forrock model.
rockmineral
mineral Therefore, the
identification.
identification. The combination
The clustering
clustering
of a deep
algorithm learning
algorithm could algorithm
could also also improve and clustering
improve the algorithm
the performance
performance ofof the is an effective
the deep way
deep learningfor rock
learning model. mineral identification.
model. Therefore,
Therefore, the the
Meanwhile,
combination
combination the
ofofacolor
adeep
deep features
learning and
learning texture features
algorithm
algorithm and of rock algorithm
andclustering
clustering minerals
algorithmare isisansignificant
effectiveproperties
aneffective way
wayfor for
forrock
rock
identification.
mineral The identification
mineralidentification.
identification. Meanwhile, accuracy
Meanwhile, the ofcolor
thecolor Inception-v3
features
featuresand model could features
andtexture
texture befeatures
improved greatly
ofofrock by texture
rockminerals
minerals are
are
and color
significant extraction.
significantproperties
propertiesfor foridentification.
identification.The Theidentification
identificationaccuracyaccuracyofofInception-v3
Inception-v3model modelcould couldbebe
Furthermore,
improved
improved greatly
greatlyby the elaborate
bytexture
texture and mineral
andcolor images, with clear mineral features, are adopted. If the
colorextraction.
extraction.
features, such as cleavage
Furthermore,
Furthermore, the and luster,
theelaborate
elaborate can beimages,
mineral
mineral extracted,
images,with theclear
with comprehensive
clearmineral model can
mineralfeatures,
features, are
arebe further tested
adopted.
adopted. IfIfthe
the
for mineral
features,
features,such specimen
suchasascleavageidentification,
cleavageand even
andluster,
luster,can for field survey.
canbebeextracted,
extracted,the thecomprehensive
comprehensivemodel modelcan canbebefurther
further
tested
testedfor
formineral
mineralspecimen
specimenidentification,
identification,even
evenfor
forfield
fieldsurvey.
survey.
Author
AuthorContributions:
Contributions:Modules
Modulesdesign
designandanddevelopment,
development,C.L.;
C.L.;Overall
Overallframework
frameworkdesign,
design,M.L.;
M.L.;Data
Data
collection
collectionand
andapplication
applicationtest,
test,Y.Z.
Y.Z.and
andY.Z.;
Y.Z.;Algorithm
Algorithmand
andmodel
modelimprovement,
improvement,S.H.
S.H.
Minerals 2019, 9, 516 15 of 17

Author Contributions: Modules design and development, C.L.; Overall framework design, M.L.; Data collection
and application test, Y.Z. and Y.Z.; Algorithm and model improvement, S.H.
Funding: This research was funded by the National Natural Science Foundation for Excellent Young Scientists of
China (Grant no. 51622904), the Tianjin Science Foundation for Distinguished Young Scientists of China (Grant no.
17JCJQJC44000) and the National Natural Science Foundation for Innovative Research Groups of China (Grant
no. 51621092).
Conflicts of Interest: We declare that we have no financial and personal relationships with other people or
organizations that can inappropriately influence our work, there is no professional or other personal interest
of any nature or kind in any product, service or company that could be construed as influencing the position
presented in, or the review of, the manuscript entitled.

References
1. Yeshi, K.; Wangdi, T.; Qusar, N.; Nettles, J.; Craig, S.R.; Schrempf, M.; Wangchuk, P. Geopharmaceuticals of
Himalayan Sowa Rigpa medicine: Ethnopharmacological uses, mineral diversity, chemical identification
and current utilization in Bhutan. J. Ethnopharmacol. 2018, 223, 99–112. [CrossRef] [PubMed]
2. Rustom, L.E.; Poellmann, M.J.; Johnson, A.J.W. Mineralization in micropores of calcium phosphate scaffolds.
Acta Biomater. 2019, 83, 435–455. [CrossRef] [PubMed]
3. Laura, B.; Nina, E.; Mona, W.M.; Udo, Z.; Andò Sergio Merete, M.; Reidar, I.K. Quick, easy, and economic
mineralogical studies of flooded chalk for eor experiments using raman spectroscopy. Minerals 2018, 8, 221.
4. Ramil, A.; López, A.J.; Pozo-Antonio, J.S.; Rivas, T. A computer vision system for identification of
granite-forming minerals based on RGB data and artificial neural networks. Measurement 2017, 117, 90–95.
[CrossRef]
5. Sadeghi, B.; Madani, N.; Carranza, E.J.M. Combination of geostatistical simulation and fractal modeling for
mineral resource classification. J. Geochem. Explor. 2015, 149, 59–73. [CrossRef]
6. Li, R.; Albert, N.N.; Yun, M.; Meng, Y.S.; Du, H. Geological and Geochemical Characteristics of the Archean
Basement-Hosted Gold Deposit in Pinglidian, Jiaodong Peninsula, Eastern China: Constraints on Auriferous
Quartz-Vein Exploration. Minerals 2019, 9, 62. [CrossRef]
7. Rajendran, S.; Nasir, S. ASTER capability in mapping of mineral resources of arid region: A review on
mapping of mineral resources of the Sultanate of Oman. Ore Geol. Rev. 2019, 108, 33–53. [CrossRef]
8. Shi, B.; Liu, J. Nonlinear metric learning for kNN and SVMs through geometric transformations.
Neurocomputing 2018, 318, 18–29. [CrossRef]
9. Sachindra, D.; Ahmed, K.; Rashid, M.M.; Shahid, S.; Perera, B. Statistical downscaling of precipitation using
machine learning techniques. Atmos. Res. 2018, 212, 240–258. [CrossRef]
10. Vassilev, S.V.; Vassileva, C.G. A new approach for the combined chemical and mineral classification of the
inorganic matter in coal. 1. Chemical and mineral classification systems. Fuel 2009, 88, 235–245. [CrossRef]
11. Vassilev, S.V.; Vassileva, C.G. A new approach for the classification of coal fly ashes based on their origin,
composition, properties, and behaviour. Fuel 2007, 86, 1490–1512. [CrossRef]
12. Zaini, N.; Van Der Meer, F.; Van Der Werff, H. Determination of Carbonate Rock Chemistry Using
Laboratory-Based Hyperspectral Imagery. Remote. Sens. 2014, 6, 4149–4172. [CrossRef]
13. Adep, R.N.; Shetty, A.; Ramesh, H. EXhype: A tool for mineral classification using hyperspectral data.
ISPRS J. Photogramm. Remote. Sens. 2017, 124, 106–118. [CrossRef]
14. Othman, A.A.; Gloaguen, R. Integration of spectral spatial and morphometric data into lithological. J. Asian
Earth Sci. 2017, 146, 90–102. [CrossRef]
15. Li, Y.; Chen, C.; Fang, R.; Yi, L. Accuracy enhancement of high-rate GNSS positions using a complete
ensemble empirical mode decomposition-based multiscale multiway PCA. J. Asian Earth Sci. 2018, 169, 67–78.
[CrossRef]
16. Wang, W.; Chen, L. Flotation Bubble Delineation Based on Harris Corner Detection and Local Gray Value
Minima. Minerals 2015, 5, 142–163. [CrossRef]
17. Chen, J.; Chen, Y.; Wang, Q. Synthetic Informational Mineral Resource Prediction: Case Study in Chifeng
Region, Inner Mongolia, China. Earth Sci. Front. 2008, 15, 18–26. [CrossRef]
18. Shardt, Y.A.; Brooks, K. Automated System Identification in Mineral Processing Industries: A Case Study
using the Zinc Flotation Cell. IFAC-Papers OnLine 2018, 51, 132–137. [CrossRef]
Minerals 2019, 9, 516 16 of 17

19. Ślipek, B.; Młynarczuk, M. Application of pattern recognition methods to automatic identification
of microscopic images of rocks registered under different polarization and lighting conditions.
Geol. Geophys. Environ. 2013, 39, 373. [CrossRef]
20. Młynarczuk, M.; Górszczyk, A.; Ślipek, B. The application of pattern recognition in the automatic classification
of microscopic rock images. Comput. Geosci. 2013, 60, 126–133. [CrossRef]
21. Shu, L.; McIsaac, K.; Osinski, G.R.; Francis, R. Unsupervised feature learning for autonomous rock image
classification. Comput. Geosci. 2017, 106, 10–17. [CrossRef]
22. Coates, A.; Ng, A.Y.; Lee, H. An analysis of single-layer networks in unsupervised feature learning.
In Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale,
FL, USA, 11–13 April 2011; pp. 215–223.
23. Raina, R.; Battle, A.; Lee, H.; Packer, B.; Ng, A.Y. Self-taught learning: Transfer learning from unlabeled
data. In Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR, USA,
20–24 June 2007; pp. 759–766.
24. Aligholi, S.; Lashkaripour, G.R.; Khajavi, R.; Razmara, M. Automatic mineral identification using color
tracking. Pattern Recognit. 2016, 65, 164–174. [CrossRef]
25. Li, N.; Hao, H.; Gu, Q.; Wang, D.; Hu, X. A transfer learning method for automatic identification of sandstone
microscopic images. Comput. Geosci. 2017, 103, 111–121. [CrossRef]
26. Google Deep Learning. Available online: http://deeplearning.net/tag/google/ (accessed on 21 March 2019).
27. Zhang, Y.; Li, M.C.; Han, S. Automatic identification and classification in lithology based on deep learning in
rock images. Acta Petrol. Sin. 2018, 34, 333–342.
28. Kitzig, M.C.; Kepic, A.; Grant, A. Near Real-Time Classification of Iron Ore Lithology by Applying Fuzzy
Inference Systems to Petrophysical Downhole Data. Minerals 2018, 8, 276. [CrossRef]
29. Xiong, Y.; Zuo, R.; Carranza, E.J.M. Mapping mineral prospectivity through big data analytics and a deep
learning algorithm. Ore Geol. Rev. 2018, 102, 811–817. [CrossRef]
30. Zuo, R.; Xiong, Y.; Wang, J.; Carranza, E.J.M. Deep learning and its application in geochemical mapping.
Earth-Sci. Rev. 2019, 192, 1–14. [CrossRef]
31. Iglesias, J.C.; Álvarez; Santos, R.B.M.; Paciornik, S. Deep learning discrimination of quartz and resin in
optical microscopy images of minerals. Miner. Eng. 2019, 138, 79–85. [CrossRef]
32. Hinton, G.E.; Salakhutdinov, R.R. Reducing the Dimensionality of Data with Neural Networks. Science 2006,
313, 504–507. [CrossRef]
33. Hinton, G.; Mohamed, A.-R.; Jaitly, N.; Vanhoucke, V.; Kingsbury, B.; Deng, L.; Yu, D.; Dahl, G.; Senior, A.;
Nguyen, P.; et al. Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of
Four Research Groups. IEEE Signal Process. Mag. 2012, 29, 82–97. [CrossRef]
34. Gong, M.; Yang, H.; Zhang, P. Feature learning and change feature classification based on deep learning for
ternary change detection in SAR images. ISPRS J. Photogramm. Remote Sens. 2017, 129, 212–225. [CrossRef]
35. Pham, C.C.; Jeon, J.W. Robust object proposals re-ranking for object detection in autonomous driving using
convolutional neural networks. Signal Process. Image Commun. 2017, 53, 110–122. [CrossRef]
36. Esteva, A.; Kuprel, B.; Novoa, R.A.; Ko, J.; Swetter, S.M.; Blau, H.M.; Thrun, S. Dermatologist-level
classification of skin cancer with deep neural networks. Nature 2017, 542, 115–118. [CrossRef]
37. Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the Inception Architecture for Computer
Vision. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Las Vegas, NV, USA, 26 June–1 July 2016; pp. 2818–2826.
38. Qureshi, A.S.; Khan, A.; Zameer, A.; Usman, A. Wind power prediction using deep neural network based
meta regression and transfer learning. Appl. Soft Comput. 2017, 58, 742–755. [CrossRef]
39. Xu, G.; Zhu, X.; Fu, D.; Dong, J.; Xiao, X. Automatic land cover classification of geo-tagged field photos by
deep learning. Environ. Model. Softw. 2017, 91, 127–134. [CrossRef]
40. Xu, B.; Shen, S.; Shen, F.; Zhao, J. Locally linear SVMs based on boundary anchor points encoding. Neural Netw.
2019, 117, 274–284. [CrossRef]
41. Pellant, C. Rocks and Minerals—Smithsonian Handbook; DK Publisher: New York, NY, USA, 2002.
Minerals 2019, 9, 516 17 of 17

42. The Geological Museum of China. Available online: http://www.gmc.org.cn (accessed on 28 March 2019).
43. International Telecommunication Union. Available online: https://www.itu.int/rec/R-REC-BT.601-7-201103-
I/en (accessed on 12 March 2019).

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).