Professional Documents
Culture Documents
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
ABSTRACT Multi-spectral transmission image provides a possibility for the detection of early breast cancer.
However, in the process of acquiring multi-spectral transmission images, the recognition of heterogeneities has
many difficulties due to the image blur caused by the scattering effect of light source in biological tissues and
weak transmission signals. This paper proposes a combination method of modulation-demodulation-frame
accumulation technique and pattern recognition to achieve heterogeneous classification. Firstly, the acquisition
experiment of the phantom multi-spectral images is designed. Then, the signal-to-noise ratio (SNR) of the image
is improved by the modulation-demodulation and frame accumulation technique, and the 14-dimensional feature
information (firmness, angular second-order distance, contrast, gray-scale correlation, entropy, inverse gap,
smoothness, dissimilarity, consistency, center of gravity, area, perimeter, long diameter of irregular image, short
diameter of irregular image) of the heterogeneous region are extracted from the image with high SNR. Finally, the
heterogeneous classification accuracy of different models is compared. The results show that: Compared with the
classification accuracy of the traditional multi-spectral image classification models, Random Forest (RF) and
Extreme Learning Machine (ELM) models have better classification effect when subdividing the four types of
heterogeneity based on the data set of this paper. Among them, the RF and ELM models established by the dataset
of four-wavelength combination have the best classification effect, and the classification accuracy rate reaches
100%, secondly, it is the three-wavelength combined model. The single-wavelength model has the worst
classification effect. And the operating efficiency of ELM is significantly higher than RF. In conclusion, the image
quality is improved by modulation-demodulation and frame accumulation technique. And compared with the
classification accuracy of the traditional multi-spectral image classification models, the RF and ELM models
established in this paper have better classification effect, which may promote the application of multi-spectral
transmission imaging in early screening of breast tumors.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
standard of detection. The cost of MRI testing is too high to be the detection of early breast cancer.
promoted. In addition, the spatial distribution of early tumors In this paper, we combine the method of enhancing the LLL
in these images is ambiguous, making it difficult to identify image signal with the model of pattern recognition to achieve
heterogeneous regions. Considering the mammary gland is a effective classification of different heterogeneous tissues in
transparent body without bone, and the tumor tissue contains a multi-spectral transmission images. Different from the
large number of new blood vessels and hemoglobin, there will traditional multi-spectral image classification method
appear a large shadow in the transmission imaging (in the tissue, mentioned above, which adaptively extracts image feature
uneven medium called heterogeneity [16]). Therefore, optical information, we extract the feature information of the
transmission imaging provides a feasible and simple method heterogeneities on the different wavelength images from three
for early detection of breast cancer. aspects of spatial feature, texture feature and shape feature as
In optical transmission imaging, multi-spectral non- the classification basis of the heterogeneities. And considering
destructive optical testing has become a hot topic due to its the nonlinearity among experimental data, we use the
advantages of real-time, non-invasive, safe, specific and traditional pattern recognition method to classify the
sensitive and has been widely used in many fields [17-21], but heterogeneous tissues. The traditional pattern recognition is
there are few studies on the application of multi-spectral divided into supervised classification and unsupervised
transmission images in the medical field. This is mainly due to classification. Different heterogeneities are used to simulate
the strong scattering effect of the incident light in the tumor masses, and the categories of heterogeneities are known.
transmission process of biological tissue, which makes the Therefore, we mainly study the methods of supervised
signal in the multi-spectral transmission image weak and classification pattern recognition. The methods commonly
unable to obtain the rich characteristic information of used in supervised pattern recognition include RF with multi-
heterogeneities. In recent years, the technique of modulation- layer neural networks and ELM with single-layer neural
demodulation (loading shaped signals) and frame network. RF, first proposed by Breiman, is widely used in data
accumulation has become one of the most effective methods to processing, text classification and other fields because of its
enhance low-light-level (LLL) diffuse reflection image signals. advantages such as simple implementation, strong anti-
Among them, Li et al uses the combination method of frame overfitting ability, parallel processing and good ability to deal
accumulation and shaping signal technology to greatly enhance with nonlinear modeling problems [40]. In order to solve the
the SNR of the LLL image and improve the resolution of the problem that multi-neural networks consume a lot of time in
image [22-24]. Therefore, this paper, for the first time, attempts the process of classification optimization algorithm, Huang
to apply the combination method of modulation-demodulation proposed a single hidden layer feed-forward neural network
and frame accumulation to the transmission images to improve ELM algorithm, which has the characteristics of simple model
the image quality. design, fast operation speed and high generalization
In addition, in the multi-spectral transmission imaging performance, and has good performance in multi-label learning
process, the optical properties of different tissues at different [41]. Finally, the SNR and gray resolution of the images are
wavelengths are different, which is beneficial to the improved through the technique of modulation-demodulation
classification of heterogeneities. Although the spatial structure and frame accumulation. And the 14-dimensional feature
of the early tumor tissue is similar to that of the normal tissue, information of the heterogeneous region are extracted from the
the characteristic of the above multi-spectral image provides a image with high SNR for pattern recognition. The results show
good idea for identifying early tumor tissue in the image. And that compared with the accuracy of traditional multi-spectral
as a powerful technique to analyze and extract unique image classification models (SVM, LDA, LRR, SR and MLR),
information from multi-spectral image, feature information- the RF and ELM we established both have a better
based classification has been an active research topic in recent classification effect when subdividing 4 heterogeneities under
years. Many classification algorithms have been developed to the data set of this paper, which promotes the application of
perform multi-spectral image classification. For example, transmission multi-spectral imaging in early screening of
support vector machines (SVM) [25,26], linear discriminant breast cancer.
analysis (LDA) [27,28], low-rank representation (LRR) [29-
31], sparse representation (SR) [32-34] and multinomial II. RELATED TECHNOLOGY
logistic regression (MLR) [35] are well-known image
classification methods. Among them, Gao et al used the
A. MODULATION-DEMODULATION AND FRAME
multiple kernel learning method to fuse different features of
samples to improve the classification accuracy of SVM in ACCUMULATION TECHNOLOGY
multi-spectral images [36]. Wang et al proposed a novel LDA In the process of multi-spectral transmission imaging, the
to obtain all representative subspaces by adaptively, which distribution of light intensity may be affected due to the
improved the classification accuracy of different objects in influence of the external environment and the scattering effect
multi-spectral images [37]. Wang et al proposed a local and of heterogeneities themselves, resulting in weak signal and low
structural-regularized LRR for multi-spectral image definition of transmission images. Therefore, this paper needs
classification, which overcomes the limitations of multi- to perform certain pre-processing experiments on the obtained
dimensional image feature extraction [38]. Gao et al improved transmission images. Among them, the weak signal in the
SR to improve the precision of classification result by image includes two meanings: one is that the absolute value of
incorporating the neighboring information of the test pixel [39]. the signal light intensity is low, and the other is that the SNR
So, multi-spectral transmission image provides a possibility for and resolution of the signal are low. The modulation-
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
Heterogeneity
Phantom Different shapes
Power
Source
Different components
Supply
Camera
LED
CCS
Sealed MFSG
Dark Box
(a) (b)
demodulation and frame accumulation technology has become neural network algorithms need to set more network
an effective mean to enhance the LLL image signal. parameters. When solving the optimal solution, local optimal
Gray level is the core of image accuracy and heterogeneity solutions may appear and the global optimal solution of the
detection sensitivity. The more grayscale of image, the richer network cannot be obtained. In the ELM solution, only the
the image information, the better the classification of number of hidden layer nodes needs to be set, and the global
heterogeneous tissue in tissue. Low-resolution images become optimal solution can be solved by randomly initializing the
obstacles to tissue classification and spatial information weights and offsets. ELM solves the feedforward neural
extraction, but the modulation-demodulation and frame network with single hidden layer, which can be divided into
accumulation technique can improve the resolution and two stages: random feature mapping and linear parameter
increase the grayscale of images to some extent [22]. The solving.
modulation and demodulation techniques are the process of
moving the spectrum of the baseband signal to the channel III. EXPERIMENT
passband to achieve the purpose of enhancing the signal, and According to the characteristics of breast tissue, this paper
then restoring the band signal brought by the channel back to designed the collection experiment of phantom. The 14-
the baseband signal [23,42]. The frame accumulation technique dimensional feature information of the different wavelength
adds the gray values of the corresponding pixel points of the images is respectively extracted on the multi-spectral
multi-frame images to obtain the average image in the time transmission images after the modulation-demodulation and
series, filters out the noise in the image, and multiples the SNR frame accumulation processing. And the different
of the image [22]. Therefore, this paper uses the technique of heterogeneities are respectively classified by pattern
modulation-demodulation and frame accumulation to improve recognition methods (RF and ELM) and traditional multi-
the SNR and resolution of the transmission images. spectral image classification models.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
FIGURE 2. Phantom multi-spectral transmission images. (a) blue wavelength transmission image; (b) green wavelength transmission image; (c) near-infrared
wavelength transmission image; (d) red wavelength transmission image.
(a) (b)
FIGURE 3. Fourier transform diagram. (a) frequency coordinate position map; (b) frequency map.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
and the camera of the camera to get a sequence of images relevant information of the image is extracted from the spatial
of the phantom. features, the texture features and shape features.
(3) The four kinds of light sources loaded with the sinusoidal Spatial feature extraction: The spatial relationship
shaped signal are respectively irradiated with the phantom. indicates that there is a certain spatial positional relationship
The phantom included four heterogeneities (two potato and directional relationship among multiple objects in the
pieces and two carrot pieces) of different sizes and image, such as the firmness of the image's adjacency
thickness, and the experiment is carried out in groups. relationship. Texture feature extraction: The gray-level co-
Each wavelength includes 6 groups, each group occurrence matrix (GLCM) is used to extract the texture
configured with different concentrations of solutions, a features of the image, which uses the probability in statistics to
total of k 24 groups. reflect the overall information of the image grayscale related
(4) Excluding the images with large errors in each group, a directions and intervals. There are 8 commonly used texture
total of n 26880 original multi-spectral images are feature parameters (angular second-order distance, contrast,
obtained, and Fig.2 are one frame of the original phantom gray-scale correlation, entropy, inverse gap, smoothness,
images x il, j ( l 1, 2, ,1120; i 1, 2, 3, 4; j 1, 2, ..., 6) of dissimilarity and consistency). Shape feature extraction:
the four wavelengths, respectively. Hough transform is used to detect the heterogeneous regional
feature information of the image, which mainly includes five
feature parameters (center of gravity, area, perimeter, long
C. IMAGE PREPROCESSING
diameter of irregular image, short diameter of irregular image).
The multi-spectral transmission images obtained in this Among them, in order to precisely determine the region of the
experiment are processed by the technique of modulation- heterogeneities, the heterogeneous feature region is obtained
demodulation and frame accumulation. The specific image by using the Ostu threshold segmentation method for different
processing steps are as follows: wavelength images. In the process of obtaining mask images
(1) Perform a fast Fourier Transform (FFT) on images by Ostu threshold segmentation method, the coincidence
x il, j ( l 1, 2, ,1120; i 1, 2, 3, 4; j 1, 2, ..., 6) , and extract
degrees of different heterogeneous mask images and true
the coordinate values corresponding to the frequency knowledge map (the actual heterogeneous images were
components of the four wavelengths, as shown in Fig.3. obtained before the experiment) are calculated as the
(2) According to the frequency coordinate value determined discrimination basis for the selection of threshold values, so as
above, the pixels of all the images of the four wavelengths to obtain the multi-wavelength mask images with the highest
are demodulated, and the demodulated images of all coincidence degree. Because the mask image needs to cover
wavelengths are obtained. the target region of the sample as much as possible, it needs to
(3) Read the demodulated images x il, j ( l 1, 2, ,11 20 ) have a high degree of coincidence with the actual target region
into MATLAB program respectively, find the sum of gray of the sample to improve the accuracy of heterogeneity
values of each frame, and draw 1120 gray value of images detection. According to comprehensive statistics, the
xil, j (l 1, 2, 3,...,1120) , as shown in Fig.4. coincidence degree of all multi-spectral mask images reached
(4) Determine the number of image frames in a single sine more than 96%, which was enough to cover the characteristic
wave period according to the curve shown in Fig.4. It can information of heterogeneity in the images. And the average
be seen from the figure that each sinusoidal shaped signal threshold value and the mask image of different heterogeneities
includes 11 frames images. are obtained, as shown in Tab.I and Fig.5. The thresholds in
(5) Perform the frame-accumulate average for every 11 Tab.I respectively represent the average thresholds of each
frames of images each group in the experiment, that is: group of the best heterogeneous mask images in different
l 11 wavelength light sources, so as to ensure more accurate
x l
i, j extraction of the feature information of the heterogeneous
xk
i, j l
(1) region. And as can be seen from Tab.I, the floating range of
f image segmentation threshold in the same wavelength is small,
i 1, 2,3, 4; j 1, 2,..., 6; l 1, 2, ,1109; k 1, 2,..., 606; f 11. while the variation range between different wavelengths is
(6) According to formula (1), a total of n2 2424 frame large. Finally, a total of 9696×14 dimensional information is
accumulative multi-spectral images are obtained, in obtained for all wavelength images, and each wavelength
which each wavelength includes 606 images. includes 2424×14 samples.
2) DATA SET PRODUCTION
D. HETEROGENEOUS CLASSIFICATION
All wavelength heterogeneous feature information (9696×14)
is made into a data set, which included a total of 14 data sets.
1) FEATURE INFORMATION EXTRACTION ①Single wavelength test data set, a total of 4 groups. ②Two-
In order to accurately extract the information in each wavelength combined test data set, a total of 6 groups. ③
heterogeneous region, each preprocessed image is reasonably Three-wavelength combined test data set, a total of 3 groups.
cropped into four regions of uniform size without affecting the ④Four-wavelength combined test data set, a total of 1 group.
accuracy of the algorithm and image quality (each region To promote the possibility of multi-spectral transmission
includes one heterogeneity). Different from the traditional images in early breast cancer detection, the wavelength
multi-spectral image classification methods mentioned above, combination process is sequentially arranged in the order of
which adaptively extracts image feature information, the blue light, green light, near-infrared light and red light. Data
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
TABLE I
AVERAGE MASK THRESHOLD OF ALL IMAGES AT DIFFERENT WAVELENGTHS
Image B11 B12 B13 B14 B21 B22 B23 B24 B31 B32 B33 B34
Threshold 0.18 0.16 0.13 0.12 0.14 0.11 0.11 0.10 0.14 0.16 0.14 0.14
Image B41 B42 B43 B44 B51 B52 B53 B54 B61 B62 B63 B64
Threshold 0.15 0.16 0.14 0.12 0.10 0.08 0.09 0.07 0.16 0.14 0.14 0.15
Image G11 G12 G13 G14 G21 G22 G23 G24 G31 G32 G33 G34
Threshold 0.12 0.2 0.07 0.05 0.24 0.1 0.06 0.08 0.18 0.1 0.08 0.07
Image G41 G42 G43 G44 G51 G52 G53 G54 G61 G62 G63 G64
Threshold 0.12 0.38 0.05 0.05 0.32 0.1 0.21 0.2 0.1 0.4 0.07 0.08
Image I11 I12 I13 I14 I21 I22 I23 I24 I31 I32 I33 I34
Threshold 0.42 0.37 0.4 0.47 0.30 0.35 0.31 0.37 0.38 0.43 0.44 0.49
Image I41 I42 I43 I44 I51 I52 I53 I54 I61 I62 I63 I64
Threshold 0.33 0.36 0.35 0.43 0.40 039 0.41 0.42 0.22 0.24 0.25 0.28
Image R11 R12 R13 R14 R21 R22 R23 R24 R31 R32 R33 R34
Threshold 0.36 0.41 0.36 0.45 0.38 0.4 0.36 0.42 0.37 0.39 0.36 0.39
Image R41 R42 R43 R44 R51 R52 R53 R54 R61 R62 R63 R64
Threshold 0.35 0.42 0.35 0.39 0.35 0.38 0.35 0.36 0.2 0.22 0.21 0.19
Note: B11 represents the blue wavelength image of the first heterogeneity in the first group obtained, and so on. G represents the wavelength of green light; I
represents the wavelength of near-infrared light; R represents the wavelength of red light.
sets of different wavelengths are randomly divided into training After the multi-spectral transmission image is processed by the
sets and test sets. Since the data set in this paper is small, the modulation-demodulation and frame accumulation technique,
ratio is set to 4:2 according to the traditional division ratio of the SNR is significantly enhanced and the gray level is
the machine learning field. In the solution, the heterogeneities significantly increased, as shown in Fig.6. And it can be seen
are divided into 4 types: two pieces of potato and two pieces of from the figure that the gray level of the processed images
carrot of different sizes and thicknesses. increases significantly, which makes the heterogeneous regions
3) MODEL DEBUGGING in all wavelength images more prominent. And compared with
The sample data is normalized during the model debugging the traditional multi-spectral image classification model, the
process. In order to find the optimal model, different RF and ELM models established in this paper achieve better
combinations of heterogeneous feature information (single classification accuracy.
wavelength, two wavelengths, three wavelengths and four (1) Modulation-demodulation and frame accumulation
wavelengths) of four wavelengths are input into the network technique significantly improves the quality of the phantom
for training, and each set of data sets is trained 100 times. image. The results are shown in Tab.II. It is found that by
According to the classification results of RF and ELM models, calculation that the peak signal-to-noise ratio (PSNR) of the
the number of optimal decision trees and neurons in the models image before and after frame accumulation is positive, which
are determined respectively, and the best detection model for indicates that the gray level of the image significantly increases
different heterogeneous classifications is obtained. Finally, the after preprocessing. The image SNR after frame accumulation
classification accuracy, correlation coefficient R2, root mean has been improved to a certain extent, which will be conducive
square error (RMSE) and running time of the RF and ELM to the extraction of regional feature information of
models are obtained. The results are shown in Tab.IV. heterogeneity in the image.
(2) Heterogeneous classification results: Based on the
modulation-demodulation and frame accumulation technique
IV. RESULTS AND ANALYSIS
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
FIGURE 6. Gray level comparison chart before and after image preprocessing. (a) blue light image; (b) green light image; (c) near infrared light image;
(d) red light image. a2-d2 are the original phantom images of 4 different wavelengths, and a3-d3 are the pre-processed images. And a1-d1 and a4-d4 are the
gray histogram distribution of a2-d2 and a3-d3 respectively.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
(a) (b)
FIGURE 7. RF and ELM model parameter results image. (a) Determination process of the number of decision trees in the RF model; (b) Process of
determining the number of neurons in the ELM model.
TABLE III
QUALITY FACTOR RESULTS FOR EACH WAVELENGTH COMBINATION VARIABLE
Image a b c d ab ac ad bc bd cd abc abd bcd abcd
Firmness 0.121 0.192 0.040 0.094 0.104 0.163 0.162 0.112 0.164 0.077 0.142 0.190 0.128 0.134
Angular 0.091 0.092 0.180 0.097 0.170 0.125 0.142 0.164 0.115 0.167 0.167 0.119 0.173 0.118
Contrast 0.102 0.126 0.209 0.112 0.181 0.157 0.126 0.170 0.129 0.161 0.138 0.141 0.141 0.122
Correlation 0.143 0.249 0.141 0.089 0.220 0.190 0.181 0.220 0.214 0.114 0.259 0.164 0.207 0.212
Entropy 0.104 0.077 0.113 0.084 0.185 0.167 0.146 0.102 0.102 0.145 0.133 0.138 0.114 0.153
Inverse 0.107 0.125 0.062 0.091 0.135 0.133 0.163 0.081 0.139 0.099 0.117 0.161 0.093 0.147
Smoothness 0.018 0.171 0.176 0.396 0.083 0.183 0.174 0.173 0.266 0.259 0.093 0.175 0.223 0.188
Dissimilarity 0.004 0.014 0.028 0.004 0.014 0.014 0.012 0.020 0.014 0.019 0.007 0.008 0.015 0.007
Consistency 0.001 0.025 0.029 0.015 0.019 0.016 0.018 0.030 0.029 0.019 0.020 0.016 0.022 0.009
Center 0.001 0.033 0.028 0.021 0.017 0.011 0.018 0.046 0.032 0.026 0.032 0.019 0.038 0.019
Area 0.001 0.035 0.033 0.013 0.015 0.009 0.012 0.037 0.025 0.018 0.022 0.019 0.028 0.016
Perimeter 0.008 0.002 0.004 0.002 0.002 0.002 0.002 0.006 0.007 0.003 0.005 0.004 0.009 0.005
Long 0.006 0.034 0.017 0.001 0.013 0.012 0.010 0.021 0.023 0.006 0.021 0.019 0.032 0.017
Short 0.003 0.038 0.010 0.002 0.016 0.015 0.016 0.026 0.030 0.005 0.029 0.018 0.019 0.019
Note: ①The green mark indicates the factor with the highest weight of the variables in the different wavelength combinations. ②The red mark indicates the highest
weight of the same factor in different wavelength combinations.
accuracy with the traditional multi-spectral image (3) If there are K trees in RF, the importance of feature x i
classification models, and the results are shown in Tab.Ⅴ. In in the whole RF can be calculated by equation (4):
addition, based on the Gini index in RF, the importance of each 1 K
characteristic variable in the model prediction process is also IMPi Gini IMPi Gini
K k 1
_k
obtained, as shown in Tab.III. The specific analysis process of (4)
the importance of characteristic variables based on Gini index Where: IM PinG in i represents the average change of Gini index
is as follows: of the i -th feature on all RF decision tree nodes; I G ( n )
(1) The importance of feature x i on node n , that is, the represents the exponential change of node n ; I G (nl ) and
Gini index change before and after the data on node n I G ( nr ) represent respectively the change of Gini index before
is divided into its left and right sub-nodes n l and n r
and after distributing the data on node n to its left and right
is shown in equation (2): sub-nodes nl and n r ; IM Pi G_ ink i represents the importance
IM PinGini I G ( n ) I G ( nl ) I G ( n r )
(2)
of the i -th characteristic variables in the k -th decision tree
(2) If the node set of feature x i as node segmentation (the set of nodes where the node partitioning attribute is N );
attribute in the k -th decision tree is N , the importance IM Pi G in i represents the importance of the i -th characteristic
of feature in this decision tree can be obtained from variable in the whole RF.
equation (3): As shown in Tab.III, the importance of all characteristic
k
IM Pi G_ ini n N IM PinG ini
(3) variables in the model establishment process is obtained based
on Gini index. It can be seen from table that in the model
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
TABLE IV
RF AND ELM MODEL OPERATION RESULTS
Model RF ELM
2 2
Test Training% Test% R RMSE Time/s DT Var Training% test% R RMSE Time/s Neurons Var
a 92.08 78.34 0.34 1.03 760.51 3 2.73 88.37 85.77 0.54 0.85 146.01 4 2.39
b 99.94 96.91 0.98 0.18 1437.44 3 1.05 92.33 91.09 0.92 0.29 147.27 45 2.04
c 99.01 89.60 0.87 0.41 936.12 3 2.21 93.81 92.57 0.88 0.39 145.02 27 2.37
d 100 93.69 0.95 0.25 1302.29 3 1.54 89.29 88.24 0.91 0.34 145.32 28 2.48
ab 100 100 1 0 1172.74 5 0 100 99.38 0.98 0.15 145.99 28 0.35
ac 100 100 1 0 1395.82 5 0 100 100 1 0 149.04 35 0
ad 100 100 1 0 1085.09 5 0 100 100 1 0 148.57 36 0
bc 100 99.01 0.99 0.099 1491.12 5 0.59 96.53 96.04 0.97 0.20 151.02 21 1.14
bd 100 100 1 0 1656.84 5 0 97.40 97.65 0.98 0.15 147.20 25 0.82
cd 100 97.65 0.98 0.15 1798.54 5 0.76 97.34 94.80 0.95 0.24 151.46 23 1.28
abc 100 100 1 0 1729.82 6 0 100 100 1 0 154.76 19 0
abd 100 100 1 0 1544.67 6 0 100 100 1 0 150.79 19 0
bcd 100 99.88 0.98 0.13 1994.85 6 0.53 98.02 97.65 0.98 0.15 154.47 27 0.68
abcd 100 100 1 0 1501.31 6 0 100 100 1 0 158.87 23 0
Note: ‘a’ represents the blue wavelength. ‘b’ represents the green wavelength. ‘c’ represents the near-infrared wavelength. And ‘d’ represents the red wavelength.
DT=Decision trees. DT represents the number of optimal decision trees in each model. Neurons represents the number of optimal neurons in each model. Var=
variance.
TABLE Ⅴ
AVERAGE ACCURACY OF DIFFERENT CLASSIFICATION MODELS
Wavelength\Model SVM LDA LRR SR MLR RF1 ELM1 RF ELM
Sigle 89.51 82.04 88.86 84.60 89.04 57.79 55.34 89.64 89.42
Two 90.89 82.74 83.88 83.25 97.14 62.65 60.05 99.44 97.98
Three 98.78 92.15 71.72 67.74 99.12 63.08 64.4 99.96 99.22
Four 99.90 93.28 84.28 81.70 93.93 73.84 76.08 100 100
Note: RF1 and ELM1 respectively represent the classification accuracy of heterogeneity in unpreprocessed multi-spectral transmission images. RF and ELM
respectively represent the classification accuracy of heterogeneity in preprocessed multi-spectral transmission images.
building process, the gray-scale correlation, smoothness, the establishment of the optimal ELM model, the number of
contrast and firmness of the variables extracted from the multi- neurons in ELM is relatively disorderly distributed. And
spectral transmission images account for a relatively high compared with single-network model, multi-network model
proportion. And the same variable factor distribution with a takes more time to build. Therefore, the following conclusions
higher specific gravity in different wavelength combinations is can be drawn: ①The overall classification of RF and ELM is
relatively dispersed, but mainly concentrated in single ideal. ②The classification effect of the combined wavelength
wavelength and two wavelength combinations. This indicates is better than the single-wavelength classification effect, and
that as the wavelength combination increases, the proportion of the four-wavelength combination classification effect is
each characteristic variable tends to be equalized during the optimal. ③With the increase of wavelength, the classification
model establishment process, enriching the information degree accuracy of the models (RF and ELM) is gradually improved.
of the heterogeneities. ④ In the classification effect of different wavelength
The average results of classification after multiple training combinations, RF is superior to ELM. ⑤ In the model
of the model are shown in Tab.IV. Under the experimental data, classification running time, ELM is shorter than RF, and the
the classification results of the RF and ELM models are difference of time amount is large. ⑥Under the premise of
compared by the accuracy of the classification, R2, RMSE and small difference in overall classification accuracy, ELM's
the running time. Among them, R2 and RMSE are calculated operation efficiency is significantly higher than RF.
by formulas (5) and (6). It can be seen from Tab.IV that as the m
phenomenon of the model is reasonably suppressed. Moreover, Where m represents the number of test samples; yi
as can be seen from Tab.IV, in the process of obtaining the best represents the predicted value of the test sample; yi represents
RF model, the number of classification decision trees gradually
the actual value of the test sample; y i represents the actual
increases with the increase of wavelength combination. But in
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
average value of the test sample. established in the data set of four-wavelength combination
The average classification accuracy of different models is have the best classification effect, and the classification
shown in Tab.Ⅴ. It can be seen from the table that compared accuracy is up to 100%, secondly, it is the three-wavelength
with the RF and ELM models established by multi-spectral combined model. And the single-wavelength model has the
transmission images before preprocessing, the classification worst classification effect. And, in terms of model operation
accuracy of all wavelength combination RF and ELM models efficiency, ELM is significantly higher than RF. In summary,
established by multi-spectral transmission images after the technique of modulation-demodulation and frame
preprocessing is significantly improved, which indicates the accumulation has improved the gray resolution of images. And
effectiveness of the proposed method to enhance image quality. compared with the classification accuracy of the traditional
And as can also be seen from the table that in the models of multi-spectral image classification models, the RF and ELM
single wavelength and four wavelength combination, the models established in this paper have better classification effect.
traditional SVM classification effect is the best, reaching 89.51 In addition, potatoes and carrots are selected as heterogeneities
and 99.90, respectively. In the models of the two and three to simulate breast cancer according to the characteristics of
wavelength combination, the traditional MLR classification is strong transmissibility and tomographic distribution of breast
the best, reaching 97.14 and 99.12, respectively. However, tissue, which may be limited by other unknown conditions, but
compared with the combination of different wavelengths, the provides a good idea for the detection of heterogeneity in breast
RF and ELM models established after the preprocessed of tissue. Through further research, the improved method can be
multi-spectral transmission images in this paper have better adapted to more complex situations and even clinical
classification effect, and the RF and ELM models have better applications. It is expected that this method can promote the
overall classification effect. clinical application of multi-spectral transmission imaging in
Analysis of experimental results: The image preprocessed by early screening of breast tumors in the future.
the modulation-demodulation and frame accumulation
technique, it is found that the SNR of the image is improved to ACKNOWLEDGMENT
a certain extent, which makes the heterogeneities in the image We thank the State Key Laboratory of Precision Measuring
more prominent and the extracted feature information more Technology and Instruments for the use of their equipment.
perfect. With the same error rate and iteration times, RF's
classification accuracy is slightly higher than ELM, while ELM FUNDING
is slightly superior in classification speed. Therefore, ELM
This research did not receive any specific grant from funding
operates slightly more efficiently than RF. And compared with
agencies in the public, commercial, or not-for-profit sectors.
the accuracy of traditional multi-spectral image classification
models, the RF and ELM we established have better
classification effect. CONFLICT OF INTEREST
The authors declare that they have no conflict of interest.
V. CONCLUSION
In this paper, combined with the characteristics of breast tissue, REFERENCES
multi-spectral transmission image acquisition experiment is [1] Joëlle A.C. Desreux, “Breast cancer screening in young women,”
European Journal of Obstetrics & Gynecology and Reproductive Biology,
designed. The image quality is improved by modulation- vol. 230, pp. 208-211, 2018.
demodulation and frame accumulation technique, and the [2] Sharon W.W. Chan et al., “Surgical options for Chinese patients with early
heterogeneous classification is studied by pattern recognition invasive breast cancer: Data from the Hong Kong Breast Cancer Registry,”
Asian Journal of Surgery, vol. 40, no. 6, pp. 444-452, 2017.
(RF, ELM) and traditional multi-spectral image classification [3] Zeng Jie Ye et al., “A biopsychosocial model of resilience for breast
models (SVM, LDA, LRR, SR and MLR). Firstly, in the cancer: A preliminary study in mainland China,” European Journal of
process of acquiring multi-spectral transmission images, Oncology Nursing, vol. 36, pp. 95-102, 2018.
[4] C. Sosa et al., “Compact energy dispersive X-ray microdiffractometer for
sinusoidal signals with frequency of 4HZ are used as carrier diagnosis of neoplastic tissues,” Radiation Physics and Chemistry, vol.
signals to enhance the information degree of the image. Then, 137, pp. 125-129, 2017.
in image preprocessing, the PSNR and SNR of the image are [5] Damien Barbes et al., “Material-specific imaging system using energy-
dispersive X-ray diffraction and spatially resolved CdZnTe detectors with
improved through the modulation-demodulation and frame potential application in breast imaging,” Nuclear Instruments and
accumulation technology. The result shows that the PSNR of Methods in Physics Research Section A: Accelerators, Spectrometers,
the image before and after frame accumulation is positive, Detectors and Associated Equipment, vol. 848, pp. 91-98, 2017.
which indicates that the gray level of the image significantly [6] Daisuke Shimao et al., “Imaging with ultra-small-angle X-ray scattering
using a Laue-case analyzer and its application to human breast tumors,”
increases after preprocessing. And the image SNR after frame Physica Medica, vol. 44, pp. 236-242, 2017.
accumulation has also been improved to a certain extent, which [7] Min Xian et al., “Automatic breast ultrasound image segmentation: A
will obtain more abundant feature information of the image survey,” Pattern Recognition, vol. 79, pp. 340-355, 2018.
[8] Huay-Ben Pan, “The Role of Breast Ultrasound in Early Cancer
heterogeneous region. Finally, in the aspect of heterogeneous Detection,” Journal of Medical Ultrasound, vol. 24, no. 4, pp. 138-141,
classification, the different wavelength combination data sets 2016.
of phantom feature information are trained as the training [9] Rongrong Guo et al., “Ultrasound Imaging Technologies for Breast
Cancer Detection and Management: A Review,” Ultrasound in Medicine
sample, and the heterogeneous classification accuracy of & Biology, vol. 44, no. 1, pp. 37-70, 2018.
different models is compared. The results show that compared [10] Shuang-Qing Chen et al., “Application of Abbreviated Protocol of
with the classification accuracy of the traditional multi-spectral Magnetic Resonance Imaging for Breast Cancer Screening in Dense
image classification models, RF and ELM models have better Breast Tissue,” Academic Radiology, vol. 24, no. 3, pp. 316-320, 2017.
[11] Albert Gubern-Mérida et al., “Automated detection of breast cancer in
classification effect. Among them, the RF and ELM models false-negative screening MRI studies from women at increased risk,”
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2019.2930296, IEEE Access
Fulong Liu: Classification of heterogeneity on multi-spectral transmission image based on modulation-demodulation-frame accumulation
and pattern recognition
European Journal of Radiology, vol. 85, no. 2, pp. 472-479, 2016. orthogonal linear discriminant analysis for classification of hyper-spectral
[12] Seyedamir Tavakoli Taba et al., “Toward Improving Breast Cancer data,” Eur. J. Remote Sens, vol. 50, no. 1, pp. 111–124, 2017.
Imaging: Radiological Assessment of Propagation-Based Phase-Contrast [28] H. Wan et al., “A new linear discriminant analysis method to address the
CT Technology,” Academic Radiology, 2018. over-reducing problem,” in Proc. Int.Conf. Pattern Recognit. Mach. Intell,
[13] Ying Yang et al., “Near-infrared light triggered liposomes combining pp. 65–72, 2015.
photodynamic and chemotherapy for synergistic breast tumor therapy,” [29] Y. Li et al., “Learning low-rank representations with classwise block-
Colloids and Surfaces B: Biointerfaces, vol. 173, pp. 564-570, 2019. diagonal structure for robust face recognition,” in Proc. AAAI, pp. 2810–
[14] R.G. Pleijhuis et al., “Near-infrared fluorescence (NIRF) imaging in 2816, 2014.
breast-conserving surgery: Assessing intraoperative techniques in tissue- [30] L. Li et al., “Learning low-rank and discriminative dictionary for image
simulating breast phantoms,” European Journal of Surgical Oncology classification,” Image Vis. Comput, vol. 32, no. 10, pp. 814–823, 2014.
(EJSO), vol. 37, no. 1, pp. 32-39, 2011. [31] G. Liu et al., “Robust recovery of subspace structures by low-rank
[15] Meihua Zhang et al., “Near-infrared photothermal therapy using EGFR- representation,” IEEE Trans. Pattern Anal. Mach. Intell, vol. 35, no. 1, pp.
targeted gold nanoparticles increases autophagic cell death in breast 171–184, Jan. 2013.
cancer,” Journal of Photochemistry and Photobiology B: Biology, vol. [32] C. Li et al., “Hyperspectral image classification with robust sparse
170, pp. 58-64, 2017. representation,” IEEE Trans. Geosci. Remote Sens, vol, 13, no. 5, pp.
[16] Xue Yang et al., “Assessment of spatial information for hyperspectral 641–645, Mar. 2016.
imaging of lesion,” International Society for Optics and Photonics, 2016. [33] J. Li et al., “Efficient superpixel-level multi-task joint sparse
[17] Francisco Peñaranda et al., “Multimodal registration of optical representation for hyperspectral image classification,” IEEE Trans.
microscopic and infrared spectroscopic images from different tissue Geosci. Remote Sens, vol. 53, no. 10, pp. 5338–5351, Oct. 2015.
sections: An application to colon cancer,” Digital Signal Processing, vol. [34] L. Xu et al., “Bayesian classification of hyperspectral imagery based on
68, pp. 1-15, 2017. probabilistic sparse representation and Markov random field,” IEEE
[18] Jonathan W. Uhr et al., “Molecular profiling of individual tumor cells by Geosci. Remote Sens. Lett, vol. 11, no. 4, pp. 823–827, Apr. 2014.
hyperspectral microscopic imaging,” Translational Research, vol. 159, no. [35] J. Li et al., “Semisupervised hyperspectral image classification using soft
5, pp. 366-375, 2012. sparse multinomial logistic regression,” IEEE Geosci. Remote Sens. Lett,
[19] P. Musto et al., “Hyperspectral Raman imaging of human prostatic cells: vol. 10, no. 2, pp. 318–322, Mar. 2013.
An attempt to differentiate normal and malignant cell lines by univariate [36] W. Gao et al., “Ideal kernel-based multiple kernel learning for spectral-
and multivariate data analysis,” Spectrochimica Acta Part A: Molecular spatial classification of hyperspectral image,” IEEE Geosci. Remote Sens.
and Biomolecular Spectroscopy, vol. 173, pp. 476-488, 2017. Lett, vol. 14, no. 7, pp. 1051–1055, Jul. 2017.
[20] Qingli Li et al., “Methyl green and nitrotetrazolium blue chloride co- [37] Q. Wang et al., "Locality Adaptive Discriminant Analysis for Spectral–
expression in colon tissue: A hyperspectral microscopic imaging analysis,” Spatial Classification of Hyperspectral Images," in IEEE Geoscience and
Optics & Laser Technology, vol. 64, pp. 337-342, 2014. Remote Sensing Letters, vol. 14, no. 11, pp. 2077-2081, Nov. 2017.
[21] S. Dontu et al., “Combined spectral-domain optical coherence [38] Q. Wang et al., "Locality and Structure Regularized Low Rank
tomography and hyperspectral imaging applied for tissue analysis: Representation for Hyperspectral Image Classification," in IEEE
Preliminary results,” Applied Surface Science, vol. 417, pp. 119-123, Transactions on Geoscience and Remote Sensing, vol. 57, no. 2, pp. 911-
2017. 923, Feb. 2019.
[22] Li G et al., “Employment of frame accumulation and shaped function for [39] Q. Gao et al., "Hyperspectral Image Classification Using Joint Sparse
upgrading low-light-level image detection sensitivity,” Optics Letters, Model and Discontinuity Preserving Relaxation," in IEEE Geoscience
vol.37, no. 8, pp. 1361-1363, 2012. and Remote Sensing Letters, vol. 15, no. 1, pp. 78-82, Jan. 2018.
[23] Yang X et al., “Effect on measurement accuracy of transillumination using [40] BREIMAN L, “Random forest,” Machine Learning, vol. 45, no. 1, pp. 5-
sawtooth-shaped-function optical signal,” Review of Scientific 32, 2001.
Instruments, vol. 87, no. 11, pp. 115106, 2016. [41] Huang G B et al., “Extreme learning machine: theory and applications,”
[24] Hu YJ et al., “Optimum method of image acquisition using sawtooth- Neurocomputing, vol. 70, no. 1-3, pp. 489-501, 2006.
shaped-function optical signal to improve grey-scale resolution,” Journal [42] Gang Li et al., “Optimum method of applying and removing a shaped-
of Modern Optics, vol. 63, pp. 1539-1543, 2016. function signal for low-light-level image detection,” Appl. Opt, vol. 52,
[25] G. Camps-Valls et al., “Spatio-spectral remote sensing image pp. 7934-7941, 2013.
classification with graph kernels,” IEEE Geosci. Remote Sens. Lett, vol. [43] ALTMANN A et al., “Permutation importance: a corrected feature
7, no. 4, pp. 741–745, Oct. 2010. importance measure,” Bioinformatics, vol. 26, no. 10, pp. 1340-1347,
[26] X. Guo et al., “Support tensor machines for classification of hyperspectral 2010.
remote sensing imagery,” IEEE Trans. Geosci. Remote Sens, vol. 54, no. [44] Hamed Vavadi et al., “Compact ultrasound-guided diffuse optical
6, pp. 3248–3264, Jun. 2016. tomography system for breast cancer imaging,” JOURNAL OF
[27] H. R. Shahdoosti et al., “Spectral–spatial feature extraction using BIOMEDICAL OPTICS, vol. 24, no. 2, 2019.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/.