This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

Date of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2018.Doi Number

A Glioma Segmentation Method Using

CoTraining and Superpixel-based Spatial and
Clinical Constraints
Tianming Zhan1,2,#, Fangqing Shen3,4,#, Xunning Hong5,*, Xihu Wang2, Yunjie Chen6,
Zhenyu Lu7, Guowei Yang1
1
Nanjing Audit University, School of Information and Engineering, Nanjing, Jiangsu 211815 China
2
Jiangsu University, School of Computer Science and Communication Engineering, Zhenjiang, Jiangsu 212013 China
3
Macau University of Science and Technology, Faculty of Hospitality and Tourism Management, Macau China
4
Jiangsu Maritime Institute, School of Humanity and Art, Nanjing, Jiangsu 211100 China
5
the First Affiliated Hospital of Nanjing Medical University, Department of Radiology, Nanjing, Jiangsu 210029 China
6
Nanjing University of Information Science and Technology, School of Math and Statistics, Nanjing, Jiangsu 210044 China
7
Nanjing University of Information Science and Technology, School of Electronic and Information Engineering, Nanjing, Jiangsu 210044 China

Corresponding author: Xunning Hong (hongxunning@sina.com).

# those authors contributed equally to this paper
This work is supported by the National Nature Science Foundation of China under Grant (No. 61502206, 61772277, 61773220, 61672291),
the Nature Science Foundation of Jiangsu Province under Grant (No. BK20150523, BK20161580).

ABSTRACT Various brain tumors are very harmful to human health, and their incidence rate has risen
gradually in recent years. Magnetic resonance imaging (MRI) is widely used in the evaluation and
treatment of brain tumors, but a radiologist’s manual segmentation of massive images is time-consuming
and produces some personal bias, and so a computer-aided diagnostic system could improve the working
efficiency of radiologists. In recent years, a variety of automatic segmentation methods for brain tumor
images has been proposed. Compared with other methods, machine-learning-based methods are able to
obtain better segmentation accuracy, but such methods require the manual labeling of lots of samples to
ensure the accuracy of the segmentation. In this study, semi-supervised learning theory and image spatial
and clinical a priori knowledge of brain tumors are combined to propose a new brain-tumor segmentation
method that can improve the segmentation accuracy with multiple classifier collaborative training
(CoTraining) under the premise of fewer labeled data. In addition, according to the prior knowledge that
image adjacent pixels belong to similar classes and clinical knowledge, we used superpixel graphs to
construct a spatial and clinical constraint to improve brain-tumor segmentation accuracy further. The
proposed method was tested on the Brain Tumor Segmentation Challenge 2012 and 2013 datasets (BRATS
2012, 2013). The experimental results demonstrate that the proposed method was able to overcome the
drawbacks of fewer training samples effectively and obtained remarkable brain-tumor segmentation
performance.

INDEX TERMS brain tumor; multi-sequence MRI; collaborative training; superpixel

I. INTRODUCTION Intracerebral tumors are consisted of glioma, ependymoma,

A tumor is the abnormal growth of tissue formed by cell lymphoma, medulablastoma and metastasis etc. [2]. Brainstem
proliferation under the action of a variety of tumor-inducing tumors (brainstem gliomas) account for 1.4% of intracranial
factors that most often becomes an occupying mass. tumors, mainly as gliomas. Astrocytomas and polar
According to the cell characteristics of the abnormal regions glioblastoma are more common, followed by glioblastoma,
and the degree of harm to the body, tumors are mainly ependyma glioma, and medulloblastoma. Besides,
divided into two categories: benign and malignant [1]. hemangiomas, cysts, teratoma, tuberculoma, abcess etc. are
Intracerebral tumors, which are also called brain tumors, also examined for clinical diagnosis. In particular, children,
affect the nervous system and occur within the cranial cavity. especially 5-9-years-old, and adolescents suffer from such
2169-3536 © 2018 IEEE. Translations and content mining are permitted for academic research only.
VOLUME XX, 2018 Personal use is also permitted, but republication/redistribution requires IEEE permission. 1
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access
diseases. According to the degree of malignancy of a glioma,
it can be divided into low-grade or high-grade [3].
Multi-sequence magnetic resonance imaging (MRI) is
often used in the evaluation of gliomas to assist in clinical
diagnosis, qualitative analysis, and treatment. The commonly
used image sequences include T1-weighted, T2-weighted,
fluid-attenuated inversion recovery (FLAIR), and post-
contrast T1-weighted images [4]. The appearance of a glioma
in the various sequence image methods are different. In the
T1-weighted sequence images, the regions of brain tissue
cannot be clearly segmented because the image intensities of
the tumor region are similar to those of the gray matter. In
the post-contrast T1-weighted images, the active and necrotic
regions of a tumor can be clearly distinguished for that the
active tumor shows bright while the necrotic tissue
demonstrates dark. Meanwhile, compared to images of brain
tissues, the intensities of edema and tumor regions are much
higher in the T2-weighted sequence images and FLAIR
images and the intensities of cerebrospinal fluid (CSF) is
higher in the T2-weighted images and lower in FLAIR
images.
At present, there is an urgent necessity to extract brain
glioma images automatically for clinical diagnosis to provide
accurate tumor information for subsequent treatment
planning and assessment. In recent years, many glioma
segmentation methods using multi-sequence MRI have been
proposed, and among them, methods based on machine-
learning classification using the already marked samples to
train a classification model and then utilizing the trained
model to classify the test samples have been widely
employed [5-12]. The commonly used machine-learning
methods include support vector machines (SVMs)[13], sparse
representation classification (SRC)[14], multinomial logistic
regression[15], and so on. With the explosive development of
deep learning methods, convolutional neural networks
(CNNs)[16,17], deep belief networks (DBNs)[18], and other
neural networks have also been applied in brain-tumor
segmentation. However, these neural network methods
require a large amount of marked data to train a good
classification model, and the classification performance
cannot be guaranteed when the number of training samples is
limited. However, manually labeling enough data is time-
consuming. In a clinical setting, it is necessary to use a
smaller number of labeled data to train a good classification
model and thus obtain preferable tumor segmentation results.
In clinical applications, unlabeled data is relatively easy to
obtain, and so the semi-supervised learning method[19-21] has
become a hot topic in the research field of machine learning
in recent years. This method can use a small number of
labeled data and a large number of unlabeled data to train a
good classifier for perfectly solving the problem of fewer
labeled data in practical applications. Among them, the
difference-based semi-supervised learning method obtains
the difference between the two base classifiers and utilizes
these differences to achieve their complementarity. Such a
VOLUME XX, 2018
2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
method is very simple and easy to use for unlabeled data and
does not require additional prior knowledge for the model
assumptions. It is a very effective semi-supervised learning
approach in practical applications.
Aimed at the clinical application of glioma segmentation,
we propose a new glioma segmentation method using the
multiple classifier collaborative training (CoTraining)
algorithm based on the difference between SVM and SRC
classifiers, and superpixel-based spatial and clinical
constraints. The method includes the following key steps.
First, labeled brain MRI data are used to train the SVM and
SRC classifiers for selecting high confidence data from the
testing samples as pseudo-labeled samples. Second, the
pseudo-labeled samples obtained from each of the classifiers
are added to the other side of the training set to retrain the
corresponding classifier; this process is executed iteratively
until a stable result is obtained. Finally, a superpixel graph is
constructed on the post-contrast T1-weighted image to build
a spatial and clinical constraint. The glioma segmentation
model is then established according to the classification
probability and prior constraint, and the glioma segmentation
results are obtained by optimizing the preceding model. This
method has two advantages. First, the classification accuracy
is improved by the iterative method for the two-classifier
collaborative training when the number of labeled samples is
insufficient, and second, the construction of the spatial and
clinical constraint using a superpixel graph can overcome the
interference caused by noise and false positive regions, and
thus further improve the segmentation accuracy. The
proposed method was tested on Brain Tumor Segmentation
Challenge 2012 and 2013 datasets (BRATS 2012, BRATS
2013) and the experimental results demonstrate that this
method was able to effectively overcome the drawbacks of
insufficient training samples and unstable spatial distribution,
and obtained competitive glioma segmentation accuracy.
II. THE PROPOSED METHOD
The overall process of the proposed glioma segmentation
method using CoTraining and superpixel-based spatial
and clinical constraints is shown in Figure 1.
FIGURE 1. A flowchart of the proposed method
A. DATA PREPROCESSING
Due to the limitations of the imaging mechanism, if the
intensity distribution of an MR image is nonhomogeneous, it
9
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

will lead to differences in intensity for the same tissues. We where  'j are the slack variables on the pseudo-labeled data
used the N4ITK method[22] to carry out nonhomogeneous
and 1 is a weight parameter. The minimization of this
image correction and to adjust the intensity values of each
MR image sequence into [0 255]. After this, we intercepted model can be transferred to solve the dual problem, the
details of which are available in reference [21].
the image block of each pixel in different sequence images
and transformed it into a vector. The values of vector are the 2) THE WEIGHTED SRC METHOD
intensities of the pixel and its neighborhood in different After setting D as the labeled dataset and y as the
sequences. For each pixel on one slice of an image, its block unlabeled sample, the SRC classifier can obtain the sparse
size is    . For k sequence images, the initial feature size representation coefficient  of y on D by
of one pixel is     k and its corresponding vector size min   1  D  y
2
. （3）
2 2
is k  1 . We set k =4 in this study and used these 
*
intensity vectors as the input for the CoTraining method. After optimal coefficient  is obtained, the corresponding
class of y can be determined by the representation residuals
B. THE COTRAINING METHOD on each class as
We used the SVM classifier and SRC classifier as the base
y* = arg min ek ( y ) ， ek ( y )  y  Dk  k* .（4）
classifiers for CoTraining. Those samples with high 2

confidence are selected as pseudo-labeled samples to classify Where Dk represents the data subset of the k -th class and
the unlabeled data. The pseudo-labeled samples obtained by
 k* is the optimal representation coefficient on Dk .
one classifier are added to the training set for retraining with
the other classifier; this cross process is executed iteratively After the introduction of the pseudo-labeled data, objective
until a stable result has been obtained. We also assign function (2) can be rewritten as
min(   1  D  y 2 +2 (   ' 1  D ' ' y 2 )) , （5）
2 2
weights for the pseudo-labeled data to modify the target  ， '
functions of the SVM and SRC classifiers and thus propose a where D ' is the pseudo-labeled dataset and  ' is the
weighted SVM and SRC classification model. coefficient of y on D ' . The details of the optimization of (5)
can be found in reference [16].
After calculating the optimal coefficients  * and  '* , the
appropriate class of y can be determined by
y * = arg min f k ( y ) ，
f k ( y )  y  Dk  k* +2 y  D 'k  '*k . （6）
2 2

3) THE CALCULATION OF CONFIDENCE

According to the classification results of the two base
classifiers, we can calculate the classification probabilities of
FIGURE 2. The workflow for the CoTraining algorithm
each pixel (the details of the classification probability of
1) THE WEIGHTED SVM METHOD SVM classifiers can be found in reference [21]). The
We set xi , i  1, 2, , n as labeled samples for the SVM classification probability of SRC classifier can be constructed
classifier to establish a classification model by seeking a as:
decision surface represented by f ( x)  wT x  b . The K
pSRC (ck | y)  f k ( y ) /  fl ( y) , (7)
parameters {w, b} can be calculated by optimizing the l 1
following objective function:
n
where ck is the k -th class, cmax_ k represents the class with
1 2
min w  C  i ; （1） the maximum prediction probability, and csub max_ k represents
w, b 2
i 1
the class with the second largest value of the predicted
s.t. yi (( wT xi )  b)  1  i , i  1, 2, , n ;
probability. The confidence values can be calculated using
i  0 , i  1, 2, , n ; the probabilities obtained by the two base classifiers as
SVM SVM
k | y )  pSVM (csub max_ k | y ) , (8)
where i are the slack variables on the labeled data and C is Conf SVM ( y )  2 pSVM (cmax_
a penalty factor. According to reference [21], objective SRC SRC
Conf SRC ( y )  2 pSRC (cmax_ k | y )  pSRC (csub max_ k | y ) . (9)
function (1) after the introduction of pseudo-labeled data can
be rewritten as
n m 4) THE SVM-SRC COTRAINING ALGORITHM
1 2
min w  C（ i +1   j'）; （2）
w, b 2
i 1 j 1 The SVM-SRC CoTraining algorithm is provided as follows.
s. t. yi (( w xi )  b)  1  i , i  1, 2, , n ;
T
Algorithm 1
 y j (( wT x j )  b)  1   j' , j  1, 2, , m ; Input: A set of labeled samples X and unlabeled samples Y
i  0 , i  1, 2, , n ;   0 , j  1, 2, , m ;
'
j

VOLUME XX, 2018 9

2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access
Initialize the pseudo-labeled datasets P1 , P2   and
the two unlabeled datasets: YSVM , YSRC =Y ;
Step 1: Train the SVM classifier using X to classify the data
in Y and calculate the confidences;
Step 2: Use X as the SRC dictionary to classify the data in
Y and the calculate the confidences;
Step 3: Select the high confidence samples from Y to set
pseudo-labeled dataset P1 and update YSVM =Y  P1 ;
Step 4: Select the high confidence samples from
pseudo-labeled dataset P2 and update YSRC =Y  P2 ;
Step 5: Use X and P1 to retrain SVM, and carry out
classification for YSVM ; Calculate the confidence and
update P1 and YSVM ;
Step 6: Use X and P2 to retrain SRC and carry out
classification for YSRC ; Calculate the confidence and
update P2 and YSRC ;
Step 7: Determine whether the algorithm is at convergence
or not; If not go to step 5; Otherwise go to step 8；
Step 8: Output the classification model and classification
results.
C. THE SVM-SRC COTRAINING ALGORITHM
The influence of spatial information on segmentation results
is very strong, thus in addition to using the pixel intensities,
we should also introduce the spatial constraints of the image
to improve glioma segmentation accuracy. Superpixel
segmentation is widely used in the field of computer vision
for tasks such as target recognition, image segmentation, etc.
[25]. A superpixel is an image block that adaptively changes
its size and shape according to its features, and inside one,
the features of all the pixels are close. Therefore, the pixels
located in the same superpixel belong to the same class.
Accordingly, using superpixels to select the homogeneous
regions can overcome the shortcomings of the traditional
rectangular windows that are always containing
heterogeneous information. We can use the entropy-rate
superpixel-segmentation method [25] to generate the
superpixels on a post-contrast T1-weighted image, as shown
in Figure 3. From Figure 3, the intensities of each
homogeneous region in a post-contrast T1- weighted image
is close, and the area of each region is also very large. Thus,
we set  =5 and 100 superpixels when running the entropy-
rate superpixel-segmentation method on the post-contrast T1-
weighted image in the following experiments.
VOLUME XX, 2018
2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
FIGURE 3. The superpixels obtained from a post-contrast T1-weighted
image
Y to set
In each superpixel, all of the pixels belong to the same
class. However, it is possible to classify all of the pixels into
an incorrect class if a single pixel classification is wrong,
although prior clinical knowledge can resolve this problem.
When we think that the class corresponding to a manually
selected pixel is correct, the corresponding classes of all
other pixels in its superpixel should be the same as the class
of the labeled samples. As shown in Figure 4, we introduce
the following clinical constraint:
l j  l f ( i ) ， f (i), j  St ,
where, l f (i ) represents the class of the i -th labeled sample,
St is the t -th superpixel, and f denotes a collection of
labeled samples.
Manually labeled pixel
0.6 1
0.5 0.7 0.6 0.5 1 1 1 1
1 0.7 1
0.4 0.5 1 1
Superpixel
FIGURE 4. The constraint result using clinical a priori knowledge
D. THE FINAL SEGMENTATION MODEL AND
ALGORITHM
Accurate classification results cannot be obtained by using
only the intensities of multi-sequence MR images. Spatial
and clinical knowledge also plays a very important role in
glioma identification. In simulation dataset, the ground truth
contains glioma region, edema, and brain tissues. While, in
the clinical dataset, the ground truth of clinical data
contains necrosis, enhanced tumor, non-enhanced tumor,
and edema. Thus, the corresponding classification result of
simulation data obtained by CoTraining algorithm contains
5 classes: tumor region, edema, white matter, gray matter
and CSF. The corresponding classification result of clinical
data obtained by CoTraining algorithm contains 7 classes:
necrosis, enhanced tumor, non-enhanced tumor, edema and
3 normal brain tissues categories.
In order to extract the complete glioma region from the
image, we integrate the classification regions into three
9
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

classes: glioma, edema, and brain tissue. Setting L  {1, 2,3}

Eqs. (13)-(15) according to different
to represent the labels for these three classes, the probability
dataset;
of a pixel belonging to one of them in simulation data and
Step4: Generate the superpixel graph on the post-
clinical data can be calculated as Eqs. (10)-(12) and Eqs.
contrast T1-weighted image;
(13)-(15), respectively:
Step5: Minimize Eq. (18) to obtain the
p ( yi  L1 | xi )  p ( yi  l1 | xi ) , (10) classification results;
p( yi  L2 | xi )  p( yi  l2 | xi ) , (11) Step6: Fine-tune the brain tumor and edema
5 regions;
p ( yi  L3 | xi )   p ( yi  l j | xi ) . (12) Step7: Output the segmentation results;
j 3
3 The fine-tuning in the sixth step consists of two steps:
p ( yi  L1 | xi )   p ( yi  l j | xi ) , (13) (1) determine the regions located in the inner region of the
j 1
brain tumor labeled as other classes and then reclassify
p( yi  L2 | xi )  p( yi  l4 | xi ) , (14) them as the correct class and (2) calculate all of the
7 connected image block sizes of the glioma. A block is
p ( yi  L3 | xi )   p ( yi  l j | xi ) . (15) regarded as a false positive region and is removed if it is
j 5
smaller than the maximum tumor mass of 10%.
Next, the segmentation model using the probabilities and
the spatial constraint is given by III. EXPERIMENTAL RESULTS
 2 2
Q*  argmin  P  Q F    Wi, j Qi  Qj  , (16) A. THE DATASETS
2
Q  i |i  j|  Two datasets were used to verify the proposed method from
s.t. Q  0 , 1T Qi  1 , i  1, 2,  , I ; the MICCAI BRATS Challenge [27] in 2012 (Brats 2012,
http://www.imm.dtu.dk/projects/BRATS2012）and in 2013
where P is the classification probability vector generated
(Brats 2013, http://martinos.org/qtim/miccai2013/data.html).
by (10)-(12) or (13)-(15), Q is the smoothness term of P ,
Each dataset contained a low-grade glioma dataset and a
 is the regularization parameter, and W is the index of
high-grade gliomas dataset, in which each case consisted of
whether two pixels are located in the same superpixel or not,
a T1-weighted sequence image, a T2-weighted sequence
which can be derived as
image, a FLAIR sequence image, and a post-contrast T1-
1 Si  S j weighted image. All of the sequence images for each case
Wi , j =  , (17)
0 Si  S j had been registered and contained a standard segmentation
where Si and S j represent the superpixels containing the result given by a radiologist. For each case, the ground truth
of the simulation data contained tumor, edema, white
i -th and j -th pixels. We set  to represent the index of matter, gray matter, and CSF, while the ground truth of the
the pixels in the superpixel where all the labeled samples clinical data contained necrosis, enhanced tumor, non-
are located. With this clinical constraint, the segmentation enhanced tumor, and edema. In our experiment, we
model of (16) can be rewritten as classified the images into three classes: glioma, edema, and
 2 2 brain tissue. Some labeled voxels are randomly selected as
Q*  argmin  P  Q F    Wi, j Qi  Qj  , (18) training samples, and the remaining voxels are used as
2
Q  i |i  j| 
testing samples in our experiments.
s.t. Q  0 , Q  Z  , 1T Qi  1 , i  1, 2,  , I .
B. THE NUMBER OF ITERATIONS
The optimization of this model can be solved through the
For clinical high-grade glioma datasets, 30, 10, and 90
alternating direction method of multipliers (ADMM)
labeled voxels were randomly selected from glioma, edema,
algorithm, the details of which can be found in reference
and brain tissues to establish a training sample set. After 30
[26].
iterations, the classification results for glioma and edema
The complete process for the proposed method is given
using SVM and SRC were obtained (see Figure 5(a)). For
in the following algorithm.
the low-grade glioma datasets, the same procedure was
Algorithm 2 followed but with 30, 5, and 60 samples (see Figure 5(b)).
Input: Multi-sequence MR images, manually With an increasing number of iterations, the total
labeled samples classification accuracy of the SVM and SRC methods
Step1: Preprocessing； tended to increase and the classification accuracy of the
Step2: Calculate the classification probability former increased more significantly than the latter, although
using Algorithm 1; their final classification accuracies were close together.
Step3: Calculate the glioma, edema, and brain Moreover, the classification accuracy increased slowly
tissue probabilities using Eqs. (10) - (12) or when the number of iterations reached 20, so we chose the
optimal iteration number in the range of [20, 30].

VOLUME XX, 2018 9

2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

(a) (b)
FIGURE 6. The average accuracy of tumor and edema segmentation
results for the simulation datasets with increasing iterations

C. THE WEIGHT COEFFICIENTS

In the SVM and SRC classification models, two
coefficients 1 and 2 are used to regulate the weight of the
labeled samples and the pseudo-labeled samples during the
classification process. Figure 7 and Figure 8 show the
average classification accuracy of the two methods when
we set the weight value between 0.1 and 1 for the clinical
and simulation datasets, respectively. It can be seen from
the results that the average classification accuracy was
much higher when the weight coefficient was between 0.3
(b) and 0.6, and thus we set 1 =0.5 and 2 =0.5 in our
FIGURE 5. The average accuracy of tumor and edema segmentation
experiments.
results for the clinical datasets with increasing iterations
For the simulation high-grade glioma datasets, 30, 10,
and 90 labeled voxels were randomly selected from glioma,
edema, and brain tissues to establish a training sample set.
After 30 iterations, the classification results of glioma and
edema using SVM and SRC were obtained (see Figure 6(a)).
A similar procedure was carried out for the simulation low-
grade glioma datasets but with 30, 5, and 60 samples (see
Figure 6(b)). As can be seen from the results, the
classification accuracy of the SVM and SRC methods
increased with an increasing number of iterations and the
final classification accuracies of the two methods were very
close. The classification accuracy rose slowly when the (a)
number of iterations reached 10, so we set the number of
iterations larger than 10 for the simulation datasets.

(b)
FIGURE 7.The average accuracies for SVM and SRC executed on the
clinical dataset with different parameter values

(a)

VOLUME XX, 2018 9

2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

(a) FIGURE 9. The average accuracies of SVM and SRC executed on the
clinical dataset with different numbers of training samples

FIGURE 10. The average accuracies of SVM and SRC executed on the
(b) simulation dataset with different numbers of training samples
FIGURE 8. The average accuracies for SVM and SRC executed on the
simulation dataset with different parameter values E. THE SPATIAL AND CLINICAL CONSTRAINTS’
SEGMENTATION RESULTS
D. THE NUMBER OF INITIAL TRAINING SAMPLES Figure 11 and Figure 12 demonstrate the segmentation
The number of initial training samples determines the results of the two datasets using CoTraining, CoTraining
classification effect of CoTraining. The CoTraining method with the spatial constraint, and CoTraining with the spatial
does not play a significant role in improving the and clinical constraints. From the results, the noise effect
classification accuracy when the number of initial training was overcome very well and the result was much smoother
samples is very large but may lead to a further reduction in after adding the spatial knowledge during the segmentation
classification accuracy even though the effect is not as good process. The false positive region was eliminated
as the result of a single classifier when the number of initial effectively and the segmentation accuracy was further
samples is small. Figures 9 and 10 show the classification improved when the clinical constraint was considered
accuracy for the clinical and simulation datasets, during the segmentation.
respectively, using different brain tumor initial training
samples. Edema and brain tissues were set according to the
ratio of 3:1 to 1:3 in the experiment. As can be seen from
the images, the classification accuracy was very low when
the number of initial training samples was within 10 and
basically met the clinical requirement when the initial
sample number was between 20 and 40. In our experiment,
the number of initial brain-tumor training samples was set
at 30.

VOLUME XX, 2018 9

2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

FIGURE 11. The segmentation results on clinical datasets. The rows in

order: the post-contrast T1-weighted image, CoTraining segmentation,
CoTraining with the spatial constraint, CoTraining with the spatial and
clinical constraints, and the ground truth.

FIGURE 13. The full segmentation results on BraTS 2013 datasets. The
rows in order: the post-contrast T1-weighted image, CoTraining with the
spatial and clinical constraints and the ground truth.

F. COMPARISON OF SEGMENTATION ACCURACY

In Table 1, we compare the proposed method with
some glioma extraction methods, some with deep learning
methods, using the BRATS2012 and BRATS2013 datasets.
For the simulation dataset we mainly compared the
complete tumor region (necrosis, enhanced tumors, non-
enhanced tumors, and edema), and for the clinical dataset,
we mainly compared the tumor core region (necrotic,
enhanced and non-enhancing tumors). From Table 1, we
can see that no single method was able to achieve the best
result for all of the datasets, although the dice score
obtained by the proposed method was either the highest or
close to it for each case. This result demonstrates that the
proposed method is competitive in the automatic extraction
of a brain glioma image.
TABLE I
THE AVERAGE DICE SCORES OBTAINED BY DIFFERENT SEGMENTATION
METHODS ON THE BRATS DATASETS
BRATS 2012 BRATS 2012 BRATS 2013
Dataset Simulation Clinical Clinical
FIGURE 12. The segmentation results on simulation datasets. The rows Dataset Dataset dataset
in order: the post-contrast T1-weighted image, CoTraining segmentation, Zikic et al. [28] 0.91 0.75 ~
CoTraining with the spatial constraint, CoTraining with the spatial and
clinical constraints, and the ground truth.
Bauer et al. [29] 0.87 0.75 ~
LIPC [30] 0.92 0.82 0.86
Figure 13 demonstrates the full segmentation results of CNNs [16] ~ ~ 0.88
BraTS 2013 data set. The tumor region is further divided Cordier et al. [31] ~ ~ 0.86
SVM_SC 0.84 0.70 0.76
into necrosis region, enhancing tumor by mapping the SRC_SC 0.86 0.75 0.80
classification results of tumor region using CoTraining SVM_SC_SP 0.86 0.78 0.82
method into the tumor region obtained by CoTraining with SRC_SC_SP 0.89 0.81 0.85
spatial and clinical constraints. In further segmentation of The proposed
0.91 0.84 0.88
tumor, the spatial and clinical constraints are not introduced. method
The reason is that the boundaries between necrosis region
and enhancing tumor region are not clear in clinical In Table II, we compile the enhancing tumor
analysis. The segmentation of necrosis and active tumor segmentation results of several methods in the challenge
will be poor if we introduce the spatial and clinical data set of BraTS 2013. Appraising the results in Table II,
constraints during the further segmentation of tumor region. no method is yet able to obtain the best tumor segmentation

VOLUME XX, 2018 9

2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

results. However, the segmentation results obtained by the [4] Kircher M F, Mahmood U, King R S, et al. A multimodal
proposed method are at the forefront and are competitive. nanoparticle for preoperative magnetic resonance imaging and
TABLE II intraoperative optical brain tumor delineation[J]. Cancer Research,
RESULTS IN THE CHALLENGE DATASET OF BRATS 2013. 2003, 63(23):8122-8125.
[5] Qian J, Yang J, Gao G. Discriminative histograms of local dominant
DSC PPV Sensitivity
Dataset orientation (D-HLDO) for biometric image feature extraction.
Core Enh. Core Enh. Core Enh. Pattern Recognition, 2013, 46(10): 2724-2739.
Kwon et al. [32] 0.83 0.72 0.90 0.74 0.78 0.72 [6] Zhang Y, Peng B, Wang S, et al. Image processing methods to
Tustison et al. [33] 0.78 0.74 0.74 0.69 0.88 0.83 elucidate spatial characteristics of retinal microglia after optic nerve
CNNs [16] 0.83 0.77 0.87 0.74 0.83 0.81 transection. Scientific reports, 2016, 6:21816.
Urban et al. [34] 0.75 0.73 0.75 0.79 0.79 0.80
[7] Hong R, Tang L, Hu J, et al. Advertising object in web videos.
Havaei et al. [35] 0.79 0.73 0.79 0.68 0.79 0.80
Davy et al. [36] 0.74 0.68 0.74 0.62 0.78 0.77 Neurocomputing, 119: 118-124,2013.
The proposed [8] Wu Z, Wang Q, Plaza A, et al. Parallel Spatial–Spectral
0.81 0.74 0.87 0.74 0.84 0.83
method Hyperspectral Image Classification With Sparse Representation and
Markov Random Fields on GPUs[J]. IEEE Journal of Selected
IV. CONCLUSIONS Topics in Applied Earth Observations & Remote Sensing, 2015,
A CoTraining method with spatial and clinical 8(6):2926-2938.
constraints is proposed in this paper aimed at the problem [9] Hong R, Cao W, Pang J, et al. Directional projection based image
of automatic glioma image extraction from multi-sequence fusion quality metric. Information Sciences, 2014, 281: 611-619.
MR images. First, two base classifiers SVM and SRC are [10] Xu Y, Wu Z, Wei Z. Spectral–Spatial Classification of Hyperspectral
trained with a small number of labeled samples, then high Image Based on Low-Rank Decomposition[J]. IEEE Journal of
confidence samples are selected as pseudo-labeled samples Selected Topics in Applied Earth Observations & Remote Sensing,
according to the classification results for the two base 2015, 8(6):2370-2380.
classifiers. Next, the pseudo-labeled samples obtained from [11] Sun L, Wu Z, Liu J, et al. Supervised Spectral–Spatial Hyperspectral
each classifier are added to each other's training sets, and Image Classification With Weighted Markov Random Fields[J].
this process is repeated iteratively until the classification IEEE Transactions on Geoscience & Remote Sensing, 2015,
results are stable. Finally, we use a T1- enhanced sequence 53(3):1490-1503.
image to construct a superpixel graph and design spatial [12] Wu Z, Wang Q, Plaza A, et al. Real-Time Implementation of the
and clinical constraints. A segmentation model based on Sparse Multinomial Logistic Regression for Hyperspectral Image
classification probability and spatial and clinical constraints Classification on GPUs[J]. IEEE Geoscience & Remote Sensing
is then minimized to obtain the final glioma segmentation Letters, 2015, 12(7):1456-1460.
results. [13] Ramakrishnan T, Sankaragomathi B. A professional estimate on the
By using the two base classifiers to carry out cross- computed tomography brain tumor images using SVM-SMO for
training iteratively, the proposed method was able to classification and MRG-GWO for segmentation[J]. Pattern
improve the classification accuracy when the labeled Recognition Letters, 2017, 94.
samples were insufficient. In addition, by constructing [14] Li Y, Jia F, Qin J. Brain tumor segmentation from multimodal
image space and clinical a priori knowledge constraints, the magnetic resonance images via sparse representation[J]. Artificial
interference of noise and false positive regions was Intelligence in Medicine, 2016, 73:1.
removed and the segmentation precision was further [15] Ting G E, Ning M U, Li L I. A Brain Tumor Segmentation Method
improved. The proposed method was performed on the Based on Softmax Regression and Graph Cut[J]. Acta Electronica
BRATS2012 and BRATS2013 datasets, and the average Sinica, 2017, 45(3):644-649.
segmentation accuracy demonstrated that the proposed [16] Pereira S, Pinto A, Alves V, et al. Brain Tumor Segmentation using
method was able to obtain good performance for glioma Convolutional Neural Networks in MRI Images.[J]. IEEE
segmentation. Transactions on Medical Imaging, 2016, 35(5):1240-1251.
REFERENCES [17] Iqbal S, Ghani M U, Saba T, et al. Brain tumor segmentation in
[1] Khwairakpam A D, Monisha J, Banik K, et al. Chemoresistance in multi-spectral MRI using convolutional neural networks (CNN)[J].
Brain Cancer and Different Chemosensitization Approaches[M]// Microsc Res Tech, 2018(10).
Cancer Cell Chemoresistance and Chemosensitization. 2018. [18] Zhan T, Chen Y, Hong X, et al. Brain Tumor Segmentation Using
[2] Vierhout M, Daniels M, Mazzotta P, et al. The views of patients Deep Belief Networks and Pathological Knowledge[J]. CNS &
with brain cancer about palliative care: a qualitative study[J]. Neurological Disorders Drug Targets, 2017, 16(2):129-136.
Current Oncology, 2017, 24(6):374. [19] Saha S, Alok A K, Ekbal A. Brain image segmentation using semi-
[3] Baene W D, Rutten G, Sitskoorn M M. P04.09 Low-grade and high- supervised clustering[J]. Expert Systems with Applications, 2016,
grade glioma patients show different remote effects of the brain 52(C):50-63.
tumor on the functional network topology of the contralesional [20] Yang L, Zhang Y, Chen J, et al. Suggestive Annotation: A Deep
hemisphere[J]. Neuro-Oncology, 2017, 19(suppl_3):iii41-iii42. Active Learning Framework for Biomedical Image Segmentation[J].

VOLUME XX, 2018 9

2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

2017:399-407. [36] Havaei M, Davy A, Warde-Farley D, et al. Brain tumor

[21] Abdelgayed T S, Morsi W G, Sidhu T S. Fault Detection and segmentation with Deep Neural Networks[J]. Medical Image
Classification based on Co-Training of Semi-Supervised Machine Analysis, 2017, 35:18-31.
Learning[J]. IEEE Transactions on Industrial Electronics, 2017,
65(2):1595-1605. Tianming Zhan received the B.S. degree
and M. S. degree from the School of Math
[22] Tustison N J, Avants B B, Cook P A, et al. N4ITK: improved N3 and Statistics, Nanjing University of
bias correction[J]. IEEE Transactions on Medical Imaging, 2010, Information Science and Technology,
29(6):1310-20. Nanjing, China, in 2006 and 2009,
respectively, and Ph. D degree with the
[23] Jiang Z, Zhang S, Zeng J. A hybrid generative/discriminative
School of Computer Science and Engineering,
method for semi-supervised classification[J]. Knowledge based Nanjing University of Science and
System, 2013,37(2):137-145. Technology, in 2013. He is currently an
[24] Shen C, Chen L, Priebe C E. Sparse Representation Classification associated professor with the School of
Information and Engineering, Nanjing Audit
Beyond L1 Minimization and the Subspace Assumption[J]. University. His research interests include
Statistics, 2015. medical image processing, hyperspectral image processing, machine
[25] Cong W, Song J, Wang L, et al. A Fuzzy c-Means Clustering learning, and data analysis.
Scheme Incorporating Non-Local Spatial Constraint for Brain
Magnetic Resonance Image Segmentation[J]. Journal of Medical Fangqing Sheng received the M.S. degree
Imaging & Health Informatics, 2015, 5(8):1821-1825. in the School of Geography Science, Nanjing
Normal University, PR China, in 2014. Now
[26] Sun L, Wu Z, Liu J, et al. Supervised hyperspectral image
she is pursuing the Ph.D. degree in Tourism
classification using sparse logistic regression and spatial-TV Management from Macau University of
regularization[C]// Geoscience and Remote Sensing Symposium. Science and Technology. Her current
IEEE, 2014:1019-1022. research interests include data mining and
smart tourism. E-mail: kiralice@163.com
[27] Menze B H, Jakab A, Bauer S, et al. The Multimodal Brain Tumor
Image Segmentation Benchmark (BRATS)[J]. IEEE Transactions on
Medical Imaging, 2015, 34(10):1993-2024.
[28] D. Zikic, B. Glocker, E. Konukoglu, J. Shotton, A. Criminisi, D. H.
Xunning Hong graduated from the medical
Ye,C. Demiralp, O. M. Thomas, T. Das, R. Jena, and S. J. Price, department of Shanghai Medical University
Contextsensitive classification forests for segmentation of brain in 1992. After graduation, he engaged in
tumor tissues[C]// MICCAI BRATS Challenge. 2012., Nice, France, medical imaging diagnosis. In 2003, he
graduated from Fudan University and
2012.
received a Ph. D. in imaging medicine.
[29] S. Bauer, T. Fejes, J. Slotboom, R. Wiest, L.-P. Nolte, and M. Reyes. During his period, he studied the MRI
Segmentation of brain tumor images based on integrated research on nervous system diseases at the
Huashan Hospital of Fudan University,
hierarchical classification and regularization[C]// MICCAI BRATS
especially the functional MRI research. He
Challenge. 2012., Nice, France, 2012. went to the York University medical school
[30] Huang M, Yang W, Wu Y, et al. Brain Tumor Segmentation Based for further training in neuroimaging diagnosis
on Local Independent Projection-Based Classification[J]. IEEE during 2007-2008. He is current a chief physician in Radiology
Department, Jiangsu Province Hospital and an associated professor in the
transactions on bio-medical engineering, 2014, 61(10):2633-45. School of Medical Imaging, Nanjing Medical University. His research
[31] Cordier N, Menze B, Delingette H, et al. Patch-based Segmentation interests include medical imaging analysis, brain tumor classification, and
of Brain Tissues[C]// MICCAI Challenge on Multimodal Brain brain disease diagnosis.
Tumor Segmentation, 2013, Nagoya, Japan, 2013. Xihu Wang received his B. S. degree in the
[32] Kwon D. et al., Combining generative models for multifocal glioma School of Computer Science and Engineering,
segmentation and registration[C]. In Medical Image Computing and Jining Medical Institute Shandong, in 2014.
He is currently pursuing M. S. degree in the
Computer-Assisted Intervention–MICCAI 2014. Springer, 2014, pp.
School of Computer Science and
763–770. Communication Engineering, Jiangsu
[33] Tustison N J, Shrinidhi K L, Wintermark M, et al. Optimal University, Zhenjiang China. His research
Symmetric Multimodal Templates and Concatenated Random interests include medical imaging
segmentation and registration.
Forests for Supervised Brain Tumor Segmentation (Simplified) with
ANTsR[J]. Neuroinformatics, 2015, 13(2):209-225.
[34] Urban G, Bendszus M, Hamprecht F A, et al. Multi-modal brain
Yunjie Chen received the B.S. degree and
tumor segmentation using deep convolutional neural networks[C]//
M. S. degree from the School of Math and
MICCAI BraTS (Brain Tumor Segmentation) Challenge. Statistics, Nanjing University of Information
Proceedings, winning contribution. 2014. Science and Technology, Nanjing, China, in
2002 and 2005, respectively, and Ph. D
[35] Havaei M, Davy A, Warde-Farley D, et al. Brain tumor
degree with the School of Computer Science
segmentation with Deep Neural Networks[J]. Medical Image and Engineering, Nanjing University of
Analysis, 2017, 35:18-31. Science and Technology, in 2009. He is

VOLUME XX, 2018 9

2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
 This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/ACCESS.2018.2873674, IEEE Access

currently a professor with the School of Math and Statistics, Nanjing

University of Information Science and Technology. His research interests
include medical image processing, Statistic analysis, and machine learning.

Zhenyu Lu was born in Changzhou, Jiangsu,

Chinia, in 1976. He received his Ph. D.
degree in optical engineering at Nanjing
University of Science and Technology,
Nanjing, China in 2008. Now, he is a
professor in the School of Electronic and
Information Engineering at Nanjing
University of Science and Information
Technology. His research interests include
intelligent control and the stochastic control,
pattern recognition, and data mining.

Guowei Yang was born in Zhangshu

Jiangxi China, in 1964. He received the B. S.
degree and M. S. degree from the Department
of Mathematics, Jiangxi Normal University,
in 1985 and 1988, respectively, and received
his Ph. D. degree from the School of
Information Engineering, Beijing University
of Science and Technology, in 2004. He was
an advanced visiting scholar at University of
Illinois during 2008 to 2009. He is current a
professor with the School of Information and
Engineering, Nanjing Audit University. Also,
He is a director of Artificial Intelligence Society, vice director of extension
engineering of Specialized Committee, China Society of artificial
intelligence. His research fields include Extenics, and Big data processing.

VOLUME XX, 2018 9

2169-3536 (c) 2018 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.