A Classification method of the Osteonecrosis of
Femoral Head based on Deep Learning
Liyang Zhu, Jungang Han
Shaanxi Key Laboratory of Network Data Analysis and
Intelligence Processing
School of Computer, Xi’an University of Posts &
Telecommunications
Xi’an, China
e-mail: zlyinter@sina.com
Wei Chai
Department of Orthopaedics
Chinese PLA General Hospital
Beijing, China
email: royal1860@sina.cn
Abstract—Aiming at the problem of unstable classification of
osteonecrosis of the femoral head, a classification method based
on deep learning is proposed. The 3D U-net is used to segment
the CT data to obtain the complete femoral region. The
Convolutional Autoencoder is used to reduce the dimension and
feature extraction, and the deep information of the segmented
femur is obtained. Finally, K-means is used to cluster the
extracted features and obtain the result. Experiments on 120
cases of patients shows that all of the 4 indexes, including Dice
coefficient, which are used to evaluate the segmentation effect,
are higher than 94%. The peak signal-to-noise ratio is adopted
to evaluate the input and output image of Convolutional
Autoencoder, and the result is higher than 34dB. Then, the
average inter-class similarity and intra-class similarity of the
classification method are 51% and 87%, respectively, indicating
that there are great differences between the types of femurs, and
there is a high similarity within the class. The proposed method
has good clinical application value for individual treatment of
patients and can assist doctors to carry out more targeted
treatment for patients with early osteonecrosis of the femoral
head.
Keywords—Osteonecrosis of Femoral Head, 3D U-net,
Convolutional Autoencoder, K-means Clustering
I. INTRODUCTION
Osteonecrosis of Femoral Head (ONFH) is a common
complication in orthopaedics, which is probably caused by the
death of bone cells, bone marrow components and the
subsequent cell repairing reaction with the damage or
interruption of femoral head blood supply. What’s more
severe, it could lead to joint pain and joint dysfunction of
patients [1]. In addition, lots of other causes could result from
the ONFH, such as hip trauma, application of corticosteroids,
alcoholism, decompression sickness, sickle cell anemia and so
on [1]. If a patient exists the symptom of articular surface
collapse, physician usually has to replace the joint, which is
not only costly but also risky. Hence, it is particularly
significant to research the classification of the early ONFHs
and develop the individualized treatment plans for patients.
In the previous works of ONFH classification, Marcus [2],
Ficat and Arlet [3], Steinberg [4] and the University of
Pennsylvania [5] were all based on X-ray images and clinical
symptoms with some limitations. Currently, a relatively ideal
staging method is the international standard of ONFH
provided by the Association Research Circulation Osseous
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
Shaojie Tang
Shaanxi Key Laboratory of Network Data Analysis and
Intelligence Processing
School of Automation, Xi’an University of Posts &
Telecommunications
Xi’an, China
email: tangshaojie@xupt.edu.cn
(ARCO) in 1992, which includes the performance of
Magnetic Resonance Imaging (MRI) and the assessment of
femoral head involvement degree [6]. There are also other
several ONFH classification methods, such as Steinberg
classification [4], improved Kerboul classification [7],
Japanese Investigation Committee classification (JIC) [8] and
China-Japan Friendship Hospital (CJFH) three pillars
structure classification [9].
ONFH is divided into four stages according to its scope
and the collapse degree of femoral head by using the staging
of ARCO [6]. ONFH is categorized into three groups by using
the Steinberg classification based on the proportion of femoral
head necrosis in total femoral head volume [4], while it is
divided into three categories by the improved Kerboul
classification with the angle of the coronal and sagittal planes
of femoral head [7]. In the JIC classification, femoral head is
separated into three parts on the coronal plane, and ONFH is
grouped into four types according to the extent of necrosis
areas [8]. In addition, the CJFH classification divides the
femoral head into three parts on the coronal plane, and ONFH
has five categories according to the extent of necrosis areas
[9]. Li et al. have observed the natural course of ONFH for
many years, concluding the high accuracy of CJFH
classification in predicting mild or severe ONFH and the low
accuracy on the middle extent of ONFH. As demonstrated in
Ref. [10], the consistency of JIC classification among
different observers shows good performance, while those of
Steinberg classification and improved Kerboul classification
are relatively inferior. Even if the JIC classification
demonstrates accurate results in predicting the ONFH, its
accuracy on the moderate lesions is still low.
The existing classification of osteonecrosis of femoral
head has a certain degree of limitations, mainly depends on
the personal experience of doctors. It is not stable enough, and
the results cannot be detailed to understand the disease and
pathological changes. It also cannot predict the prognosis of
patients accurately enough. However, there are few reports on
the combination of diagnosis of osteonecrosis of femoral head
and big data artificial intelligence.
Aiming at the instability of traditional classification
methods, a new method based on deep learning is proposed in
this paper:
 （1）The 3D U-net network is used to segment the CT
data of the femur in order to obtain the complete information
of the femur and remove the interference of other tissues and
organs in the CT section. In order to solve the problem of
insufficient memory and slow training caused by high
dimensionality of 3-D data, this paper proposes an
improvement in the data processing. The original 512 ×512 ×
128 3-D CT femoral data is randomly cropped into 64 × 64 ×
64 3-D sub-data through the algorithm, and this data is used
for training. This method can effectively reduce the memory
consumption and time during training.
（2）The Convolutional Autocoder is used to extract the
reduced dimension feature of the segmented 3-D femoral
information in order to obtain the deep implicit information of
the femoral region. In this paper, an improvement is proposed
in the coding part. Different convolution kernels are generally
used for convolution operation and then superimposed. This
can obtain refraction fields of different scales and extract
multi-scale information between layers, which is beneficial to
improve the accuracy of the encoding so that the original
information can be better compressed.
（3）We use K-means to cluster the data in order to get
the final classification results of osteonecrosis of the femoral
head.
II. RELATED WORK
A. 3D U-net
Recently, deep learning has developed rapidly in the field
of image processing, and medical image segmentation is one
of the important contents of medical image research.
Ronneberger et al. [11] propose U-Net network, which
combines deep and shallow information through cascade
operation to achieve better segmentation effect. However, U-
Net network can not directly process 3-D medical images and
only cut 3-D images into multiple 2D slices before processing,
which will lead to the loss of relevant information between the
middle layers of 3-D images, resulting in poor segmentation
details. Therefore, Çiçek et al. propose 3D U-net [12], which
can solve this problem and improve the accuracy of
segmentation.
The network structure of 3D U-net is shown in Figure 1.
In the coding part, each layer contains two 3×3×3 convolution
B. Convolutional Autoencoder
The autoencoder is a learning algorithm based on Back
Propagation (BP), which learns the identity function
h w,b ( x)  x to make the targets close to inputs and effectively
encode the data. The input vector x is mapped from the
mapping function f to the hidden layers y , formally:
ni
y = hw,b ( x) = f (W T x + b) = f (Wi xi + b) ()
i =1
ni is the number of Autoencoder layers, W is a weight matrix
with d ' d dimensions, and b is the bias vector,
f ( z) = 1/ (1 + exp(− z)) as the sigmoid activation function. The
objective function for dataset {( x1 , y1） ,...,( xm , y m )} can be
formulated as:
1
 ()
m
J (W , b) = i =1
L( x i , y i )
m
It is demonstrated that Convolutional Neural Networks
have shown excellent performance in the feature extraction of
images. In the experiments, we define the structure of 3-D
Convolutional Autoencoder. After training with the 3-D femur
data segmented by 3D U-net, we could obtain the deep feature
of raw data. As a comparison, this feature can better reflect the
implicit information of femur data with less dimensions.
The 3-D femur data segmented is used as the input of
Convolutional Autoencoder. To ensure the isotropy in images,
3-D femur data are processed as 128×128×128. In the encoder
layers, we take the convolution with 3×3×3 and 5×5×5
convolution kernels respectively, adding them together to
extract the multi-scale information of data. Meanwhile, we use
3×3×3 convolution kernels for the deconvolution layers, with
the stride convolution layers instead of pooling layers in the
decoder parts. In this way, the outputs are as consistent as
possible with the inputs, and we can extract the mid-layer
feature as the deep implicit information of raw data. The
Convolutional Autoencoder network structure is shown as
Figure 2.
Input
128×128×128×1
Output
128×128×128×1
128 × 128 × 128 × 16 128 × 128 × 128 × 16
64 × 64 × 64 × 32 64 × 64 × 64 × 32

operations, and after each convolution, a Rectified Linear 32 × 32 × 32 × 64 32 × 32 × 32 × 64

Unit(ReLU) is included. In the decoding section, each layer
16 × 16 × 16 × 128 16 × 16 × 16 × 128
consists of an upper convolution of 2×2×2. Each dimension
has a step size of 2, followed by two 3×3×3 convolutions, and 8 × 8 × 8 × 256 8 × 8 × 8 × 256

each of which is ReLU after convolution. At the last layer, the 4×4×4×512 4×4×4×512
1×1×1 convolution reduces the number of output channels to
2×2×2×1024 2×2×2×1024
the number of tags. In this paper, we set the number of tags to
2 in order to distinguish the femoral region from the 1 × 1 × 1 × 2048

background area.
Kernel=3×3×3
2 32 64 Strides=2
64+128 64 64 2
Conv
Kernel=3×3×3
Strides=2
Conv
64 128
128+256 128 Kernel=5×5×5
Strides=2
Conv

128 256 256+512 256 concat Fig. 2. Convolutional Autoencoder network structure
conv ( +BN ) + Relu
max pooling
256 up-conv
C. K-means Clustering
conv The clustering analysis divides objects into realistic
groups, which is helpful for further understanding and
Fig. 1. 3D U-net network architecture
description of data. The definition of cluster was first
proposed by Everitt in 1974, which requires that the objects in 2 X ÇY
the group should be relatively compact in space, and the Dice =
X +Y ()
spacing between any two objects in the group should be less
than that between any two objects each in a different group. TP
Recall =
K-means is an unsupervised algorithm based on the prototype. TP + FN
It evaluates the similarity using the distance between objects. ()
There is a greater similarity between closer objects, otherwise TP
a smaller similarity [13]. Precision =
TP + FP
The advantages of K-means clustering are mainly efficient, ()
concise, scalable and effective in the large scale of dataset. In F - measure = 2  Precision 
Recall
our work, K-means clustering is applied on the deep feature Precision + Recall
extracted by Convolutional Autoencoder, so we obtain an ()
unsupervised classification result for the early ONFH.Our where TP is the number of femoral area pixels correctly
experimental results below have shown that there is a great segmented, TN is the number of background pixels correctly
intra-class similarity and a significant inter-class difference. segmented. FP is the number of background pixels incorrectly
segmented into femoral areas, and FN is the number of
III. EXPERIMENT femoral pixels incorrectly labeled as background. X and Y
The femur CT data are provided by the Chinese PLA represent the segmented results by physicians and networks,
General Hospital contain 120 cases, with 128 CT slices respectively. We evaluate the average values for different
include for each case. The data are processed by batch parts of CT slices in Table 1.
desensitization, removal of the basic information of patients
such as name and identification number. The patients are TABLE I. 3D U-NET SEGMENTATION RESULT
45~65 years old. The 10 percent of the total data is randomly Region Dice Precision Recall F-measure
selected as the verification set to explore the best parameters Upper 0.9521 0.9645 0.9624 0.9518
for the model, then select 10 percent of the total data as the Middle 0.9544 0.9436 0.9411 0.9602
test set, while the remaining is used as the training dataset. Lower 0.9602 0.9589 0.9612 0.9601

A. Femoral segmentation
In our experiment, 3D U-net is used to segment the 3-D
CT slices of the femur. We adopt the Adam algorithm for the
gradient descent (β1 = 0.5, β2 = 0.999), with the learning rate
of 0.0002, mini-batch = 1, and the iterations of 300. We
implement our models with TensorFlow, and we train the
models on a NVIDIA-DGX-1V machine equipped with 8
Tesla V100 GPUs, each graphic card with 512 GB memory.
Figure 3 shows the segmentation effect of different parts
of CT slices. Image is the original CT slice, Ground truth is
the manually marked region of the femur, and Result is the
region of the femur segmented by 3D U-net.
As shown in Table 1, the segmentation results of 3D U-net
for the femoral region in CT slices are higher than 0.94 on
Dice, Recall, Precision and F-measure. The experimental
results show that the 3D U-net used in this paper can
effectively segment the femoral region of different slices.
B. Feature extraction
Convolutional Autoencoder is used to reduce dimensions
and extract features of femur. Figure 4 shows the effects of
our model for three femoral CT images.
Input

Upper

Output

Upper Middle Lower

Middle
Fig. 4. The effects of the proposed 3-D Convolutional Autoencoder for
three femoral CT images

We use the Peak Signal-to-Noise Ratio (PSNR) to

evaluate the similarity between input and output of the
Lower Convolutional Autoencoder.
1 m−1 n −1
MSE =   I (i, j ) − K (i, j )
2

mn i =0 j =0
(a)Image (b)Ground Truth (c)Result
()
Fig. 3. 3D U-net segmentation of the femur region results MAX I2
PSNR = 10  log10 ( )
MSE
For the evaluation of 3D U-net performance, four
()
evaluation indexes are used: Dice, Recall, Precision and F-
measure. The specific definitions are as follows: where I and K represent two images with m × n dimensions.
MSE is the mean square error, and MAXI represents the
maximum grayscale value of the image. The average of
PSNRs in all slices for individual cases with some typical D. Result analysis
patients are shown in Table 2. Cosine similarity is used to quantitatively evaluate the
inter-class difference and intra-class similarity in the
TABLE II. THE AVERAGE OF PSNR FOR INDIVIDUAL CASES
clustering results,
Case No. 1 2 3 4
n n n
sim ( x, y ) =  xi yi x y
PSNR(dB) 34.234 35.876 34.621 34.761 2 2 ()
i i
i =1 i =1 i =1
Table 2 indicates that the PSNRs of outputs and inputs are 1 ()
both higher than 34dB by the Convolutional Autoencoder, SIM ( C1 , C2 ) =
N1 N 2
  sim( x, y)
xC1 yC2
proving they are essentially similar. The experimental results
show there is a good feasibility in the feature extraction based where x and y represent the Eigenvectors of two different
on the 3-D Convolutional Autoencoder. images. Ci represents the categories generated by clustering,
and Ni is the number of samples contained in the Ci class.
C. Clustering
In the previous subsections, we accomplish the work of In order to quantitatively calculate the experimental results,
reducing dimensions and extracting features by the we select the slice data of the upper, middle and lower
Convolutional Autoencoder in the 3-D femoral data. segments of the femur to calculate the cosine similarity.
Compared with the raw data, the features can better reflect the Figure 7 shows the cosine similarity of K-means clustering
deep implicit information of data with 2048 dimensions. For results.
clustering, We first select a appropriate k with the Silhouette
c1 c1 0.96
Coefficient (SC),
c2 c2
()
0.88
SC = | a − b | max(a, b)
c3 c3 0.80
where a stands for the mean distance of different samples
c1 c2 c3 c1 c2 c3
within class, and b is the mean distance between a sample Upper Middle
0.72

within class and samples in other class. Figure 5 shows the 0.64
Silhouette Coefficients for different k values. c1 c1
0.56

c2 c2
0.48

c3 c3
0.40

c1 c2 c3 c1 c2 c3
Lower Average
(a) calculation results of inter-class similarity
0.93 0.92
0.91 0.90
0.87 0.85

c1 c2 c3 c1 c2 c3
Fig. 5. Silhouette Coefficients for different k values Upper Middle

The SC value is maximum with k=3. So, we take k=3. 0.88

0.92
0.84 0.85 0.84
0.91

Then, K-means algorithm is used to cluster the features, and

adapte Principal Component Analysis (PCA) to reduce
dimensions for display. The clustering results are shown in
Figure 6.
c1 c2 c3 c1 c2 c3
Lower Average
(b) calculation results of intra-class similarity
Fig. 7. Cosine similarity calculation result

After calculation, the average cosine similarity between

classes is 0.51, which indicates that there is a big difference
between the inter-class data; The average cosine similarity in
the class is 0.87, which indicates that the intra-class data has
high similarity.
E. Comparison and Analysis of different algorithms
Fig. 6. The clustering results of K-means (k=3) In order to verify the effectiveness of the proposed method,
the density-based method DBSCAN and the hierarchical
method BIRCH are used to cluster the femoral data after
image segmentation and feature extraction, and the
corresponding cosine similarity is calculated. The results of
clustering the raw data and performing corresponding
calculations are compared, and the results are shown in Table
3.
TABLE III. COMPARISON OF COSINE SIMILARITY RESULTS OF
DIFFERENT CLUSTERING METHODS
Inter-class Intra-class
Method
similarity similarity
DBSCAN 0.63 0.75
BIRCH 0.61 0.78
K-means on raw data 0.74 0.77
Proposed Method 0.51 0.87
As can be seen from Table 3, compared with other
algorithms, the proposed method is superior to other
algorithms in inter-class similarity and intra-class similarity of
femoral data; The final classification results can be greatly
optimized by segmentation and feature extraction of femoral
data. Figure 8 shows the classification difference of the femur
obtained by proposed method:
In the first class, the bone distribution is more uniform, and
the bone is relatively full, with no obvious black area;
ln the second class, the bone distribution is relatively
uniform, with part of the black area;
In the third class, the bone distribution is uneven. There is
an obvious black area, and the boundary with the white area is
obvious.
(a) Class1 (b) Class 2 (c) Class 3
Fig. 8. Final classification result map
IV. DISCUSSIONS AND CONCLUSION
In this paper, we propose an automatic approach for
classification of the early ONFH with deep learning. First of
all, the respective 3D U-net is used to sequentially segment
the femoral areas with CT data and the experimental results
prove that there is an accuracy higher than 94% with the
proposed approach. The segmentation method can obtain the
regional details of femur data accurately. Secondly, we
propose a 3-D Convolutional Autoencoder to extract the deep
features of femur, so a drastic reduction in dimensions is
achieved from 128×128×128 to 2048. PSNR is employed for
evaluating the similarity of inputs and outputs, and the result
is higher than 34dB. Finally, through K-means algorithm, we
cluster the deep features extracted into three groups, and the
results demonstrate there is a great intra-class similarity and a
significant inter-class difference. The proposed approach has
a valuable clinical merit in classification of the early ONFH,
and hopefully it can assist physicians to apply more
individualized treatment to patients.
Until present, the application of AI for the auxiliary
diagnosis of ONFH is still in the primary stage, and the work
reported in this paper is a preliminary exploration in this field.
Traditional classifications rely much more on the experiences
of physicians, while the approach proposed in this paper is
based on deep learning, which is relatively objective as
compared to those traditional classifications. Currently, we are
carrying out a comparison research with clinical classification,
and the results will be reported in the near future.
ACKNOWLEDGMENT
This work was supported in part by the project for the
innovation and entrepreneurship in Xi’an University of Posts
and Telecommunications, China (2018SC-03), by the Key
Lab of Computer Networks and Information Integration
(Southeastern University), Ministry of Education, China
(K93-9-2017-03), by the Department of Education Shaanxi
Province, China (15JK1673), and by Shaanxi Provincial
Natural Science Foundation of China (2016JM8034).
REFERENCES
[1] Sugano, Nobuhiko, et al. "Prognostication of nontraumatic avascular
necrosis of the femoral head. Significance of location and size of the
necrotic lesion." Clinical orthopaedics and related research 303 (1994):
155-164.
[2] Marcus, Neal D., W. F. Enneking, and Robert A. Massam. "The silent
hip in idiopathic aseptic necrosis: treatment by bone-grafting." JBJS
55.7 (1973): 1351-1366.
[3] Ficat, R. P. "Idiopathic bone necrosis of the femoral head. Early
diagnosis and treatment." The Journal of bone and joint surgery. British
volume 67.1 (1985): 3-9.
[4] Steinberg M E, Hayken G D, Steinberg D R. A quantitative system for
staging avascular necrosis. The Journal of Bone and Joint Surgery.
British volume, 1995, 77-B (1): 34-41.
[5] Steinberg, David R., and Marvin E. Steinberg. "The University of
Pennsylvania classification of osteonecrosis." Osteonecrosis. Springer,
Berlin, Heidelberg, 2014. 201-206.
[6] Gardeniers, J. W. "A new international classification of osteonecrosis
of the ARCO-committee on terminology and classification." ARCO
news 4 (1992): 41-46.
[7] Kerboul, M., et al. "The conservative surgical treatment of idiopathic
aseptic necrosis of the femoral head." The Journal of bone and joint
surgery. British volume 56.2 (1974): 291-296.
[8] Ono, K. "Diagnostic criteria, staging system, and roentgenographic
classification of avascular necrosis of the femoral head (steroid induced,
alcohol associated, or idiopathic nature)." Annual report of Japanese
investigation committee for intractable diseases, avascular necrosis of
the femoral head, under the auspices of Ministry of Health and Welfare
331 (1987).
[9] Zirong, L. I., et al. "The classification of osteonecrosis of the femoral
head based on the three pillars structure: China Japan Friendship
Hospital (CJFH) classification." Chinese Journal of Orthopaedics 32.6
(2012): 515-520.
[10] Takashima, Kazuma, et al. "Which classification system is most useful
for classifying osteonecrosis of the femoral head?." Clinical
Orthopaedics and Related Research 476.6 (2018): 1240-1249.
[11] Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. "U-net:
Convolutional networks for biomedical image segmentation."
International Conference on Medical image computing and computer-
assisted intervention. Springer, Cham, 2015.
[12] Çiçek, Özgün, et al. "3D U-Net: learning dense volumetric
segmentation from sparse annotation." International conference on
medical image computing and computer-assisted intervention.
Springer, Cham, 2016.
[13] Zhang, Tian, Changchuan Yin, and Lin Pan. "Improved clustering and
association rules mining for university student course scores." 2017
12th International Conference on Intelligent Systems and Knowledge
Engineering (ISKE). IEEE, 2017.