KOA Using Radiographic Images 1

INTRODUCTION

About Machine Learning :

The concept of classical programming is that an engineer defines a set of rules, called an
algorithm, as shown in Figure, which uses input data to calculate some form of output data.

Classical Programming Pipeline

A machine learning algorithm is an algorithm that can learn from data as shown in Figure.

It can be used to calculate these rules automatically, so they do not have to be specified by
hand. Three components are needed for such an approach:

 Input data the algorithm is supposed to transform.

 Output data the algorithm is supposed to predict.

 A measurement to validate the performance of a prediction.

Machine Learning Pipeline

It works by feeding input and output data into a pipeline, which will learn to transform one
into the other. With the advantage that no explicit programming is needed to generate the
rules, comes the disadvantage that prior input and output.
data is required for the initial learningprocess. There are different subparts to machine learning
like supervised and unsupervised learning. Supervised learning is used when it is clear what
the output data looks like, whereasunsupervised learning can help to find unknown patterns
in the data.

Dept.of MCA VTU CIT

KOA Using Radiographic Images 2

About Knee Osteoarthritis:

Knee osteoarthritis (KOA) is a leading cause of disability among elderly adults,and it causes
pain and discomfort and limits the functional independence of such adults. Knee Osteoarthritis
is a degenerative disease of the knee joint and the most common form of arthritis causing
pain, mobility limitation, affecting independence and quality of life in millions of people.

There is no known cure for ΚΟА, but there are several medical, biological and environmental
risk factors, both modifiable and non- modifiable, that are known to be involved in the
development and progression of the disease. The aim of this study was the development of
an automated classification model for KOA, based on the Kellgren–Lawrence (KL) grading
system, using radiographic images.

There are two types of Osteoarthritis (OA), Primary OA seen in aged people due to genetic
reasons or aging. Secondary OA tends to show up earlier in life due to some injury, diabetes,
obesity, athletics or patients with rheumatoid arthritis. The sample of normal and affected
Osteoarthritis knee image as shown in Figure.

Knee osteoarthritis (OA) is a leading cause of disability in older adults, with no effective
treatments currently available and dissatisfaction rates of nearly 20% in patients who undergo
joint replacement surgery. To facilitate the development of treatments, there is a need to make
disease staging more efficient. Several methods currently exist for OA staging.
Radiographically derived evaluations, particularly joint space narrowing measurements and
the Kellgren-Lawrence (KL) scoring system are some of the most common.

Dept.of MCA VTU CIT

KOA Using Radiographic Images 3

Evaluations derived from MRI, suchas cartilage volume, morphology, and T2 mapping, have
also been shown to provide sensitive measurements of OA worsening. The KL scoring system
is particularly valuable in research, where it is often used to define study cohorts and to
evaluate how theradiographic manifestation of OA relates to clinical outcomes. For example,
previous studies have shown that presurgical KL grade is predictive of surgical success.

However, interclinician agreement and interclinician repeatability are not optimal. More
accurate, consistent diagnosis of OA stage would ensure that investigative treatments are
evaluated in patients in the severity range intended by the investigators. Furthermore, a time-
and cost-efficient approach would accelerate clinical trials, which are often slowed by their
relianceon experts to first screen large populations with radiography to identify patients with
the appropriate level of OA osteoarthritis severity to be included. In clinical practice, having
aconsistent, automated mechanism for evaluating sequential radiographic examinations of
individual patients would enable better tracking of their disease progression.

The current gold standard for the radiographic assessment of KOA is the Kellgren– Lawrence
(KL) grading scale. The scale ranges from 0 to 4. A score of 0 indicatesthat there is no evidence
of osteoarthritis (OA; green); a score of 1 indicates the possibilityof joint space narrowing
(orange) and osteophyte formation (blue); a score of 2 indicates definite osteophyte formation
and possible joint space narrowing; a score of 3 indicates multiple osteophytes, definite joint
space narrowing, sclerosis (purple), and possibly bone deformity (pink); a score of 4 indicates
end-stage OA, marked by severe sclerosis, joint space narrowing (sometimes bone-on-bone
contact), and large osteophytes as shown in Figure.

Dept.of MCA VTU CIT

KOA Using Radiographic Images 4

REQUIREMENT ANALYSIS

Proposed System:
In this project, we tackled the problem of automating the process of grading individual
osteoarthritis features in knee plain radiographs. Our method is based on Deep Learning and
simultaneously predicts the Kellgren Lawrence grades as well as the grades for osteophytes
and joint space narrowing compartment-wise according to the Osteoarthritis Research
Society International (OARSI) grading atlas. In particular, we used transfer learning from
ImageNet and trained an ensemble of neural networks to achieve the final results.

Hardware Requirements:
This project requires a hardware requirement of a processor of speed 333MHz, 32MB or
above RAM capacity, a graphic card, a keyboard and a min 100kb of hard disk space.

Requirements in detail:
 System : Intel IV 2.4 GHz.

 Ram : 4GB

 Hard Disk : 500GB

Software Requirements:
A software requirements specification (SRS), requirements specifications for a software
system, is a description of the behavior of a system to be developed and may include a set of
use cases that describe interactions the users will have with the software. In addition, Non-
functional requirements impose constraints on the design or implementation.

Operating system : Windows 7 / 8 / 10

Language Used : Python 3.7 and

Above IDE : Spyder

Libraries Used : tensorflow, pandas, Keras, MatPlot, tkinder.

Dept.of MCA VTU CIT

KOA Using Radiographic Images 5

TensorFlow:
TensorFlow is a free and open-source software library for machine learning and artificial
intelligence. It can be used across a range of tasks but has a particular focus on deep neural
networks. TensorFlow is a rich system for managing all aspects of a machine learning
system; however, this class focuses on using a particular TensorFlow API to develop and
train machine learning models. TensorFlow was developed by the Google Brain team for
internal Google use in research and production. The initial version was released under the
Apache License 2.0.
Google released the updated version of TensorFlow, named TensorFlow 2.0. TensorFlow
can be used in a wide variety of programming languages, including Python, JavaScript, C++,
and Java. The name TensorFlow derives from the operations that such neural networks
perform on multidimensional data arrays, which are referred to as tensors.

Keras:
Keras is an open-source software library, source high-level Neural Network library that
provides a Python interface for artificial neural networks. Keras acts as an interface for the
TensorFlow library. Keras is written in Python is capable enough to run on Theano,
TensorFlow. It was developed by one of the Google engineers. It is made user-friendly,
extensible, and modular for facilitating faster experimentation with deep neural networks.It
supports both Convolutional Networks and Recurrent Networks individually but also their
combination.

Matplotlib:
Matplotlib is an amazing visualization library in Python for 2D plots of arrays. Matplotlib is
a multi-platform data visualization library built on Numpy arrays. One of the greatest benefits
of visualization is that it allows us visual access to huge amounts of data in easily digestible
visuals. Matplotlib consists of several plots like line, bar, scatter, histogram etc.
Tkinter:
Python offers multiple options for developing GUI (Graphical User Interface). Out of all the
GUI methods, tkinter is the most commonly used method. It is a standard Python interface to
the Tk GUI toolkit shipped with Python. Python with tkinter is the fastest andeasiest way to
create the GUI applications. Creating a GUI using tkinter is an easy task. Tocreate a tkinter
app: Importing the module – tkinter, Create the main window, add any number of widgets to
the main window, Apply the event Trigger on the widgets.

Dept.of MCA VTU CIT

KOA Using Radiographic Images 6

Pandas:
Pandas is a software library written for the Python programming language for data
manipulation and analysis. Pandas is mainly used for data analysis and associated manipulation
of tabular data in Data Frames. Pandas allows importing data from various fileformats such as
comma-separated values, JSON, Parquet, SQL database tables or queries,and Microsoft Excel.
Pandas allows various data manipulation operations such as merging,reshaping, selecting, as
well as data cleaning, and data wrangling features. The Pandas library is built upon another
library Numpy, which is oriented to efficiently working with arrays instead of the features of
working on Data Frames.

Dept.of MCA VTU CIT

KOA Using Radiographic Images 7

CODING:
import tkinter as tk
from tkinter import messagebox
from PIL import Image, ImageTk # pip install pillow
from tkinter.filedialog import askopenfilename

class FirstPage(tk.Frame):
def __init__(self, parent, controller):
tk.Frame.__init__(self, parent)

load = Image.open("img1.jpg")
photo = ImageTk.PhotoImage(load)
label = tk.Label(self, image=photo)
label.image=photo
label.place(x=0,y=0)

border = tk.LabelFrame(self, text='Login', bg='ivory', bd = 10, font=("Arial", 20))

border.pack(fill="both", expand="yes", padx = 150, pady=150)

L1 = tk.Label(border, text="Username", font=("Arial Bold", 15), bg='ivory')

L1.place(x=50, y=20)
T1 = tk.Entry(border, width = 30, bd = 5)
T1.place(x=180, y=20)

L2 = tk.Label(border, text="Password", font=("Arial Bold", 15), bg='ivory')

L2.place(x=50, y=80)
T2 = tk.Entry(border, width = 30, show='*', bd = 5)
T2.place(x=180, y=80)

def verify():
try:
with open("credential.txt", "r") as f:
info = f.readlines()

Dept.of MCA VTU CIT

KOA Using Radiographic Images 8

i =0
for e in info:
u, p =e.split(",")
if u.strip() == T1.get() and p.strip() == T2.get():
controller.show_frame(SecondPage)
i=1
break
if i==0:
messagebox.showinfo("Error", "Please provide correct username and
password!!")
except:
messagebox.showinfo("Error", "Please provide correct username and password!!")

B1 = tk.Button(border, text="Submit", font=("Arial", 15), command=verify)

B1.place(x=320, y=115)

def register():
window = tk.Tk()
window.resizable(0,0)
window.configure(bg="pink")
window.title("Register")
l1 = tk.Label(window, text="Username:", font=("Arial",15), bg="white")
l1.place(x=10, y=10)
t1 = tk.Entry(window, width=30, bd=5)
t1.place(x = 200, y=10)

l2 = tk.Label(window, text="Password:", font=("Arial",15), bg="white")

l2.place(x=10, y=60)
t2 = tk.Entry(window, width=30, show="*", bd=5)
t2.place(x = 200, y=60)

l3 = tk.Label(window, text="Confirm Password:", font=("Arial",15), bg="white")

l3.place(x=10, y=110)
t3 = tk.Entry(window, width=30, show="*", bd=5)

Dept.of MCA VTU CIT

KOA Using Radiographic Images 9

t3.place(x = 200, y=110)

def check():
if t1.get()!="" or t2.get()!="" or t3.get()!="":
if t2.get()==t3.get():
with open("credential.txt", "a") as f:
f.write(t1.get()+","+t2.get()+"\n")
messagebox.showinfo("Welcome","You are registered successfully!!")
else:
messagebox.showinfo("Error","Your password didn't get match!!")
else:
messagebox.showinfo("Error", "Please fill the complete field!!")

b1 = tk.Button(window, text="Sign in", font=("Arial",15), bg="#ffc22a",

command=check)
b1.place(x=170, y=150)

window.geometry("470x220")
window.mainloop()

B2 = tk.Button(self, text="Register", bg = "dark orange", font=("Arial",15),

command=register)
B2.place(x=1000, y=20)

class SecondPage(tk.Frame):
def __init__(self, parent, controller):
tk.Frame.__init__(self, parent)

load = Image.open("img2.jpg")
photo = ImageTk.PhotoImage(load)
label = tk.Label(self, image=photo)
label.image=photo
label.place(x=0,y=0)

Dept.of MCA VTU CIT

KOA Using Radiographic Images 10

l1 = tk.Label(self, text="Patient ID:", font=("Arial",15), bg="deep sky blue")

l1.place(x=880, y=300)
t1 = tk.Entry(self, width=30, bd=5)
t1.place(x = 1050, y=300)

l2 = tk.Label(self, text="Patient Name:", font=("Arial",15), bg="deep sky blue")

l2.place(x=880, y=350)
t2 = tk.Entry(self, width=30, bd=5)
t2.place(x = 1050, y=350)

l3 = tk.Label(self, text="Patient Age:", font=("Arial",15), bg="deep sky blue")

l3.place(x=880, y=400)
t3 = tk.Entry(self, width=30, bd=5)
t3.place(x = 1050, y=400)

l4 = tk.Label(self, text="Patient Gender:", font=("Arial",15), bg="deep sky blue")

l4.place(x=880, y=450)
t4 = tk.Entry(self, width=30, bd=5)
t4.place(x = 1050, y=450)

Button = tk.Button(self, text="Submit",bg = "dark orange", font=("Arial", 15),

command=lambda: controller.show_frame(ThirdPage))
Button.place(x=1100, y=550)

Button = tk.Button(self, text="Next",bg = "dark orange", font=("Arial", 15),

command=lambda: controller.show_frame(ThirdPage))
Button.place(x=1100, y=50)

Button = tk.Button(self, text="Back",bg = "dark orange", font=("Arial", 15),

command=lambda: controller.show_frame(FirstPage))
Button.place(x=50, y=50)

class ThirdPage(tk.Frame):
def __init__(self, parent, controller):

Dept.of MCA VTU CIT

KOA Using Radiographic Images 11

tk.Frame.__init__(self, parent)

"""load = Image.open("image3.png")
photo = ImageTk.PhotoImage(load)
label = tk.Label(self, image=photo)
label.image=photo
label.place(x=0,y=0)"""

self.configure(bg='skyblue')

#Label = tk.Label(self, text="Store some content related to your \n project or what your
application made for. \n All the best!!", bg = "orange", font=("Arial Bold", 25))
#Label.place(x=40, y=150)
w2 = tk.Label(self, text="Knee Osteoarthritis Detection", bg="orange" ,fg="white"
,width=30 ,height=1,font=('times', 30, 'italic bold underline'))
w2.place(x=350,y=10)

'''
l=Button(self,text="Build Training Model", command=self.buildModel, bg="red"
,fg="white" ,width=20 ,height=1,font=('times', 20, 'italic bold underline'))
l.place(x=200,y=200)
'''
k=tk.Button(self,text="Browse For Input Image", command=self.showImgg, bg="blue"
,fg="white" ,width=20 ,height=1,font=('times', 20, 'italic bold underline'))
k.place(x=800,y=200)

t=tk.Button(self,text="Classify Test Image", command=self.Classify, bg="blue"

,fg="white" ,width=20 ,height=1,font=('times', 20, 'italic bold underline'))
t.place(x=800,y=300)

Button = tk.Button(self, text="Home",bg = "pink", font=("Arial", 15), command=lambda:

controller.show_frame(FirstPage))
Button.place(x=1100, y=50)

Dept.of MCA VTU CIT

KOA Using Radiographic Images 12

Button = tk.Button(self, text="Back",bg = "pink", font=("Arial", 15), command=lambda:

controller.show_frame(SecondPage))
Button.place(x=50, y=50)

def Classify(self):
print("Classify the Test Image")
#from tensorflow.keras.models import Model
from tensorflow.keras.models import load_model
#from keras.models import load_model
model = load_model('knee_model_for_2000_epochs.h5')

#Compiling the model

model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics =
['accuracy'])
#(model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics =
['accuracy']))
#Making New Prediction
import numpy as np
from tensorflow.keras.preprocessing import image

test_image = image.load_img(self.load,target_size = (64,64,3))

test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image,axis = 0)
result = model.predict(test_image)
print(result)
print(result[0][0])

if result[0][0] == 1.0:
a = "KL Grade 0(Normal):No evidence of osteoarthritis"
print("KL Grade 0(Normal):No evidence of osteoarthritis")

elif result[0][1] == 1.0:

a = "KL Grade 1(Doubtful):Minute Osteopytes"
print("KL Grade 1(Doubtful):Minute Osteopytes")

Dept.of MCA VTU CIT

KOA Using Radiographic Images 13

elif result[0][2] == 1.0:

a = "KL Grade 2(Mild):Definite osteophyte, normal joint space"
print("KL Grade 2(Mild):Definite osteophyte normal joint space")
elif result[0][3] == 1.0:
a = "KL Grade 3(Moderate):Multiple osteophytes, Moderate joint space reduction"
print("KL Grade 3(Moderate):Multiple osteophytes, Moderate joint space reduction")
elif result[0][4] == 1.0:
a = "KL Grade 4(Severe):End-stage OA, Severe Sclerosis, Joint space greatly reduced"
print("KL Grade 4(Severe):End-stage OA, Severe Sclerosis, Joint space greatly
reduced")

s = tk.Label(self, text=a, bg="white" ,fg="black", font=('arial',12))

s.place(x=650,y=500)

def showImgg(self):
self.load = askopenfilename(filetypes=[("Image File",'.png')])

im = Image.open(self.load)

render = ImageTk.PhotoImage(im)

# labels can be text or images

img = tk.Label(self, image=render,width=500,height=500)
img.image = render
img.place(x=100, y=100)

class Application(tk.Tk):
def __init__(self, *args, **kwargs):
tk.Tk.__init__(self, *args, **kwargs)

Dept.of MCA VTU CIT

KOA Using Radiographic Images 14

#creating a window
window = tk.Frame(self)
window.pack()

window.grid_rowconfigure(0, minsize = 800)

window.grid_columnconfigure(0, minsize = 1300)

self.frames = {}
for F in (FirstPage, SecondPage, ThirdPage):
frame = F(window, self)
self.frames[F] = frame
frame.grid(row = 0, column=0, sticky="nsew")

self.show_frame(FirstPage)

def show_frame(self, page):

frame = self.frames[page]
frame.tkraise()
self.title("Application")

app = Application()
app.maxsize(1300,800)
app.mainloop()

Dept.of MCA VTU CIT

KOA Using Radiographic Images 15

RESULTS:
 Login Page of the Knee Osteoarthritis application as shown in Fig.

Login Page

 The register page contains username, password and confirm password which will be
useful for the login to the application as shown in Fig 6.2. When we click on the register
button placed at right side of the page the register page gets appear for the user.

Register Page

Dept.of MCA VTU CIT

KOA Using Radiographic Images 16

 Patient Details page which contains their basic details which should be entered intheir
respective fields as shown in Fig.

Patient Details Page

 Adding knee images using the button and classifying them by Kellgren- Lawrence
Grading Scale as shown in Fig.

GUI for Classifying the knee data

Dept.of MCA VTU CIT

KOA Using Radiographic Images 17

 Adding knee images and classified as KL Grade as shown in Fig.

Classified as KL Grade

Dept.of MCA VTU CIT

KOA Using Radiographic Images 18

ADVANTAGES AND DISADVANTAGES:

‘‘Automatic knee osteoarthritis diagnosis from plain radiographs: A deep

learning-basedapproach,’’

This paper proposes a Automatic knee osteoarthritis diagnosis from plain radiographs. In this
study, we present a new transparent computer-aided diagnosis method based on the Deep
SiameseConvolutional Neural Network to automatically score knee OA severity according to
the Kellgren-Lawrence grading scale. We trained our method using the data solely from the
Multicenter Osteoarthritis Study and validated it on randomly selected 3,000 subjects (5,960
knees) from Osteoarthritis Initiative dataset. We believe that our model is useful for clinical
decision making and for OA research.

Advantages:
It presents new state-of-the art results in automated knee OA diagnostics from plain
radiographs outperforming the existing approaches. Mainly using a new approach for utilizing
Siamese deep neural networks for medical images with symmetry, which significantly reduces
thenumber of learnable parameters, thus making model more robust and less sensitive to noise.
This method releases a standardized dataset for knee X-ray OA diagnosis algorithms.

Disadvantages:
Various infections involves the skin, muscles, or artificial joint. Often times the artificial
joint must be surgically removed and replaced at a later time after antibiotics.
Finally, we can't detect this type of infections in this method while transferable image
representation by performing an evaluation on a dataset.

Dept.of MCA VTU CIT

KOA Using Radiographic Images 19

FUTURE SCOPE:
During the development of this study, new lines of work have been identified that relate to
improving the performance of the knee OA severity assessment as well as to serving as the
foundation for a supporting diagnosis tool.

The main goal is to develop computer aided diagnosis tools that assist practitioners in
assessing knee OA severity. This study has set the foundation to that purpose by defining
a workflow and a methodology that allow to localize and extract a knee joint from X-ray
images,and to perform a prediction of disease severity. Further steps would consist of
combining all individual steps into a single pipeline, i.e. image pre-processing, knee
location and extraction, disease severity prediction and visualization. The resulting
workflow can be contained into anAPI hosted on a cloud-based website. This would allow
non-technical users to benefit from themodel’s prediction capability by uploading a single
image.

Dept.of MCA VTU CIT

KOA Using Radiographic Images 20

CONCLUSION:
It was very interesting and adopting learning kind of experience to work with medical images
in this project as it is playing a very big role in the medical field. This project took us through
the various phases of project development and gave us the real insight into the world of machine
learningtechniques.
The joy of working and the thrill involve while tackling the various problems and challenges
gave us a feel of developer’s industry. It was due to this project we came to know howdifferent
machine learning techniques and methods can be designed in order to achieve a state of art
solution to the problems like image classification and object detection.
The proposed model basedon CNN and radiographic images was demonstrated to improve the
accuracy of diagnosing the severity of KOA using the KL grading system. The automatic
classification of KOA using the proposed method can reduce the work of the clinician and
improve the reliability of the KL gradingsystem.

This Master’s thesis focuses on robust methods used for localization of knee joints out of
a X- ray image and prediction of knee Osteoarthritis diagnosis. The main goal is to
develop computer aided diagnostic tools to assist professionals on the assessment of knee
Osteoarthritis (OA) severity by developing deep learning based automatic methods.

Dept.of MCA VTU CIT