Deeksha Final Report 1

VISVESVARAYA TECHNOLOGICAL UNIVERSITY
BELAGAUM, KARNATAKA-590014
A Project report on
“A Deep Learning Approach for Liver and Tumor

Segmentation.”
Submitted in Partial fulfilment of the requirements for the 8th semester
BACHELOR OF ENGINEERING
IN
COMPUTER SCIENCE AND ENGINEERING

By
BAVITHA VYAS 1SP18CS012
BIBEK K SAH 1SP19CS016
DEEKSHITHA R 1SP19CS023
MUSKAAN KUMARI 1SP19CS060
Under the guidance

of:
Prof. SOUJANYA C N
Asst. Professor Dept of CSE
Department of Computer Science and Engineering
S.E.A. COLLEGE OF ENGINEERING AND TECHNOLOGY

BENGALURU-560049
2022-2023
S.E.A COLLEGE OF ENGINEERING AND TECHNOLOGY
Ekta Nagar, Basavanpura, Virgonagar Post, K. R. Puram, Bengaluru, Karnataka 560059
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
CERTIFICATE
This is to certify the project work entitled “A Deep Learning Approach for Liver and
Tumor Segmentation” has been successfully carried out by Ms. Muskaan

Kumari(1SP19CS060), Ms. Deekshitha R(1SP19CS023), Ms. Bavitha Vyas (1SP18CS012),
Bibek K Sah(1SP19CS016) of VIII Sem in partial fulfilment for the award of Bachelor of
Engineering in Computer Science and Technology of the Visvesvaraya Technological University,
Belgaum during the year 2022.
The project report has been approved as it satisfied the academic requirement in respect of the
Project work prescribed for the Bachelor of Engineering.
Signature of Guide Signature of HOD Signature of Principal

Prof. Soujanya C N Dr. B. Loganayagi Dr. B Venkata Narayana
Signature of Internal Examiner Signature of External Examiner
1 1
2 2
DECLARATION
We, the students of final semester of Computer Science and Engineering, S.E.A. COLLEGE OF
ENGINEERING & TECHNOLOGY, BANGALORE – 49, declare that the work entitled "A
DEEP LEARNING APPROACH FOR LIVER AND TUMOR SEGMENTATION " has been
worked under the guidance of Prof Soujanya Computer Science Department, S.E.A. COLLEGE
OF ENGINEERING & TECHNOLOGY, BANGALORE. This dissertation work is submitted to
Visvesvaraya Technological University in partial fulfillment of the requirements for the award of
Degree of Bachelor of Engineering in Computer Science and Engineering during the academic year
2022 - 2023. Further the matter embodied in the project report has not been submitted previously
by anybody for the award of any degree or diploma to any university.
Place:
Date:
Team Members USN SIGNATURE
BAVITHA VYAS 1SP18CS012
BIBEK K SAH 1SP19CS016
DEEKSHITHA R 1SP19CS023
MUSKAAN KUMARI 1SP19CS060

ACKNOWLEDGEMENT
Firstly, I thank the Management late Shri A Krishnappa, Chairman SEA College of
Engineering and Technology for Providing the Necessary infrastructure and creating a good
environment.
We would like to thank to our respected Dr.Bhagavant K Deshpande, Director of SEA

College Of Engineering and Technology for the encouragement and support given by him.
I would like to express my profound thanks to our respected principal Dr. B Venkata Narayana
For the encouragement and support given by him.
I would like to express my sincere thanks to our respected Dr. B. Loganayagi, HOD of
COMPUTER SCIENCE AND ENGINEERING department, for his assistance and guidance.
I am thankful for the support rendered by our Project guide and coordinator
Prof. Soujanya C N for her valuable suggestions.
I am also obliged, to the faculty members of CSE Department who rendered their valuable
assistance for the Project.
And finally, I would like to express my heart full gratitude to my friends and all those who have
extended their help throughout 0ur Project.
ABSTRACT
A fully automatic technique for segmenting the liver and localizing its unhealthy tissues is a convenient tool in
order to diagnose hepatic diseases and assess the response to the according treatments. In this work we propose a
method to segment the liver and its lesions from Computed Tomography (CT) scans using Convolutional Neural
Networks (CNNs), that have proven good results in a variety of computer vision tasks, including medical
imaging. The network that segments the lesions consists of a cascaded architecture, which first focuses on the
region of the liver in order to segment the lesions on it. Moreover, we train a detector to localize the lesions, and
mask the results of the segmentation network with the positive detections.
The segmentation architecture is based on DRIU, a Fully Convolutional Network (FCN) with side outputs that
work on feature maps of different resolutions, to finally benefit from the multi-scale information learned by
different stages of the network. The main contribution of this work is the use of a detector to localize the lesions,
which we show to be beneficial to remove false positivestriggered by the segmentation network.
INDEX
SL NO CONTENTS PAGE NO
INTRODUCTION
1 1.1 PROBLEM STATEMENT 2
1.2 EXISTING SYSTEM 3
1.3 CHALLENGES 3
1.4 OBJECTIVE 3
1.5 PROPOSED SYSTEM 4
2 LITERATURE SURVEY 5-8
SYSTEM DESIGN
3 3.1 DATA FLOW DIAGRAM 9
3.2 METHODS USED 10
3.3 IMAGE PROCESSING 11
PROJECT METHODOLOGY
4 4.1 METHODOLOGY 13
4.2 MODULES 14
4.3 ALGORITHMS 15
5 SYSTEM ARCHITECTURE 16
6 SYSTEM REQUIREMENTS
6.1 Hardware Requirements 17
6.2 Software Requirements 18
6.3 Functional requirements 19
6.4 Non Functional Requirements 19
6.5 Resource Requirements 20
7 SOURCE CODE
7.1 Dataset Connectivity 21-27
7.2 Code Deploy 27-32
8 SNAPSHOTS 33-35
9 CONCLUSION 36
REFERENCES 37
FIGURE INDEX
SL.NO Name of the figure Page
3.1 Dataflow Diagram 1 9
3.2 Methods used 9
3.3 Image processing 11
3.4 Activity Diagram 12
4.1 Project Methodology 12
4.3 Algorithms 13
5.1 Segments 15
5.4 Anaconda Installation 20
5.7 Jupyter Notebook 20
5.8 Pycharm IDE 20

A Deep Learning Approach for Liver and Tumor Segmentation
CHAPTER 1
INTRODUCTION
Segmenting the liver and its lesions on medical images helps oncologists to accurately diagnose liver cancer, as
well as to assess the treatment response of patients. Typically, doctors rely on manual segmentation techniques in
order to interpret the Contrast Tomography (CT) and Magnetic Resonance Imaging (MRI) images. Automatic tools
that are not as subjective and time-consuming have been widely studied in the recent years. Liver lesion
segmentation is a challenging task due to the low contrast between liver, lesions, and also nearby organs. Other
additional difficulties are the lesion size variability and the noise in CT scans. Building a robust system that is
able to beat these difficulties is still an open problem. Recently, methods based on deep Convolutional Neural
Networks (CNNs) have demonstrated to be robust to these challenges, and have achieved the stateof the art at this
task. In this project we adapt DRIU for the task of segmenting both the liver and its lesions from CT scans. DRIU
is a Fully Convolutional Network (FCN) that has side outputs with supervision at different convolutional stages.
This architecture has proven to be successful for the medical task of segmenting the blood vesselsand optical disk
of eye fundus images, as well as for video object segmentation in generic videos. The core of our network for
lesion and liver segmentation consists in using the strength of a segmentation network plus a detection network to
localize the lesions. For training all the networks we used the Liver Tumor Segmentation (LiTS) dataset, which is
composed of 131 CT scans for training and 70 for testing.
s Dept. of CSE, SEACET 2022-23 Page 1

1.1 PROBLEM STATEMENT:
The liver is the second largest organ in the body, located on the right side of the abdomen, and weighs about
three pounds. The liver has two lobes, right and left, and is in contact with the gallbladder, pancreas, and
intestines. Several organs are involved with the liver. Cancer in the liver may be primary (originating from
various cells that compose the liver), secondary, or metastatic (caused by cancerous cells from other organs).
Malignant hepatocellular carcinoma (HCC) is the most typical primary liver disease among all liver cancers.
To identify the formation and texture of the liver, radiologists and oncologists use a computed tomography (CT)
or magnetic resonance imaging (MRI). In both primary and secondary hepatic tumor cancer, these abnormalities
are significant biomarkers for early disease diagnosis, and progression.

1.2 EXISTING SYSTEM:
1. Existing work develop a fully automated system for segmenting liver and tumors from CT scan images in
a single run.
2. Based on prior studies and their shortcomings, the researchers in this study attempt to achieve 95%
mIOU on HCC tumors using VGG and Inception V4 based on the deep learning models. The research
technique is intended to improve accuracy and fulfillexpectations in the segmentation of liver tumors.
3. Existing done by proposing a viable method for classifying liver and tumor cells after failing to achieve the
desired results with the UNet model. Then, we develop a model that combines both ResNet and UNet,
named ResUNet. This deep neural network model utilizes leftover patterns that use escape rather than
simple convolutions, resulting in faster testing with fewdetails.

1.3 PROPOSED SYTSEM:
In this work we propose a method to segment the liver and its lesions from Computed Tomography(CT) scans using
Convolutional Neural Networks (CNNs), that have proven good results in a variety of computer vision tasks,
including medical imaging. The network that segments the lesions consists of a cascaded architecture, which
first focuses on the region of the liver in order to segment the lesions on it. Moreover, we train a detector to
localize the lesions, and mask the results of the segmentation network with the positive detections.
The segmentation architecture is based on DRIU, a Fully Convolutional Network (FCN) with side outputs that
work on feature maps of different resolutions, to finally benefit from the multi-scale information learned by
different stages of the network. The main contribution of this work is the use of a detector to localize the lesions,
which we show to be beneficial to remove false positivestriggered by the segmentation network.
Fig: Proposed Architecture

DRAWBACKS OF EXISTING SYSYTEM:
 Existing work provide a low performance summary of this technique, with comparison to afew other fully
automated techniques and define a scope for development based on new data and other features.
 It provides insufficient result due to which we have proposed a system.

CHAPTER 2
LITERATURE SURVEY
1. INTRODUCTION
A literature review is a type of academic writing that provides an overview of existing
knowledge in a particular field of research. A good literature review summarises, analyses,
evaluates and synthesises the relevant literature within a particular field of research.
2. Related Work:
Abstract: Liver segmentation from medical imaging has progressed significantly in medical practice. The
objective is to extract knowledge about the human body that has a broad range of applications, including early
disease detection and identification of the direction for a proper cure [8].There are many techniques for
processing medical images, which have their advantages and disadvantages. X-ray, molecular imaging,
ultrasound, MRI, positron emission tomography, computed tomography, PET-CT, and ultrasonic images are
commonly used imaging modalities [9,10]. Hemangioma, focal nodular hyperplasia, adenoma, hepatocellular
carcinoma, intrahepatic cholangiocarcinoma, and hepatic metastasis are all diagnosed with CT, US, and MRI
[11].
However, automated liver and its lesion segmentation remain challenging due to the inconsistent variations
between liver and lesion tissue caused by various acquisition methods, contrast agents, contrast enhancement
levels, and scanner resolutions [13]. Compared to previous examinations, convolution neural networks (CNN)
can help in deep learning with regard to liver lesions.

CNN’s performance is the best and, in some cases, has surpassed the knowledge of human radiology. In the
medical domain, CNNs have been widely used to detect various tumor forms
over the last few years. The researchers used convolutional neural networks and other deep learning systems to
incorporate ideas about liver tumor diagnosis. Both supervised and unsupervised classification is possible. The
feature sets are grouped into predefined groups in a supervised system, while they are allocated to undefined
classes in the unsupervised method. To train and evaluate the output of a classifier always requires training and
testing data [14].
3. Deep Learning Techniques Used For Segmentation:
Abstract: Deep learning (DL) techniques have excellent learning abilities [16]. Deep learning models such as
the convolutional neural networks (CNNs), stacked automatic encoder (SAE), deep belief network (DBN), and
deep Boltzmann machine (DBM) have been implemented.
Deep learning models are superior in terms of accuracy. After all, finding a suitable training dataset, which ought
to be large and constructed by specialists, remains a significant challenge. The literature revealed that DL-based
models for liver tumor detection have attained 94% accuracy. The CNN model comes in a variety of
architectures, including AlexNet, VGGNet, ResNet, and others.
4.Towards Deep Learning Based Aided Detection in CN Networks:
Abstract: Another study suggested using CNN to minimize the number of liver tomography images required
while reducing the time and money spent on maintenance.

The model has been improved, and the potential has been developed to distinguish between benign and malignant
tumors using a combination of deep learning and CNN. The researchers used a training dataset of 55,536 photos
from the 2013 database and 100 liver images from the 2016 data for testing. The experiments were repeated five
times, and the results showed an accuracy of 92%.
In a similar study, researchers used computed tomography images and a convolutional neural network to
compare various types of liver tumors. The researchers calculated the probability by segmenting each pixel using
special segmentation algorithms and a deep convolutional neural network. They used layer wrap to minimize
functions and eliminated three-dimensional variation by grouping layers into characteristics and classifying
tumors based on a fully connected layer.

CHAPTER 3
SYSTEM DESIGN
3.1 DATA FLOW DIAGRAM:
An information stream outline (DFD) is a graphical representation of the "stream" ofinformation

through a data framework. DFDs can likewise be utilized for the perception of information handling
(organized outline).
 LEVEL 0 DATA STREAM CHART
The Level 1 DFD shows how the system is divided into sub-systems (processes), each of
which deals with one or more of the data flows to or from an external agent, and which
together provide all of the functionality of the system as a whole. It also identifies internal
data stores that must be present inorder for the system to do its job, and shows the flow of data
between the variousparts of the system.
Liver Lesion
Liver Image Segmented
Segmentation
Dataset Images
1.0

 LEVEL 1 DATA FLOW DIAGRAM
The Level 1 DFD shows how the system is divided into sub-systems (processes), each of
which deals with one or more of the data flows to or from an external agent, and which
together provide all of the functionality of the system as a whole. It also identifies internal
data stores that must be present inorder for the system to do its job, and shows the flow of data
between the variousparts of the system.
Extract the Load CNN Liver

Liver Image
Features Models Segmentation
Dataset
1.0.1 1.0.2 1.0.3
Bounding Box Lesion

Segmented
Sampling Segmentation
Images
1.0.5 1.0.4
3.2 METHOD USED:
This section focuses on the steps used in implementing the liver and tumor segmentation using the
hybrid deep learning network ResUNet. The preprocessing, feature extraction, classification, and
segmentation are all part of the proposed method’s pattern in the deep learning network scheme.
Figure 1 shows the overall framework of the proposed model for liver and tumor segmentation.

3.3 IMAGE PROCESSING:
The proposed method is employed to extract useful segments from liver tumor images. Data
augmentation, preprocessing, and CNN is used to diagnose the liver and identify tumors in the
surrounding organs. In the preprocessing phase of CT images, Hounsfield unit values in the range
of −100 to 400 are passed on, neglecting the adjacent organs. Then, histogram equalization is applied
to the image to increase the contrast. Finally, some data from the magnification steps are used to
increase the data and teach the desired invariant properties, such as translation, rotation, deformation,
elasticity, and the addition of the Gaussian noise standard deviation, as shown in Figure 2.

 Histogram equalization is one of the image processing techniques used to enhance the
contrast between the liver and its neighboring organs for more visibility or understanding.
This made it easier to define the segmentation of the liver tumor. Histogram equalization is
illustrated with before and after CT images in Figure 4. Each CT image slice dataset has its
own tumor volumes and liver masks collection.

CHAPTER 4
PROJECT METHODOLOGY
4.1 METHODOLOGY:
 Data Collection
• Model Building
• Training
• Testing and Result Analysis
Load Dataset
Extract the Features
Admin
Load the CNN Models
Liver lesion
Segmentation
Bounding Box Sampling
Fig Methodology

Data Collection:
 The dataset taken was " Liver Dataset". It can be downloaded through thechallenge
link "https://competitions.codalab.org/competitions/17094".
We download the different weight from
 https://mega.nz/file/1r4XmDaB#9SPxHxMh1eYGzxtJzKN-
CTkODibYOxkxjoBY2gWVaFQ
Model Building:
 We have used pytorch for building the model.
 We used three models:-
a. The CNN model architecture consists of CNN Layer, Max Pooling, Flatten a Linear
Layers.
b. Using Transfer Learning RESNET34 Architecture.
Training - The model is trained by using variants of above layers mentioned in model
building and by varying hyperparameters. The best model was able to achieve more percent
of test accuracy.
Testing and Result Analysis – The Model was tested on total huge set of images.


CHAPTER 5
SYSTEM ARCHITECTURE
The modules we implemented are as follows

 Liver Segmentation
 Lesion Segmentation
 Bounding Box Sampling

Bounding Box Sampling: In object detection, we usually use a bounding box to describe the
spatial location of an object. The bounding box is rectangular, which is determined by the x and y
coordinates of the upper-left corner of the rectangle and the such coordinates of the lower-right
corner. Another commonly used bounding box representation is the (x,y)-axis coordinates of the
bounding box center, and the width and height of the box.
Lesion Segmentation: In the application of deep learning methods, lesion segmentation

merges the tasks of substructures segmentation, organ segmentation, and object detection.
K. Kamnitsas et al. [73] and M. Ghafoorianet al. [74] proposed a method to perform precise
segmentation with the help of local and global information using multi-stream networks.
Liver Segmentation: Liver segmentation for volumetric assessment is indicated prior to

major hepatectomy, portal vein embolisation, associating liver partition and portal vein
ligation for staged hepatectomy (ALPPS) and transplant. Segmentation software can be
categorised according to amount of user input involved: manual, semi-automated and fully
automated. Manual segmentation is considered the “gold standard” in clinical practice and
research, but is tedious and time-consuming.

CHAPTER 6
SYSYTEM REQUIREMENTS
6.1 Hardware Requirements:
 Hard disk : Minimum 100 GB.
 Ram : Minimum 4 GB.
 CPU : Intel 2.1 GHz
 Monitor : LCD minimum 15 inch.
6.2 Software Requirements:
 Operating system : Windows 7/8/10
 Programming language : Python.
 IDE : Spyder / Jupyter Notebook / Pycharm

 Software : Anaconda

6.3 FUNCTIONAL REQUIREMENTS

Functional requirements describe how a system must behave and establish constraints of its
functionality. This type of requirements is also known as the system’s quality attributes.
Attributes such as performance, security, usability, compatibility are not the feature of the
system, they are a required characteristic. They are "developing" properties that emerge from
the whole arrangement and hence we can't compose a particular line of code to execute them.
Any attributes required by the customer are described by the specification. We must include
onlythose requirements that are appropriate for our project.
6.4 NON-FUNCTIONAL REQUIREMENTS

 Reliability
The structure must be reliable and strong in giving the functionalities. The movements must be
made unmistakable by the structure when a customer has revealed a couple of enhancements.
The progressions made by the Programmer must be Project pioneer and in addition the Test
designer.
 Maintainability
The system watching and upkeep should be fundamental and focus in its approach. There
shouldnot be an excess of occupations running on diverse machines such that it gets hard to
screen whether the employments are running without lapses.
 Performance
The framework will be utilized by numerous representatives all the while. Since the system
willbe encouraged on a single web server with a lone database server outside of anyone's ability
to see, execution transforms into a significant concern. The structure should not capitulate
when various customers would use everything the while

6.4 RESOURCE REQUIREMENTS

⚫ SPYDER
Spyder, the Scientific Python Development Environment, is a free integrated development
environment (IDE) that is included with Anaconda. It includes editing, interactive testing,
debugging, and introspection features. After you have installed Anaconda, start Spyder on
Windows, macOS, or Linux by running the command spyder.Spyder is also pre-installed in
Anaconda Navigator, which is included in Anaconda. On the Navigator Home tab, click the
Spyder icon. For more information about Spyder, see the Spyder web page or the Spyder
documentation.
Anaconda command prompt is just like command prompt, but it makes sure that you are
able to use anaconda and conda commands from the prompt, without having to change
directories or your path. These locations contain commands and scripts that you can run
⚫ PYTHON PROGRAMMING
Python is a widely used general-purpose, high level programming language. It was created
by Guido van Rossum in 1991 and further developed by the Python Software Foundation. It
was designed with an emphasis on code readability, and its syntax allows programmers to
express their concepts in fewer lines of code. Python is a programming language that lets
you work quickly and integrate systems more efficiently.

CHAPTER 7
SOURCE CODES
7.1 DATASET CONNECTIVITY

import os, glob
import sys
import copy
import pydicom
import scipy
import scipy.misc
import numpy as np
import cv2
import imageio
from scipy.ndimage import rotate
from PIL import Image
from zipfile import ZipFile
Download the 3D-IRCADb-01 dataset

from https://www.ircad.fr/research/3d-ircadb-01/ and into a directory in theroot of the
repo and name it Dataset
dataset_path = "Dataset"
#ONLY DO THIS STEP IF YOU'VE DOWNLOADED THE ENTIRE DATASET AT ONCE NOT
PATIENT BY PATIENT
with ZipFile(os.path.join(dataset_path,"3Dircadb1.zip"), 'r') as zipObj:
try:
zipObj.extractall(dataset_path)
except Exception as error:
print(error)
dataset_path = "Dataset" # You can change the path of the directory if you have the
dataset elsewhere
# EXTRACT EVERY PATIENT'S .zip FILE

for file in os.listdir(dataset_path):
file_path = os.path.join(dataset_path, file)
if file_path.endswith(".zip"):
with ZipFile(file_path, 'r') as zipObj:
try:
zipObj.extractall(dataset_path)
except Exception as error:
print(error)
GO INTO THE JUST EXTRACTED D# IRECTORIES AND EXTRACT THE .zip FILES THAT
CONTAIN THE PATIENT SLICES AND MASKS
for dir_path in os.listdir(dataset_path):

dir_full_path = os.path.join(dataset_path, dir_path)
if os.path.isdir(dir_full_path):
if os.path.isfile(os.path.join(dir_full_path,"PATIENT_DICOM.zip")):
with ZipFile(os.path.join(dir_full_path,"PATIENT_DICOM.zip"), 'r') as zipObj:
zipObj.extractall(dir_full_path)
if os.path.isfile(os.path.join(dir_full_path,"MASKS_DICOM.zip")):
with ZipFile(os.path.join(dir_full_path,"MASKS_DICOM.zip"), 'r') as zipObj:
zipObj.extractall(dir_full_path)
# CHANGE DATASET STRUCTURE

# PREFIX FILES IN THE DIRECTORY WITH THE PATIENT'S ID
for folder in os.listdir("Dataset"):
x = folder.split('db')
folder_path = os.path.join('Dataset', folder)
if os.path.isdir(folder_path):
for subfolder in os.listdir(folder_path):
subfolder_path = os.path.join(folder_path,subfolder)
if os.path.isdir(subfolder_path):
for sub_sub in os.listdir(subfolder_path):
if os.path.isdir(os.path.join(subfolder_path, sub_sub)):
for image in os.listdir(os.path.join(subfolder_path, sub_sub)):
src= os.path.join(subfolder_path, sub_sub, image)
Dept. of CSE, SEACET 2022-23 Page 22

dst = os.path.join(subfolder_path, sub_sub, x[1]+'_'+image)
os.rename(src, dst)
src= os.path.join(subfolder_path, sub_sub)

dst = os.path.join(subfolder_path, x[1]+'_'+sub_sub)
os.rename(src, dst)
directory_to_move = "PATIENT_DICOM"
desired_scans_path = os.path.join("train", "patients")
desired_masks_path = os.path.join("train", "masks")
try:
os.mkdir("train")
except OSError as error:
print(error)
try:
os.mkdir(desired_scans_path)
print(error)
for patient_path in os.listdir(dataset_path):

patient_full_path = os.path.join(dataset_path, patient_path)
if os.path.isdir(patient_full_path):
patient_scans_path = os.path.join(patient_full_path, "PATIENT_DICOM")
if os.path.isdir(patient_scans_path):
for scan in os.listdir(patient_scans_path):
os.rename(os.path.join(patient_scans_path, scan), os.path.join(desired_scans_path, scan))
try:
os.mkdir(desired_masks_path)
print(error)
for patient_path in os.listdir(dataset_path):

patient_full_path = os.path.join(dataset_path, patient_path)

if(os.path.isdir(patient_full_path)):
patient_masks_path = os.path.join(patient_full_path, "MASKS_DICOM")
if os.path.isdir(patient_masks_path):
for scan in os.listdir(patient_masks_path):
os.replace(os.path.join(patient_masks_path, scan), os.path.join(desired_masks_path, scan))
try:
os.mkdir(os.path.join(desired_masks_path, "merged_livertumors"))
print(error)
# COUNT THE NUMBER OF SCANS FOR 1 PATIENT TO APPEND ON THEM

def count_scans_startwith(directory, prefix):
count = 0
for file in os.listdir(directory):
if file.startswith(prefix+'_'):
count+=1
return count
# MERGE ALL LIVERTUMORS IN 1 MASK

def merge_livertumors(scans_path, masks_path):
for scan in os.listdir(scans_path):
patient_id = scan.split('_')[0]
tumor_volume = None
for mask_dir in os.listdir(masks_path):
if mask_dir.startswith(patient_id+'_livertumor'):
current_tumor = pydicom.dcmread(os.path.join(masks_path, mask_dir, scan)).pixel_array
current_tumor = current_tumor/255.0
current_tumor = np.clip(current_tumor,0,1)
tumor_volume = current_tumor if tumor_volume is None else
np.logical_or(tumor_volume,current_tumor)
if tumor_volume is None:
tumor_volume = np.zeros((512,512))
tumor_volume = [[j*255 for j in i] for i in tumor_volume]

tumor_volume = np.array(tumor_volume)
tumor_volume = tumor_volume.astype(np.uint8)
im = Image.fromarray(tumor_volume)
im.save(os.path.join(masks_path, 'merged_livertumors', scan+'.jpg'))
#imageio.imwrite(os.path.join(masks_path, 'merged_livertumors', scan+'.jpg'), tumor_volume)
# REFLECT IMAGE AND MASK TO AUGMENT DATA

def reflect_dicom(src_img, src_mask, src_liver_mask, train_path, masks_path, liver_mask_path,
patient_id, patient_imgs_count):
reflected_img = copy.deepcopy(src_img)
reflected_img.PixelData = np.fliplr(reflected_img.pixel_array).tobytes()
reflected_liver_mask = copy.deepcopy(src_liver_mask)
reflected_liver_mask.PixelData = np.fliplr(reflected_liver_mask.pixel_array).tobytes()
reflected_liver_mask.save_as(os.path.join(liver_mask_path,
patient_id+'_image_'+str(patient_imgs_count)+'_augref'))
reflected_mask = np.fliplr(src_mask)
reflected_img.save_as(os.path.join(train_path,
patient_id+'_image_'+str(patient_imgs_count)+'_augref'))
reflected_mask = reflected_mask.astype(np.uint8)
imageio.imwrite(os.path.join(masks_path, 'merged_livertumors',
patient_id+'_image_'+str(patient_imgs_count)+'_augref.jpg'), reflected_mask)
# ROTATE IMAGE AND MASK TO AUGMENT DATA

def rotate_dicom(src_img, src_mask, src_liver_mask, train_path, masks_path, liver_mask_path,
patient_id, patient_imgs_count, angle):
rotated_img = copy.deepcopy(src_img)
rotated_img.PixelData = rotate(rotated_img.pixel_array, angle, reshape=False).tobytes()
rotated_liver_mask = copy.deepcopy(src_liver_mask)
rotated_liver_mask.PixelData = rotate(rotated_liver_mask.pixel_array, angle,
reshape=False).tobytes()
rotated_liver_mask.save_as(os.path.join(liver_mask_path,
patient_id+'_image_'+str(patient_imgs_count)+'_augrot'))
rotated_mask = rotate(src_mask, angle, reshape=False)
rotated_img.save_as(os.path.join(train_path,
patient_id+'_image_'+str(patient_imgs_count)+'_augrot'))
rotated_mask = rotated_mask.astype(np.uint8)
imageio.imwrite(os.path.join(masks_path, 'merged_livertumors',
patient_id+'_image_'+str(patient_imgs_count)+'_augrot.jpg'), rotated_mask)
# AUGMENT THE MASKS WITH TUMORS TO FIX CLASS IMBALANCING
def augment_dicom(train_path, masks_path):
rotation_angles = [x * 10 for x in range(36) if x > 0]
train_files = copy.deepcopy(os.listdir(train_path))
for scan in train_files:

mask_path = os.path.join(masks_path, 'merged_livertumors', scan+'.jpg')
tumor_mask = cv2.imread(mask_path)
if 1 in tumor_mask:
patient_id = scan.split('_')[0]
patient_imgs_count = count_scans_startwith(train_path, patient_id)
original_img = pydicom.dcmread(os.path.join(train_path, scan))
liver_mask_path = os.path.join(masks_path, patient_id+'_liver')
liver_mask = pydicom.dcmread(os.path.join(liver_mask_path, scan))
reflect_dicom(original_img, tumor_mask, liver_mask, train_path, masks_path,
liver_mask_path, patient_id, patient_imgs_count)
for angle in rotation_angles:

patient_imgs_count += 1
rotate_dicom(original_img, tumor_mask, liver_mask, train_path, masks_path,
liver_mask_path, patient_id, patient_imgs_count, angle)
masks_path = os.path.join('train','masks')
training_path = os.path.join('train','patients')
augment_dicom(training_path, masks_path)
merge_livertumors(training_path, masks_path)

print('done')
count = 0
for mask in os.listdir(os.path.join(masks_path,'merged_livertumors')):
tumor_mask = cv2.imread(os.path.join(masks_path,'merged_livertumors', mask))
if 1 in tumor_mask:
count += 1
print(count)
# REMOVE AUGMENTED DATA
for scan in os.listdir(training_path):
if scan.endswith('_augref') or scan.endswith('_augrot'):
os.remove(os.path.join(training_path,scan))
for mask_dir in os.listdir(masks_path):

if mask_dir.endswith('liver'):
mask_dir_path = os.path.join(masks_path, mask_dir)
for liver_mask in os.listdir(mask_dir_path):
if liver_mask.endswith('augref') or liver_mask.endswith('augrot') or liver_mask.endswith('aug'):
os.remove(os.path.join(mask_dir_path, liver_mask))
for mask in os.listdir(os.path.join(masks_path, 'merged_livertumors')):

if mask.endswith('_augref.jpg') or mask.endswith('augrot.jpg'):
os.remove(os.path.join(masks_path, 'merged_livertumors', mask))
print('done')
7.2 CODE DEPLOY
from future import print_function
import os
from skimage.transform import resize
from skimage.io import imsave
import numpy as np
from skimage.segmentation import mark_boundaries
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, concatenate, Conv2D, MaxPooling2D, Conv2DTranspose
from tensorflow.keras.optimizers import Adam, SGD
from tensorflow.keras.callbacks import ModelCheckpoint
from tensorflow.keras import backend as K
from skimage.exposure import rescale_intensity
from tensorflow.keras.callbacks import History
from skimage import io
from data import load_train_data, load_test_data
K.set_image_data_format('channels_last') # TF dimension ordering in this code
img_rows = int(512/2)
img_cols = int(512/2)
smooth = 1.
#We divide here the number of rows and columns by two because we undersample our data (We take
one pixel over two)
def dice_coef(y_true, y_pred):

y_true_f = K.flatten(y_true)
y_pred_f = K.flatten(y_pred)
intersection = K.sum(y_true_f * y_pred_f)
return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)
def dice_coef_loss(y_true, y_pred):

return -dice_coef(y_true, y_pred)
#The functions return our metric and loss
def get_resnet():
inputs = Input((img_rows, img_cols, 1))
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(inputs)
conv1 = Conv2D(32, (3, 3), activation='relu', padding='same')(conv1)
pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)
conv2 = Conv2D(64, (3, 3), activation='relu', padding='same')(pool1)





up6 = concatenate([Conv2DTranspose(256, (2, 2), strides=(2, 2), padding='same')(conv5), conv4],

axis=3)
conv6 = Conv2D(256, (3, 3), activation='relu', padding='same')(up6)

axis=3)

axis=3)

axis=3)
conv10 = Conv2D(1, (1, 1), activation='sigmoid')(conv9)
model = Model(inputs=[inputs], outputs=[conv10])
model.compile(optimizer=Adam(lr=1e-3), loss=dice_coef_loss, metrics=[dice_coef])

return model
#The different layers in our neural network model (including convolutions, maxpooling and
upsampling)
def preprocess(imgs):
imgs_p = np.ndarray((imgs.shape[0], img_rows, img_cols), dtype=np.uint8)
for i in range(imgs.shape[0]):
imgs_p[i] = resize(imgs[i], (img_cols, img_rows), preserve_range=True)
imgs_p = imgs_p[..., np.newaxis]

return imgs_p
#We adapt here our dataset samples dimension so that we can feed it to our network
import matplotlib.pyplot as plt
def train_and_predict():
print('-'*30)
print('Loading and preprocessing train data...')
print('-'*30)
imgs_train, imgs_mask_train = load_train_data()
imgs_train = preprocess(imgs_train)
imgs_mask_train = preprocess(imgs_mask_train)
imgs_train = imgs_train.astype('float32')
mean = np.mean(imgs_train) # mean for data centering
std = np.std(imgs_train) # std for data normalization
imgs_train -= mean
imgs_train /= std
#Normalization of the train set

imgs_mask_train = imgs_mask_train.astype('float32')
print('-'*30)
print('Creating and compiling model...')
print('-'*30)
model = get_resnet()
model_checkpoint = ModelCheckpoint('weights.h5', monitor='val_loss', save_best_only=True)
#Saving the weights and the loss of the best predictions we obtained
print('-'*30)
print('Fitting model...')
print('-'*30)
history=model.fit(imgs_train, imgs_mask_train, batch_size=10, epochs=2, verbose=1, shuffle=True,
validation_split=0.2,
callbacks=[model_checkpoint])
print('-'*30)
print('Loading and preprocessing test data...')
print('-'*30)
imgs_test, imgs_maskt = load_test_data()
imgs_test = preprocess(imgs_test)
imgs_test = imgs_test.astype('float32')
imgs_test -= mean
imgs_test /= std
#Normalization of the test set
print('-'*30)
print('Loading saved weights...')
print('-'*30)
model.load_weights('weights.h5')
print('-'*30)
print('Predicting masks on test data...')
print('-'*30)
imgs_mask_test = model.predict(imgs_test, verbose=1)
np.save('imgs_mask_test.npy', imgs_mask_test)

print('-' * 30)
print('Saving predicted masks to files...')
print('-' * 30)
pred_dir = 'preds'
if not os.path.exists(pred_dir):
os.mkdir(pred_dir)
for k in range(len(imgs_mask_test)):
a=rescale_intensity(imgs_test[k][:,:,0],out_range=(-1,1))
b=(imgs_mask_test[k][:,:,0]).astype('uint8')
io.imsave(os.path.join(pred_dir, str(k) + '_pred.png'),mark_boundaries(a,b))
#Saving our predictions in the directory 'preds'
plt.plot(history.history['dice_coef'])
plt.plot(history.history['val_dice_coef'])
plt.title('Model dice coeff')
plt.ylabel('Dice coeff')
plt.xlabel('Epoch')
plt.legend(['Train', 'Test'], loc='upper left')
plt.show()
#plotting our dice coeff results in function of the number of epochs
if name == ' main ':

train_and_predict()

CHAPTER 8
SNAPSHOTS

FIG: Acc over 02 epochs during the model ResUNet’s training progress for liver segmentation
intervals.
FIG: Acc over 50 epochs during the model ResUNet’s training progress for liver segmentation
intervals.

FIG: Results of the proposed model (a) liver segmentation and (b) tumor segmentation (without
tumor)

CONCLUSION
As a result, the hybrid ResUNet is significantly more effective in terms of training time, memory usage, and
accuracy as compared to baseline methods. The binary segmentation by classification layout was created to make
processing medical images easier. The basic 3D-IRCADB1 dataset was used to train and evaluate the proposed
model. The proposed technique properly identifies maximum tumor areas, with a tumor classification accuracy of
over 98%. However, after reviewing the data, it was discovered that there were only a small number of false
positives, which can be improved by false positive filters and training the model on a bigger dataset.
ResUNet delivered excellent results in terms of diagnosing quickly and efficiently. As we can
see from the results, deep learning neural networks assisted us in achieving our goals and are possibly the best tool
for dividing liver tumors. They can also be tried with tumors other than liver tumors, as the ResUNet showed
promising results.

REFERENCES
1.World Health Organization. World Cancer Report. 2021. Available online:

https://www.who.int/news-room/fact-sheets/ detail/cancer (accessed on 25 January 2022).
2. Key Statistics about Liver Cancer. 2022. Available online:
https://www.cancer.org/cancer/liver-cancer/about/what-is-
keystatistics.html#:~:text=The%20American%20Cancer%20cancers (accessed on 29 January2022).
3. Christ, P.F.; Elshaer, M.E.A.; Ettlinger, F.; Tatavarty, S.; Bickel, M.; Bilic, P.; Rempfler, M.; Armbruster,
M.; Hofmann, F.; D’Anastasi, M.; and 3D conditional random fields. In Proceedings of the Medical Image
Computing and Computer-Assisted Intervention (MICCAI 2016), Athens, Greece, 17–21 October 2016; pp.
415–423. [CrossRef]
4. Soler, L.; Delingette, H.; Malandain, G.; Montagnat, J.; Ayache, N.; Koehl, C.; Dourthe, O.; Malassagne, B.;
Smith, M.; Mutter, D.; et al. Fully automatic anatomical, pathological, and functional segmentation from CT
scans for hepatic surgery. Comput. Aided Surg. 2001, 6, 131–
142. [CrossRef] [PubMed]
5. Son, J.; Park, S.J.; Jung, K.H. Retinal vessel segmentation in fundoscopic images with generative adversarial
networks. arXiv 2017, arXiv:1706.09318.
6. Mharib, A.M.; Ramli, A.R.; Mashohor, S.; Mahmood, R.B. Survey on liver CT imagesegmentation methods.
Artif. Intell. Rev. 2012, 37, 83–95. [CrossRef]
7. Luo, S. Review on the methods of automatic liver segmentation from abdominal images. J. Comput.
Commun. 2014, 2, 1. [CrossRef]
8. Tariq, T.; Hassan, M.; Rahman, H.; Shah, A. Predictive Model for Lung Cancer Detection. LC Int. J. STEM
2020, 1, 61–74.
9. Latif, J.; Xiao, C.; Imran, A.; Tu, S. Medical imaging using machine learning and deep learning algorithms: A
review. In Proceedings of the 2019 2nd International Conference on Computing, Mathematics and Engineering
Technologies (iCoMET), Sukkur, Pakistan, 30–31 January 19.

Deeksha Final Report 1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Deeksha Final Report 1

Uploaded by

Copyright:

Available Formats

VISVESVARAYA TECHNOLOGICAL UNIVERSITY

“A Deep Learning Approach for Liver and Tumor

COMPUTER SCIENCE AND ENGINEERING

Under the guidance

Department of Computer Science and Engineering

S.E.A. COLLEGE OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Tumor Segmentation” has been successfully carried out by Ms. Muskaan

Signature of Guide Signature of HOD Signature of Principal

Signature of Internal Examiner Signature of External Examiner

Team Members USN SIGNATURE

BAVITHA VYAS 1SP18CS012

BIBEK K SAH 1SP19CS016

MUSKAAN KUMARI 1SP19CS060

We would like to thank to our respected Dr.Bhagavant K Deshpande, Director of SEA

2 LITERATURE SURVEY 5-8

SL.NO Name of the figure Page

3.1 Dataflow Diagram 1 9

3.2 Methods used 9

3.3 Image processing 11

3.4 Activity Diagram 12

4.1 Project Methodology 12

5.4 Anaconda Installation 20

5.7 Jupyter Notebook 20

5.8 Pycharm IDE 20

s Dept. of CSE, SEACET 2022-23 Page 1

1.1 PROBLEM STATEMENT:

s Dept. of CSE, SEACET 2022-23 Page 2

1.2 EXISTING SYSTEM:

s Dept. of CSE, SEACET 2022-23 Page 3

1.3 PROPOSED SYTSEM:

Fig: Proposed Architecture

s Dept. of CSE, SEACET 2022-23 Page 4

DRAWBACKS OF EXISTING SYSYTEM:

s Dept. of CSE, SEACET 2022-23 Page 5

s Dept. of CSE, SEACET 2022-23 Page 6

3. Deep Learning Techniques Used For Segmentation:

4.Towards Deep Learning Based Aided Detection in CN Networks:

s Dept. of CSE, SEACET 2022-23 Page 7

s Dept. of CSE, SEACET 2022-23 Page 8

3.1 DATA FLOW DIAGRAM:

An information stream outline (DFD) is a graphical representation of the "stream" ofinformation

 LEVEL 0 DATA STREAM CHART

s Dept. of CSE, SEACET 2022-23 Page 9

 LEVEL 1 DATA FLOW DIAGRAM

Extract the Load CNN Liver

Bounding Box Lesion

3.2 METHOD USED:

s Dept. of CSE, SEACET 2022-23 Page 10

3.3 IMAGE PROCESSING:

s Dept. of CSE, SEACET 2022-23 Page 11

s Dept. of CSE, SEACET 2022-23 Page 12

Extract the Features

Bounding Box Sampling

s Dept. of CSE, SEACET 2022-23 Page 13

We download the different weight from

 We have used pytorch for building the model.

 We used three models:-

b. Using Transfer Learning RESNET34 Architecture.

s Dept. of CSE, SEACET 2022-23 Page 14

s Dept. of CSE, SEACET 2022-23 Page 15

The modules we implemented are as follows

s Dept. of CSE, SEACET 2022-23 Page 16

Lesion Segmentation: In the application of deep learning methods, lesion segmentation