March 24, 2023

INTERDICIPLINARY
RESEARCH
SEMINAR
Final Progress 1

ECHOCARDIOGRAPHY IMAGE SEGMENTATION

USING A FULLY CONVOLUTIONAL NEURAL
NETWORKS AS A HEART DIAGNOSTIC TOOL
Jauharil Feti Noor Rohmah
1121800013

SUPERVISOR

1st Supervisor
RIYANTO SIGIT, ST., M.Kom., Ph.D.

2nd Supervisor
Dr. TITA KARLITA, S.Kom, M.Kom
 01
Background
 Heart disease or cardiovascular disease is still the
deadliest disease in the world

02 In the world of health or medicine, technology is

needed so that the patient's condition can be
known with certainty and get the proper treatment.
 This study uses ultrasound or echocardiography to
measure heart function 。
 The advantage of using ultrasound is that it is
cheap and can be used in critically ill patients and
the results are obtained immediately

03
 This system is used to perform cardiac diagnosis
using echocardiography or ultrasound by means of
segmentation (FCNs). This system is expected to
help speed up diagnosing the heart.
 Problem
2 4
How do you process the
obtained microdicom heart How to do echocardiographic
video into heart images that image segmentation using a
will be managed with the fully convolutional neural
Short Axis View point of network as a heart diagnosis?
view?

1 3 5
How to evaluate the
How to diagnose a normal How to do pre-
results of the
heart and a heart affected processing on the
segmentation that has
by coronary heart Ultrasound image of the
been produced?.
cardiovascular disease heart?
from Ultrasound images?
 Purpose

Perform segmentation
that can diagnose a
normal heart and a heart Develop cardiac
Looking for the
affected by coronary classification using
FCNs model so that
heart cardiovascular FCNs to assist
a higher accuracy
disease in the Left doctors in
value is produced for
Ventricle (LV) using diagnosing cardiac
diagnosing the heart.
machine learning with the ultrasound images..
FCNs method.
1 2 3
Contribution

This research is expected to provide the latest

advances in the field of medical imaging,
especially in diagnosing the heart using a
single view, namely the short axis on the left
vertical with the application of the FCNs deep
learning method.
 Related Research
IMAGE SEGMENTATION FOR VIDEO
ECOCARDIOGRAPHY USING TRIANGLE
EQUATION
This research has succeeded in achieving the Hasan Imaduddin
main research objectives, namely:
developed a new and efficient image
segmentation method for video 2020
echocardiography using the triangle equation
method The PMC and MC methods have an
average small heart cavity area error of 0.64% ECOCARDIOGRAPHIC IMAGE SEGMENTATION
and 0.93%, respectively MULTIDIMENSION AUTOMATICALLY
USING DEEP LEARNING
The study says that VGG-16 has a higher
2014 average accuracy in each of them. the viewing
angle is about 83%-93% of the architecture
Riyanto Sigit RESNET-18 with an average accuracy of
about 76%-92%.
Segmentation
Image segmentation is the process of
assigning a label to every pixel in an image
such that pixels with the same label share
certain characteristics

In this case, segment the cardiac cavity that

will be named the “jantung" class shown in
blue and in addition to cardiac cavity that will
be nambed the "background" class shown in
brown.
Differences In Segmentation And Classification
Segmentation
data set in the form of abnormal heart
patient data and data segmentation results,
then the data is trained and the data testing
will be tested
and the results of segmentation of the
abnormal heart area were detected
Classification
data set in the form of normal and abnormal
data
then the data is trained, and in testing the
data will be tested and detected whether it is
included in the normal or abnormal category
Echocardiography Transducer TTE Echocardiography Output Results

four-chamber two-chamber

Windows

long axis short axis

echocardiographic anatomy
 point of view
long axis point of view short axis point of view

probe marker approx

11 o'clock

probe marker to the left

shoulder

four-chamber point of view two-chamber point of view

probe markers
probe marker is up to 12 o'clock
around 3 o'clock
 Movement of the heart walls
Abnormal Cardiac Videos
Normal Cardiac Videos

abnormal heart beats too fast, too slow,

irregular or even not moving at all

Normally, the heart beats 60 to 100 times per minute

when at rest.
A B

a) Short Axis Anatomy (a) abnormal heart akinetic in the Atrium 2

b) Short Axis normal heart image on TEE Chamber pair
view （ Transthoracic (b) abnormal heart with dyskinetic
echocardiogram ） movement in the parasental Short Axis

SHORT AXIS
VIEW
How is Fully Convolutional Network (FCN) different from the
original Convolutional Neural Network (CNN)?

80% 40%

source : A Novel Deep Fully Convolutional Network for PolSAR Image Classification by Yangyang Li, Yanqiao Chen,
Guangyuan Liu and Licheng Jiao
School of Artificial Intelligence, Xidian University, Xi'an 710071, Shaanxi, China

 The convolutional neural network (CNN) model, based on convolution,

pooling and nonlinear transformation operations, has achieved great
performance in image classification. For the classification framework of CNN,
the input is the image, and the output is the class of the image. However, the
classification result of CNN cannot reflect the details of the image
 Fully Convolutional Network (FCN) model, which is shown in Figure and can
be trained end-to-end, pixels-to-pixels. FCN transforms fully connected
layers into convolutional layers, which enables an efficient classification net
for end-to-end dense learning
Fully Convolutional Neural Network

Fully Convolutional Networks or FCNs, are

U-Net is one of the architectural variants of CNN an architecture used mainly for semantic
which is widely used for segmentation in the medical segmentation.
field

U-Net has 2 main parts, namely contracting path FCN only has the encode path that
(encode) to take input features and creating makes up the architecture it does not
dimensions match the input, so the decode feature
which is smaller than the input, and the expansive from U-Net is needed to overcome the
path (decode) retrieves the encoded features and deficiency.
provides the best match with the main input or
planned output.
Ilustrasi
Semantic
segmentation
Ilustrasi U-Net
 Input
Collection Data
Doctor Labeling :
Primary data set – RSUD Dr. Soetomo Data labeled by a doctor

Years number of Healthy Unhealthy Not in other

Patients +ve -ve Analysis description
2008 A 35 3 6 23 3
2008 B 34 1 1 31 1
2009 A 77 - - 77 -
2009 B 33 - - 33 -
2010 A 144 15 18 111 2
2011 A 108 2 25 79 2
Total Data 431 21 50 354 8
1 patient has 17 - 18 Dicom video files
in the trial: try the file analyzed by the doctor's unhealthy statement -ve, 8 patient data is
blank. 2 patients have 9 images
So, totaling 42 patients, 408 jpg files 1 patient has 2 frames or 20 images
 system design
Input Pre-Procesing
Collection Data
Convert Video to Image

Cardiac Abnormal
Cardiac Abnormal
US Image Resizing 256 x 256 pixels
US Image Original
Segmentation

Segmentation image enhancement

Testing Training
Pixel Recall
Intersection Over
Image Enhancement
Union （ IOU)
• Grayscale
• Median filter
Pixel Accuracy
• High boost filter
• Morfology operation
• Thresholding
Evaluation
• Canny filter Fully Convolutional Networks (FCN) Pixel Precision
• watershed
 Method
Fully Convolutional Networks (FCN) U-net

(A) Architecture of a fully convolutional neural network (FCN). The FCN first takes the
whole image as input, learns image features though the encoder, gradually recovers
the spatial dimension by a series of upscaling layers (e.g., transposed convolution
layers, unpooling layers) in the decoder and then produce 4-class pixel-wise
probabilistic maps to predict regions of the left ventricle cavity (blue region), the left
ventricular myocardium (green region) and the right ventricle cavity (red region) and
background. The final segmentation map is obtained by assigning each pixel with
the class of the highest probability.
(B) Architecture of a U-net. On the basis of FCN, U-net adds “skip connections” (gray TRAIN : 75% x 408 = 306 image
arrows) to aggregate feature maps from coarse to fine through concatenation and TEST; 25% x 408 = 102 image
convolution operations.
 Method
Pixel Recall
Pixel recall is the proportion of boundary pixels in the ground
truth that were correctly classified by the segmentation.

Once learning is applied to visual data analysis, the

Pixel Accuracy
Pixel accuracy is a ratio between the number of correctly
classified pixels and the total number of pixels.
performance results of the proposed methods
should be evaluated and compared
however, performance appraisal indicators cannot
be used because they are still in the optimization
stage

Pixel Precision
Pixel precision is a ratio between the number of correctly
classified positive pixels and the total number of positive
predicted pixels.
 Method
FCN U-net
Encoder(input)
 Conv2D(16, (3, 3) input
 Dropout(0.1)  Conv2D(16, (3, 3)
 MaxPooling2D((2, 2)  Dropout(0.1)
 Flatten(32)
 Dense(512,512)
 Softmax(64)
Decoder  MaxPooling2D((2, 2)
 Conv2D(16, (3, 3)
 Dropout(0.1)
 MaxPooling2D((2, 2)
 = Conv2DTranspose(32, (2, 2)
 Result
No Parameter Nilai Keterangan
Intersection Over Union
Intersection over union, also known as Jaccard index, is the percent
overlap between the target mask and the prediction output. 1 Epoch 50 Number of iterations
2 Batch size 8 The total size of the training
data
Accuracy loss
3 Learning 0,001 System learning speed
rate
Unet akurasi: 98.92 %
IOU : 98.93

Fcn Akurasi : 98.62 %

IOU : 98.62
 Conclusion

no conclusion yet
Next
Progress 2
repair segmentation or image improvement to
match the organ marking from the actual heart
point of view

Perform various pre-processing method

scenarios in order to get the best results

Conduct training using the FCNs method with

various data ratios to get the best model

Perform comparisons with various FCNs

segmentation methods to prove that the method
used can show maximum results
Thank you