Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1
Quality Enhancement for Drone Based Video
using FPGA
1Y.
Vedavyas, 2S. Sri Harsha, 3M. Sai Subhash, 4S. Vasavi
1,2,3,4
Department of Computer Science & Engineering, VR Siddhartha Engineering College, India
2022 International Conference on Electronics and Renewable Systems (ICEARS) | 978-1-6654-8425-1/22/$31.00 ©2022 IEEE | DOI: 10.1109/ICEARS53579.2022.9751731
1
yvedavyas4077@gmail.com, 3sriharshaseelamneni448@gmail.com, 3 saisubhash654@gmail.com,
4
vasavi.movva@gmail.com
Abstract—Nowadays Drones are being widely used for
surveillance and various other activities. The video
stream produced by the drone can be disturbing or can
contain noise data which might reduce the quality of the
video stream. The video stream can be enhanced so that
there is no disturbance in the video stream. The video
enhancement can be done in real-time with the help of
field programmable gate array (FPGA) which reduces
the processing time with low energy consumption. Our
project mainly focuses on enhancing the quality of the
video stream using enhanced super-resolution
generative adversarial networks (ESRGAN),
contrast-limited Adaptive histogram equalization
(CLAHE), Gamma Correction and Saturation
Adjustment by integrating the image source in the
drone with the FPGA.
Index Terms—Image Processing, Drone Video, Field
Programmable Gate Arrays, Quality Enhancement,
Enhanced Super-Resolution Generative Adversarial
Networks, Contrast-Limited Adaptive Histogram
Equalization, Gamma Correction, Saturation
Adjustment .
I. INTRODUCTION
The term "video enhancement" refers to the process of
enhancing the quality of a video. In the video enhancement
procedure, there are a variety of techniques to choose from.
Enhancing video qualities such as brightness, contrast, and
color are the most basic approaches. In addition,
procedures such as Noise Reduction and Video De-noising
are commonly employed in video enhancement. In a wide
variety of applications, such as military, scientific
research, real-time surveillance and remote monitoring are
now becoming increasingly important. The demand for
real-time video streaming for surveillance applications is
greater than ever, necessitating computationally expensive
image processing architectures as well as innovative
encoding and decoding techniques.
As, Drones are being extensively used around the globe,
it is necessary to enhance the quality of the video generated
by these drones, to perform different analytics later on
the video generated and which will relatively increase
the accuracy of the analytics. The sensors of these cameras
are not crisp because the cameras installed inside the
drones must be cost-effective. Furthermore, these low-cost
cameras do not include advanced video enhancement
technologies. Despite the fact that these cameras are not
978-1-6654-8425-1/22/$31.00 ©2022 IEEE
orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on February 19,2023 at 14:32:59 UTC from IEEE Xplore. Restrictions ap
particularly good at capturing high-detail videos, they are
commonly employed in various applications. In the vast
majority of circumstances, these videos must be seen in real
time like to protect against trespassing, border
monitoring. It is difficult to monitor the area if the drone
cannot deliver sharp videos. Also, the observer of the drone
video may miss a mandatory scene in the video, which
could result in significant damage.
The paper will focus on the model used to enhance the
drone video using FPGA in real time. The prototype will
include various techniques like CLAHE, Saturation
Adjustment, Gamma Reduction. The Contributions can be
summarized as follows:
• We present the quality enhancement for drone-based
Video prototype to enhance the drone video generated
in real-time. The model takes the drone video as input
and is subjected to different stages of the model.
• The extracted frame from the video is sent to the
ESRGAN model, where the frame is enhanced and
sent for gamma correction and saturation adjustment.
A. MOTIVATION
In a wide variety of applications, such as military and
scientific research, real-time surveillance and remote
monitoring are now becoming increasingly important.
Drones are used to reach remote places that humans cannot
reach, and these drones can cover large areas in short time
effectively. Drone Coverage in a fraction of time is much
more effective than using the traditional approaches in any
given situation. These drones are often limited with
cost-effective cameras and the demand for real-time video
streaming for surveillance applications is greater than ever,
necessitating computationally expensive image processing
architectures as well as innovative encoding and decoding
techniques.
B. PROBLEM STATEMENT
Today Surveillance is done using drones, the images or the
videos which are collected from the drones may contain the
disturbances due to the camera which is attached or due to
the network issues. Hence the video stream which is
produced by the drone needs to be enhanced before it can be
used for any surveillance applications in real time scenarios.
The enhancement is done at the drone itself using Field
programmable Gateway array (FGPA) rather than
enhancing at receiver’s end.
C. SCOPE
The scope of the proposed system enhances the video
recorded by drones such as UAV's gliders and from go
29
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1

pros camera’s during flight time and can be used for
security surveillance for the places where humans cannot go
for gaining the information and is limited to day light with
clear conditions.
D. OBJECTIVES
• Enhance the quality of the video stream generated using
FPGA.
• Reduce the noise generated in the drone video stream.
• Adjust the color saturation of the drone video stream.
• Configure the contrast and the brightness of the video
stream using FPGA.
E. ADVANTAGES
• The model reduces the noise and distortion caused, by
using the histogram equalization and is more efficient.
• The enhancement of the video stream produced by the
drone is done in real time.
• High quality contrast along with smoother edges is
achieved by using the model.
F. APPLICATIONS
• The enhanced video sequence can be used in aerial
photography in a wide variety of domains.
• Scientific research in areas where human access is
restricted like mines.
• Increase the efficiency of the research by covering more
region is less time compared to the traditional
approaches.
• 24/7 surveillance of the borders to ensure no
trespassing.
• The enhanced video stream makes the search and rescue
process more effective.
The model in [6] will begin by converting the supplied
image to a binary sequence. The image layers are
transformed into binary sequence since any hardware
device can only work with binary values. Noise reduction,
picture enhancement, and image restoration were among the
image processing techniques employed. 8-bit 3 RGB values
are used to represent each image pixel. The complete image
is then converted to a series of pixels that can be accessed
using the row and column values, and the parallelism
method allows for faster operation and performance.
Instead of using the usual retinex method, [7] uses a
low-cost, high throughput retinex algorithm to improve the
quality of a real-time video stream. Using FPGA, they were
able to achieve a throughput of 60 frames per second for a
1920*1080 image with zero latency (0.2 MS). The entire
operation is divided into seven modules, each of which
performs an image improvement algorithm for a real-time
video stream. [8] uses the adaptive histogram equalization
with dynamic clip-threshold (AHEwDC) technique to
provide good real-time performance. They used FPGA to
improve the visibility of low-light portions in the video
sequence. At a threshold frequency of 75.84 MHz, the
technology they proposed can handle 30fps full HD video at
30 frames per second. Actual images and improved images
were compared using various AHEwDC thresholds, and the
optimized clip threshold produced the best enhanced
outcome of the input stream for all inputs. Invariant feature
extraction for object detection and classification is
described in [12].
III. PROPOSED SYSTEM
Figure 1 Presents Architecture of the proposed system.
A. PROPOSED ARCHITECTURE

II. LITERATURE SURVEY

In [1] they proposed a fast and efficient real-time video
defogging method based on the pyramid model, which
addresses issues with existing defogging algorithms such as
overestimation of transmittance, color distortion of the sky
area, and poor real-time performance. The texture was
refined by combined bilateral filtering, and the image was
returned to its original size. Finally, the degraded video was
restored using a combination of the atmospheric light
scattering model and the inter-frame video defogging
theory. Histogram equalization is commonly employed in
image improvement because of its simplicity and
efficiency; it alters the improved image's mean brightness
and introduces a lot of noise and distortion. As a result, [2]
used a fuzzy intensity measure and adaptive clipping Fig. 1. Architecture of proposed system.
histogram equalization to eliminate noise and distortion. [3]
built a deep learning image enhancement model that The proposed architecture of Quality Enhancement for
improves image quality and reduces noise. Two deep Drone Based Video Using FPGA consists of a drone
learning blocks comprise the model. The stacking denoising connected to FPGA. The video stream that is generated by
auto encoder, which comprises of three denoising auto drone is fed to FPGA which enhances the video quality,
encoder layers, is used to boost the contrast and recover the model starts with gathering input data of aerial videos and
noise in the first block, and the second block is used to the data undergoes pre-processing to remove the noise, later
reduce the noise in the image in the second block. key frames are extracted from the aerial video and these

978-1-6654-8425-1/22/$31.00 ©2022 IEEE 30

orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on February 19,2023 at 14:32:59 UTC from IEEE Xplore. Restrictions ap
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1
individual frames are enhanced and put together to form the
enhanced video, and the final video is sent to the receiver in
real time.
Each individual frame extracted from the drone video
undergoes through the same process. Initially, the frame is
given as input to the trained ESRGAN model, then the
output of the model is subjected to CLAHE, Gama
Correction, and saturation adjustment respectively. Finally,
the frame obtained after the model is free from noise and all
the parameters of the frame are optimal, so that the analysis
can be performed without any ambiguity.
Figure 2 presents Flow Chart of the proposed model
Fig. 2 Flow Chart of the proposed model
978-1-6654-8425-1/22/$31.00 ©2022 IEEE
orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on February 19,2023 at 14:32:59 UTC from IEEE Xplore. Restrictions ap
B. DATASET COLLECTION
The datasets which we used are UG2 Dataset [11] and
UAV Dataset. UG2 Dataset consists of videos collected
from UAV, Glider, and Ground videos. It consists of 684
videos with 30fps frame rate. It was published in IEEE
Winter Conference on Applications of Computer Vision
2018. UAV Dataset consists of videos collected from
different UAV's. It consists of 50 videos of 70250 frames
with 30 fps frame rate.
• link - https://github.com/rosauravidal/UG2-Dataset
C. SOFTWARE REQUIREMENTS
• Python Programming language
• TensorFlow
• OpenCV
• Python Image Library (PIL)
• Verilog
D. HARDWARE REQUIREMENTS
• Drone
• FPGA- ALTERA FPGA Cyclone II
• 4 Gigabytes of Read-Only Memory
• Available Storage Capacity of 15 Gigabytes
E. CLAHE
Contrast Limited Adaptive Histogram Equalization
(CLAHE) is a version of Adaptive Histogram Equalization
(AHE) that prevents the contrast from being over-amped.
CLAHE works on tiles, which are small areas of an image
rather than the complete image. To remove the false
boundaries, the surrounding tiles are blended using bilinear
interpolation. This algorithm is used to enhance image
contrast.
F. GAMMA CORRECTION
Controls the image's overall brightness. Images that haven't
been properly adjusted can appear bleached out or
excessively dark. Changing the amount of gamma
correction affects not just the brightness, but also the
red-green-blue ratios. We remove this using Gamma
correction because many bits are assigned to bright tones
and few bits are assigned to dark tones in the image using
(1)
O= I^(1/G) (1)
Where I is the input image, G is gamma value, O is output
image.
G. COLOR SATURATION ADJUSTMENT
The Method is linked to a correlation between light
brightness and color saturation discovered by light
simulation over exposure. When the brightness of the light
changes, so does the colour saturation. The YCbCr colour
model is used to calculate changes in colour saturation and
brightness.
H. GAN-GENERATIVE ADVERSARIAL NETWORKS
A Convolutional Neural Network (CNN) is a Deep
Learning system that can take an input image and assign
importance learnable weights and biases to various
aspects/objects in the image while also distinguishing
between them. A generative adversarial network (GAN) is
an ML model that pits two neural networks against each
other in order to improve prediction accuracy. GANs are
31
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1

often unsupervised and learn by cooperating in a zero-sum
game.
Instead of a simple residual network, the ESRGAN model
uses a Residual-in-Residual block as a fundamental
convolution block. To avoid smoothing out the artefacts in
the photos, the model lacks a batch normalization mlayer in
the generator. The ESRGAN generates images with a better
representation of the image artefacts' sharp edges. To better
approximate the probability of an image being real or fake,
the ESRGAN employs a Relativistic discriminator.
Figure 3 presents Working of ESRGAN model.
Step 3: Gamma Correction is applied on the Frame to the
overall brightness of an image using (1).
Step 4: Saturation Adjustment is applied to the frame to
make it more vibrant and colorful using (2) and (3).
cb' = ((cb-128) x saturation) + 128 (2)
cr' = ((cr -128) X saturation) + 128 (3)
Where, Y=0.2989× R+0.5866xG+0.1145* B
cb =-0.1688× R-0.3312xG+0.5000× B
cr=0.5000x R-0.4184xG-0.0816× B

IV. IMPLEMENTATION

This section presents the output and results of the proposed

system. The results are obtained when the suggested model
for Quality Enhancement for Drone Based Video using
FPGA is used to test UG2 dataset videos containing noise
and blurriness.
Figure 4 presents Unenhanced drone video frame from UG2
Dataset [11].

Figure. 3 Working of ESRGAN model [12]

I. FRAME EXTRACTION
The video is a combination of different frames or images so
for enhancing the video we need to first extract the frames
which are present in the video and then we will enhance it and
then all the enhanced frames are joined together to form a Fig. 4 Unenhanced drone video frame from UG2 Dataset
[11]
new enhanced video.
The frame enhancement will be done using ESRGAN, Figure 5 presents enhanced drone video frame After
CLAHE, gamma correction, saturation adjustment. The ESRGAN model enhancement.
frame which we extracted will be sent to the ESRGAN model
which will produce the high-quality image then the frame
will be sent to CLAHE which is a variant of Adaptive
histogram equalization which helps us in improving the level
of visibility in foggy environment and also helps in reducing
the noise while doing the histogram equalization. Then the
frame will be sent for Gamma Correction which helps in
correcting the brightness in the frame. Then the frame will be
sent to the Saturation Adjustment will enhances the image by
spreading the bits uniformly in the image such as when we
have high colored images the bit concentration will be more
at that point it will reduce so that the color contrast will be
uniform throughout the frame. Fig. 5 enhanced drone video frame After ESRGAN model
J. ALGORITHM enhancement.
Step 1: Enhance the image using pre trained ESRGAN Figure 6 presents drone video frame After gamma
model. correction.
Step 2: Contrast limiting adaptive histogram equalization is
applied on the enhanced frame from ESRGAN to adjusts the
global contrast of an image by updating the image
histogram's pixel intensity distribution.

978-1-6654-8425-1/22/$31.00 ©2022 IEEE 32

orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on February 19,2023 at 14:32:59 UTC from IEEE Xplore. Restrictions ap
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1
Fig. 6 drone video frame After gamma correction
Figure 7 presents the final enhanced drone video frame.
Fig. 7 final enhanced drone video frame
Figure 8 presents Comparison between before and after
applying CLAHE.
Fig. 8 Comparison between before and after applying
CLAHE
V. CONCLUSION AND FUTURE WORK
The proposed system can be used to enhance the drone
video which can be used for many purposes, as the drone
are being widely used for many purposes the proposed
system will be helpful in enhancing the quality of video
collected by the drones which have lower camera quality,
the proposed system also reduces the noise in the video
which is being recorded from the drone.
Future work includes improving the accuracy of the model
and the model can be developed to perform similarly for
any given climate like fog and night, which can be done by
using the techniques like defogging and night vision in this
case.
978-1-6654-8425-1/22/$31.00 ©2022 IEEE
orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on February 19,2023 at 14:32:59 UTC from IEEE Xplore. Restrictions ap
REFERENCES
[1] Lu Di, Yan Limin, A real-time video de fogging
algorithm, International Conference on Display Technology
(2020).
[2] Xiangyuan Zhu, Xiaoming Xiao, Tardi
Tjahjadi, Zhihu Wu, Jin Tang, Image Enhancement using
Fuzzy Intensity Measure and Adaptive Clipping Histogram
Equalization, IAENG International Journal of Computer
Science (2019).
[3] Che Aminudin, M.F., Suandi, S.A. Video surveillance
image enhancement via a convolutional neural network and
stacked denoising autoencoder.
Neural Comput & Applic (2021). https://doi.org/10.1007/s
00521-021-06551-0.
[4] K. Sun, M. Koch, Z. Wang, S. Jovanovic, H. Rabah and
S. Simon, "An FPGA-Based Residual Recurrent Neural
Network for Real-Time Video Super-Resolution," in IEEE
Transactions on Circuits and Systems for Video
Technology, doi: 10.1109/TCSVT.2021.3080241.
[5] B. Paulchamy, S. Chidambaram and J. Jaya, "Proficient
Technique for Satellite Image Enhancement Using Hybrid
Transformation with FPGA," 2021 International
Conference on Advances in Electrical, Computing,
Communication and Sustainable Technologies (ICAECT),
2021, pp. 1-11, doi: 10.1109/ICAECT49130.2021.9392560
[6] Sowmya, K., Rakshak Udupa, T. and Holla, S., 2020.
Implementation of an FPGA Real-Time Configurable
System for Enhancement of Lung and Heart
Images. Advances in Multidisciplinary Medical
Technologies ─ Engineering, Modeling and Findings,
pp.199-213.
[7] J. W. Park et al., "A Low-Cost and High-Throughput
FPGA Implementation of the Retinex Algorithm for
Real-Time Video Enhancement," in IEEE Transactions on
Very Large Scale Integration (VLSI) Systems, vol. 28, no.
1, pp. 101-114, Jan. 2020, doi:
10.1109/TVLSI.2019.2936260.
[8] Canran Xu, Zizhao Peng, Xuanzhen Hu, Wei Zhang,
Lei Chen, and Fengwei An, FPGA-Based Low-Visibility
Enhancement Accelerator for Video Sequence by Adaptive
Histogram Equalization with Dynamic Clip-Threshold,
IEEE Transactions on Circuits and Systems I: Regular
Papers (2020).
[9] Min-ling Zhu, Dong-yuan Ge Image Quality
Assessment Based on Deep Learning with FPGA
Implementation, (2020).
[10] Incremental Model (Software Engineering)
- javatpoint. (2019, August). www.Javatpoint.Com.
https://www.javatpoint.com/software-engineering-increme
ntal-model.
[11] Vidal, Rosaura G and Banerjee, Sreya and Grm,
Klemen and Struc, Vitomir and Scheirer, Walter J. Ug2: A
video benchmark for assessing the impact of image
restoration and enhancement on automatic visual
recognition 2018 IEEE Winter Conference on Applications
of Computer Vision (WACV).
33
 Proceedings of the International Conference on Electronics and Renewable Systems (ICEARS 2022)
IEEE Xplore Part Number: CFP22AV8-ART; ISBN: 978-1-6654-8425-1

[12] Che Aminudin, M.F., Suandi, S.A. Video surveillance

image enhancement via a convolutional neural network and
stacked denoising autoencoder.
Neural Comput & Applic (2021). https://doi.org/10.1007/s
00521-021-06551-0.
[13] Sharma, Rajesh, and Akey Sungheetha. "An Efficient
Dimension Reduction based Fusion of CNN and SVM
Model for Detection of Abnormal Incident in Video
Surveillance." Journal of Soft Computing Paradigm (JSCP)
3, no. 02 (2021): 55-69.
[14] Huang, Wenzhun, Shanwen Zhang, and Harry
Haoxiang Wang. "Efficient GAN-based remote sensing
image change detection under noise conditions." In
International conference on image processing and capsule
networks, pp. 1-8. Springer, Cham, 2020.
[15] J.-W. Chang, K.-W. Kang, and S.-J. Kang, “An
Energy-Efficient FPGA Based Deconvolutional Neural
Networks Accelerator for Single Image Super-Resolution,”
IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 1,
pp. 281–295, 2020.
[16] X. Wang et al., “ESRGAN: Enhanced super-resolution
generative adversarial networks,” in Proc. Eur. Conf.
Comput. Vis., 2018, pp. 1–16.

978-1-6654-8425-1/22/$31.00 ©2022 IEEE 34

orized licensed use limited to: AMRITA VISHWA VIDYAPEETHAM AMRITA SCHOOL OF ENGINEERING. Downloaded on February 19,2023 at 14:32:59 UTC from IEEE Xplore. Restrictions ap