Video Compression for Surveillance Application
using Deep Neural Network
Department of Computer Science and Engineering, RRCE
Rajarajeswari College of Engineering Banglore,Affilicated to Visvesvaraya Technological university
Vmsp4umohan@gmail.com , navyagowda270@gmail.com , mantoshkichni@gmail.com ,
abhileshpatil1961@gmail.com , pratibha.deshak@gmail.com
Abstrac : A fresh comparison analysis on video
compression technologies was offered in this
study. Video streaming programmes are
becoming increasingly popular as internet
technology and computers advance at a rapid
pace. As a result, uncompressed raw video
demands a lot of disc space and network
bandwidth to store and deliver today. We
describe a novel technique to video compression
surveillance that improves on the shortcomings
of previous approaches by replacing each
standard component with a neural network
counterpart. . We describe a novel technique to
video compression surveillance that improves on
the shortcomings of previous approaches by
replacing each standard component with a
neural network counterpart. It delivers a higher-
quality video stream at a consistent bit rate
(compared to previous standards). As a result,
you must select the appropriate video
compression technology to fulfil the
requirements of your video application.. Our
work is founded on a set of principles common
method for reducing the bit rate while
minimising distortions in decoded frames by
using In video frames, there is spatial and
temporal redundancy. We use a neural network
to develop a video compression in the traditional
sense strategy and encode redundant data with
fewer bits. Experiments have revealed that our
solution is successful and surpass traditional
MPEG encoding while retaining visual quality at
similar bit rates.Although our approach is geared
at surveillance, it can simply be applied to other
types of video.
Keyword- Video Compression, Algorithm, Deep
Learning,MotionEstimation, BitRate Estimation
I. INTRODUCTION
Surveillance cameras are becoming more common
in countries around the world. Approximately in
There are 760 million security cameras deployed
around the world, with the number expected to climb
to 1 billion by 2020. By 2023, the population will be
around 1 billion. Benefits such as better security and
lower crime The high level of interest in these
technologies can be linked to the rates. A Most
surveillance recordings have a lot of temporal
redundancy because the surroundings are usually
static and only a few people are moving about
Moving items cause a portion of the frame to alter.
As a result, a significant portion of bandwidth is
used to broadcast scenes that are of little or no
interest because they don't include semantically
interesting items (For example, people, vehicles, and
animals). unprocessed video feeds from all
cameras are frequently transferred to a to a single
server for survey, processing, and viewing in
surveillance a camera network with a big number of
cameras. It puts a lot of strain on the processor. the
primary server Furthermore, in most situations, the
streams must be stored for future usage, which
requires a significant amount of storage. The
monitoring system's problems can be handled by
employing a compression of video technology that
employs spatiotemporal redundancy. The key
details are preserved in the streams, making
additional analysis and storage possible.
Many existing standard for video compression, such
as MPEG and H.264, which use mathematical
techniques to compress video, are widely
used.While they have been meticulously created and
fine-tuned, they are intended to be utilised in a
specific environment. They can only be general in
their surveillance applications, which limits their
potential to be specific.
Many problems have been solved using deep
learning techniques, incorporating the field of
picture compression, which has made significant
progress. Commercial codecs have been
outperformed by ML-based image compression
techniques by a wide margin, and progress is still
being made. Advancements in hardware to support
machine learning-based applications have
accelerated this process even more However, while
image compression is mostly focused on spatial
estimate, video compression should also take into
account motion information.
Deep learning methods for video compression are
being investigated throughout the full compression
process.For a portion of a typical pipeline One
effective approach to our situation is Detecting
interesting video frame sections and allocating more
bits in the video frame might be difficult. They are
given a compressed representation. In the rest of the
frame, fewer bits are assigned.By smoothing out
information that would otherwise be included in the
transmitted video unnecessarily, a compressed
stream can be created. Despite the fact that object
detection techniques are robust and accurate, this
strategy will make the answer very specialised.
Our solution is to compress video streams using an
end-to-end deep learning model. Surveillance
systems that are both reliable and light on memory.
The process of compressing a video file such that it
takes up less space is known as video compression.
It's smaller than the original file and easier to send
over the internet. It's a compression technique of
some sort. It minimises the size of video files by
removing unnecessary and non-functional data.
Encoding of video is the process of compressing and
preparing your video file for playback in the
appropriate formats and specifications. Although
video compression reduces file size, it may have
unintended consequences. excellent video quality
Video encoding, on the other hand, compresses your
video file while maintaining its quality.
Deep Learning is a subset of machine learning that
works with algorithms influenced by data structure.
The construction and function of neural networks is
a form of brain structure. A convolutional neural
network (CNN) is a kind of neural network that
analyses large amounts of data using a deep learning
method. Assign priority to various elements in an
image and be able to distinguish one from the other
using an input image .The quantity pre-processing
required by a ConvNet is significantly less than that
of other classification algorithms.
II. EASE OF USE
A.TRADITIONAL VIDEO COMPRESSION
PIPLINE
The most often used and widely employed standard
for today's video compression is H.264, which was
conceived by a figure of important participants in the
coding for video industry.
In general, video compression algorithms take use of
the vision system of the human redundant reactions.
Compression techniques used in the past To
transform the sampled Red, Green, and Blue spatial
sequence into Luma and three components of
chroma, use the Y:Cr:Cb Color Space. The HVS's
luma components are very sensitive, however
the chroma components are not. In the
encoded signal, only two chroma components
can be sent, with the remaining components
plus luma constituting the third component.
B.PROPOSED SYSTEM
Motion Estimation: It is determined the
distance between the current and previous
frames.
Motion Compensation: Pixels from previous
frames are shifted on the basis of motion vector
or optical flow obtained during the motion
estimating process to create the frame with
motion compensation.
Residual Compression: It is calculated and
compressed the difference between the targeted
and compensated for movement frames.
Motion Compression: The motion data
gathered during the motion estimate step is
compressed and delivered to be encoded.
Encoding: After compression, the residue and
motion information are encoded and delivered
to the decoder.
FrameReconstruction :To generate a rebuilt
frame, the motion adjusted frame from the
motion compensation step is combined with
residue from the residue compression steps.
C.TRANSFORMATION FUNCTIONS
The frame data is converted from the spatial domain
using the transformations function. This thesis
focuses on the most recent and widely discussed
innovations in the last five years. As a result, H.264
and HEVC have been chosen for further
investigation. The transform basis functions are
obtained from the DCT in video coding techniques
in general and H.264, HEVC in particular. DCT is a
method for transforming image component picture
elements from the spatial domain to the frequency
domain, where redundancy can be discovered. The
frame is broken into blocks with sizes ranging from
4*4 to 64*64 in video compression. a method of
image compression JPEG breaks the image into 8*8
blocks and applies a two-dimensional discrete
cosine transform (DCT) on each of them.
The DCT function is represented by equation (1).
The Inverse Discrete Cosine Transform (IDCT) is
used to the DCT coefficient blocks in video
decompression. The IDCT function is defined by
equation(2).
where f_k and F_k is the block index, and is the pixel
value and DCT coefficient, respectively.
The uncompressed/raw as well as lossless
compression digital image requires high memory and
bandwidth. DCT significantly reduces the amount of
memory and bandwidth.
D.QUANTIZATION FUNCTION
The inevitability of representing a value as a number
with a definite number of decimal places is
quantization. The quantization scale code is divided
element-wise by the quantization matrix in video
coding, and each resultant element is rounded. The
quantization parameter determines the step size for
associating the converted coefficients with a limited
number of footsteps.When it comes to video coding,
the value is inversely proportional to the PSNR value
and directly proportional to the CR. In both cases, the
DC value is the value with zero frequency. Inverse
quantization formulas are applied to a converted
signal, and equations (3) and (4) are the quantization
equations. Equations (5) and (6), on the other hand,
are the quantization formulas for intra and inter-
coding with AC values — the non-zero frequencies.
The inverse quantization formula used on the
standards is Equation (7).
F k (x, y) is the DCT transformed signal, and is the
block index, where delta is the quantization
parameter and F k (x, y) is the block index. In video
coding, DC represents the value x=y=0, AC
represents the worth (x, y) member of (0,1),...,(n, n),
and n represents the block size.
E. MOTION COMPENSATION AND MOTION
ESTIMATION
This subsection examines the motion estimation
(ME) and motion compensation (MC) functions,
which are the most important aspects of inter coding.
In video coding, motion is utilised to encode one
frame in terms of another.The motion estimation
technique uses the block motion estimation concept,
which encodes frame data using modified versions of
other frames.
The full search (FS) algorithm is the most
fundamental and basic search strategy. FS involves
using a cost function to identify the best block match
out of all the pixels in the search range. SAD is
chosen in H.264 and HEVC because it requires fewer
resources than the other cost functions while
simultaneously having low distortion, which is
acceptable. Equation (8) examines a block of
candidates in the preceding frame (I t-1) at location
(x+u, y+v) and a template block in the present
frame at position (x,y).
where g t(.) is a pixel size in frame I t, and g t-1(.) is
a pixel size in frame I t-1. In (u v), the motion vector
(MV) is defined as equation (9).
F.ENTROPY CODING
Entropy encoding is a lossless data compression
approach that is unaffected by the medium's
particular features. Each unique symbol in the input
is given a unique prefix code through entropy coding.
Figure : Proposed end-to-end video compression
framework. There is one-to-one mapping between
the traditional video compression pipeline and our
proposed framework
G.CONVOLUTIONAL NEURAL NETWORK
Convolutional Neural Networks (CNNs) are neural
networks that are trained to handle features with a
like a grid structure, as an example photographs. A
binary representation of visual data is a as an
example. It's made up of a grid of pixels with pixel
values that specify how bright and what colour each
one should be.. When we see an image, our brain
processes a large the amount of data. Each neuron
has its own receptive field, which is linked to the
receptive fields of neighbouring neurons to cover the
full visual field. All neuron in the biological
perception system only reacts to external stimuli in a
tiny region of the visual field called the receptive
field, and each neuron in a CNN only data analysis in
its receptive area. A less complicated pattern (lines,
curves, and so on) appear first in the layers, after that
pattern that is more complicated (faces, objects,
etc.). A CNN can be used to provide sight to a
computer. The CNN's most critical component is the
convolution layer. It is in charge of the majority of
the processing power on the system.
This layer does a dot product between two-
dimensional matrix, one of which is the restricted
area of the receptive field and the other a set of
learnable parameters called a kernel.. The kernel has
more depth than a picture, despite its smaller size.
The kernel height and width will be modest if the
image has three (RGB) channels, but the depth will
span all three. A convolutional neural network is a
feed forward neural network with up to 20 or 30
layers.The convolutional layer, an unique sort of
layer, gives a convolutional neural network its
power.
CONCLUSION
In the past, video was stored on magnetic cassettes.
The discrete in video has been around for a long
time. since the beginning compression . The cosine
transform is a common tool. H.261, the first
practical video computer standard, was born out of a
series of projects. H.264 is a video compression
standard created by a number of major participants
in the video computer industry.It is today's most
widely acknowledged and used video computing
standard.
The process of reducing the file size and modifying
the format of a video is known as video
compression.. It can save money by reducing the
amount of storage space needed to store video. It
also minimises the amount of bandwidth needed to
send video, making media consumption more
enjoyable for users. It eliminates frames that are
duplicated or repeated, leaving only the ones that are
required. Assume there are two frameworks that are
quite similar. Compression removes unnecessary
data from one frame and exchanges it with an
instance of the other. The purpose of video
compression is to send video data at a low bitrate
while maintaining image quality.
Figure (a): Original(target) frame,(b): Reconstructed
frame using proposed work

Figure (a): Comparison between our model and
MPEG based on MS-SSIM, (b): Comparison
between our model and MPEG based on PSNR
References
[1] Le Gall, D. (1991). Mpeg: A video
compression standard for multimedia
applications. Communications of the ACM,
34(4), 46–58.
[2] Wiegand, T., Sullivan, G. J.,
Bjontegaard, G., & Luthra, A. (2003).
Overview of the h. 264/avc video coding
standard. IEEE Transactions on circuits and
systems for video technology, 13(7), 560–
576.
[3] Liu, H., Chen, T., Shen, Q., Yue, T., &
Ma, Z. (2018). Deep image compression via
end-to-end learning., In Cvpr workshops.
[4] Toderici, G., Vincent, D., Johnston, N.,
Jin Hwang, S., Minnen, D., Shor, J., &
Covell, M. (2017). Full resolution image
compression with recurrent neural
networks, In Proceedings of the ieee
conference on computer vision and pattern
recognition
[5] Kim, S., Park, J. S., Bampis, C. G., Lee,
J., Markey, M. K., Dimakis, A. G., &
Bovik, A. C. (2020). Adver-sarial video
compression guided by soft edge detection,
In Icassp 2020-2020 ieee international
conference on acoustics, speech and signal
processing (icassp). IEEE
[6] Lu, G., Ouyang, W., Xu, D., Zhang, X.,
Cai, C., & Gao, Z. (2019). Dvc: An end-to-
end deep video compression framework, In
Proceedings of the ieee conference on
computer vision and pattern recognition.
[7] Rippel, O., Nair, S., Lew, C., Branson,
S., Anderson, A. G., & Bourdev, L. (2018).
Learned video compression, arXiv
1811.06981
[8] Farnebäck, G. (2003). Two-frame motion
estimation based on polynomial expansion, In
Scandinavian conference on image analysis.
Springer.
[9] Fischer, P., Dosovitskiy, A., Ilg, E., Häusser,
P., Hazırbaş, C., Golkov, V., Van der Smagt, P.,
Cremers, D., & Brox, T. (2015). Flownet:
Learning optical flow with convolutional
networks. arXiv preprint arXiv:1504.06852
[10] Ranjan, A., & Black, M. J. (2017). Optical
flow estimation using a spatial pyramid
network, In Proceedings of the ieee conference
on computer vision and pattern recognition.
[11] N. Ahmed, T. N., & Rao, K. R. (1974).
Discrete cosine transform. IEEE Transactions
on Computers, C-23(1), 90–93.
[12] Sullivan, G. J., Ohm, J., Han, W., &
Wiegand, T. (2012). Overview of the high
efficiency video coding (hevc) standard. IEEE
Transactions on Circuits and Systems for Video
Technology, 22(12), 1649–1668.
[13] Hadar, O., Shleifer, A., Mukherjee, D.,
Joshi, U., Mazar, I., Yuzvinsky, M., Tavor, N.,
Itzhak, N., & Birman, R. (2017). Novel modes
and adaptive block scanning order for intra
prediction in AV1 (A. G. Tescher, Ed.). In A. G.
Tescher (Ed.), Applications of digital image
processing xl, SPIE. International Society for
Optics and Photonics.
https://doi.org/10.1117/12.2274035
[14] Laude, T., & Ostermann, J. (2016). Deep
learning-based intra prediction mode decision
for hevc, In 2016 picture coding symposium
(pcs).
[15] Cui, W., Zhang, T., Zhang, S., Jiang, F., Zuo, Simoncelli, E. P. (2016). End-to-end optimized
W., & Zhao, D. (2018). Convolutional neural image compression. arXiv preprint
networks based intra prediction for hevc. arXiv:1611.01704.

[16] Zhao, Z., Wang, S., Wang, S., Zhang, X.,

Ma, S., & Yang, J. (2018). Cnn-based bi-
directional motion compen-sation for high
efficiency video coding, In 2018 ieee
international symposium on circuits and
systems (iscas). [25] Marpe, D., Schwarz, H., & Wiegand, T.
(2003). Context-based adaptive binary
[17] Lee, J. K., Kim, N., Cho, S., & Kang, J.
arithmetic coding in the h. 264/avc video
(2018). Convolution neural network based
compression standard. IEEE Transactions on
video coding technique using reference video
circuits and systems for video technology,
synthesis, In 2018 asia-pacific signal and
13(7), 620–636.
information processing association annual
summit and conference (apsipa asc). [26] Ballé, J., Minnen, D., Singh, S., Hwang, S. J.,
& Johnston, N. (2018). Variational image
[18] Liu, J., Xia, S., Yang, W., Li, M., & Liu, D.
compression with a scale hyperprior. arXiv
(2019). One-for-all: Grouped variation
preprint arXiv:1802.01436.
networkbased fractional interpolation in video
coding. IEEE Transactions on Image Processing, [27] Kingma, D. P., & Welling, M. (2013). Auto-
28(5), 2140–2151. encoding variational bayes. arXiv preprint
arXiv:1312.6114.
[19] Ibrahim, E. M., Badry, E., Abdelsalam, A.
M., Abdalla, I. L., Sayed, M., & Shalaby, H. [28] Rissanen, J., & Langdon, G. (1981).
(2018). Neural networks based fractional pixel Universal modeling and coding. IEEE
motion estimation for hevc, In 2018 ieee Transactions on Information Theory, 27(1), 12–
international symposium on multimedia (ism). 23.
[20] Jiang, F., Tao, W., Liu, S., Ren, J., Guo, X., & [29] Wang, Z., Bovik, A. C., Sheikh, H. R., &
Zhao, D. (2018). An end-to-end compression Simoncelli, E. P. (2004). Image quality
framework based on convolutional neural assessment: From error visibility to structural
networks. IEEE Transactions on Circuits and similarity. IEEE transactions on image
Systems for Video Technology, 28(10), 3007– processing, 13(4), 600– 612.
3018.
[30] Oh, S., Hoogs, A., Perera, A., Cuntoor, N.,
[21] Ilg, E., Mayer, N., Saikia, T., Keuper, M., Chen, C.-C., Lee, J. T., Mukherjee, S., Aggarwal,
Dosovitskiy, A., & Brox, T. (2017). Flownet 2.0: J., Lee, H., Davis, L. Et al. (2011). A large-scale
Evolution of optical flow estimation with deep benchmark dataset for event recognition in
networks, In Proceedings of the ieee surveillance video, In Cvpr2011. IEEE.
conference on computer vision and pattern
recognition [31] Kingma, D. P., & Ba, J. (2014). Adam: A
method for stochastic optimization. arXiv
[22] Schwarz, H., Marpe, D., & Wiegand, T. preprint arXiv:1412.6980.
(2007). Overview of the scalable video coding
extension of the h. 264/avc standard. IEEE
Transactions on circuits and systems for video
technology, 17(9), 1103–1120. [23] Ballé, J.,
Laparra, V., & Simoncelli, E. P. (2015). Density
modeling of images using a generalized
normalization transformation. arXiv preprint
arXiv:1511.06281. [24] Ballé, J., Laparra, V., &