Session No: CO2-6

Session Topic: Video Quality Assessment

DIGITAL VIDEO PROCESSING

(Course code: 19cs3278)
Prepared & Presented by: Dr. Lambodar Jena
Session Objective
To understand the
• Video quality assessment
• Video quality assessment metrics
Poll Question-01
• The technique to quantify the quality of a video signal is called
A. Video Enhancement
B. Video Restoration
C. Video Quality Assessment
D. All of the above
Key Concepts
• Video quality assessment
• Applications of quality assessment
• HVS Modeling Based Methods
• VQM(Video Quality Metric )
• Performance of VQA Algorithms
Video quality assessment (VQA)

• Video quality assessment (VQA) is the term used for techniques that
attempt to quantify the quality of a video signal as seen by a human
observer.

• VQA algorithms play a key role in almost every aspect of video

processing.
Applications
• Applications of quality assessment can broadly be categorized as
follows:
1. Quality Monitoring
2. Performance Evaluation
3. Optimization of video processing systems
1. Quality Monitoring
• Video quality is affected by numerous factors in a video
communication system such as compression, noise, errors,
congestion, and latency in networks.
• Quality monitoring of video can allow service providers to meet
their quality of service (QoS) requirements by changing resource
allocation strategies.
2. Performance Evaluation
• Systematic evaluation of both hardware and software video
processing systems that target human users are greatly facilitated by
reliable means of VQA.
• Perceptual quality of images, and video generated by video devices
(cameras, camcorders, displays, scanners, printers) and video
processing algorithms (implemented in graphics cards, set-top boxes,
encoders, super resolution, and enhanced displays) can be
automatically measured for competitive evaluation using VQA
systems.
3. Optimization of video processing systems
• Several video processing systems are designed by either specifying a
maximum distortion of the video signal or alternately, minimizing the
video distortion for a specified system configuration.

• Example applications include bit-rate allocation and rate-distortion

design of image and video communication systems. Significant
reduction in resource requirements can be achieved using perceptual
VQA techniques that closely match visual perception in video system
design.
HVS Modeling Based Methods

• The HVS (human visual system) derives information about the

environment from light that is either emitted, transmitted or
reflected from different objects in the environment.
HVS-based IQA system
• HVS-based IQA systems typically begin by preprocessing the signal to correct for
nonlinearities, since lightness perception is a nonlinear function of luminance.
• A filterbank in the “Linear Transform” stage decomposes reference, and distorted
(test) signals into multiple spatial frequency- and orientation-tuned channels in
an attempt to model similar processing by neurons in the early stages of the
visual pathway.
• The contrast sensitivity, luminance, and contrast masking features of the HVS
are then modeled to account for perceptual error visibility as a function of
luminance and contrast of the image patches.
• A space-varying threshold map is created for each channel describing local
spatiospectral error sensitivity, and is used to normalize the differences between
reference and test images.
• In the final stage, the normalized errors for all channels are pooled through a
suitable metric such as a weighted MSE, to generate a space-varying quality map.
Fig. : Block diagram of HVS-based IQA system.
HVS-based VQA system
• This system is identical to the generic HVS-based IQA system, but for
the inclusion of a block labeled “temporal filtering.”
• In addition to the spatial filtering stage in IQA systems depicted in the
“Linear Transform” block, VQA systems utilize a temporal filtering
stage in cascade, which is equivalent to filtering the videos using a
spatiotemporal filterbank that is separable along the spatial and
temporal dimensions.
HVS-based VQA system cont...
• The “temporal filtering” block typically models two kinds of temporal
mechanisms that exist in the early stages of processing in the visual
cortex, that are often modeled using linear lowpass and bandpass
filters applied along the temporal dimension of the videos.
• One of the first HVS-based metrics for video signals was known as the
moving pictures quality metric (MPQM).
Feature Based Methods
• Feature based methods for VQA utilize statistics and features
computed from the reference and test videos to predict the visual
quality of the test video.
• Feature based approaches form the backbone of several no reference
VQA systems.
VQM(Video Quality Metric )
• Another prominent VQA system was developed at the National
Telecommunications and Information Administration (NTIA) and is
known as the Video Quality Metric (VQM) or the NTIA General Model.
• VQM and its associated calibration techniques have been adopted as
a North American standard by the American National Standards
Institute (ANSI) in 2003.
• The International Telecommunication Union (ITU) has also included
VQM as a normative method for digital cable television systems.
VQM cont…
• The first stage of processing in VQM is the calibration of the reference
and test video sequences, which includes spatial and temporal
alignment, extraction of valid regions from the videos, gain and offset
correction.
• This stage is followed by
• Feature extraction (i.e Feature Computation)
• Spatiotemporally local quality parameter computation, and
• Pooling of local quality indices into a single quality score for the entire video
sequence.
Performance of VQA Algorithms
• The performance of many VQA systems has been evaluated on the
basis of various performance indicators.
• Indicators of the performance of a VQA system include metrics such
as
• Spearman rank order correlation coefficient (SROCC),
• Linear correlation coefficient (LCC), and
• Root mean square error (RMSE) etc.
Performance of VQA Algorithms
Comparison of the performance of VQA algorithms
Prediction Model SROCC LCC
Peak Signal - Noise Ratio 0.786 0.779
Proponent P2 (Sarnoff) 0.792 0.805
Proponent P5 (PDM from EPFL) 0.784 0.777
Proponent P7 (DVQ from NASA) 0.786 0.770
Proponent P8 (Swisscom) 0.803 0.827
Proponent P9 (Precursor to VQM from NTIA) 0.775 0.782
SSIM (weighting) 0.812 0.849
MOVIE 0.833 0.821

✓Spearman rank order correlation coefficient (SROCC)

✓Linear correlation coefficient (LCC)
 References
1. Alan_C_Bovik_The_Essential_Guide_To_Video_Processing_Academic_Press.pdf