THESIS ASSIGNMENT 1

AQSA ASLAM – MSCSF18M009

TOPICS:

 Optimal MTU Size for Video

 H.264/AVC & WebM/VP8 Comparison
 Metrics Overview - Subjective and Objective

JULY 29, 2019

SUBMITTED TO: DR. OMER NAWAZ
Thesis assignment 1

Contents
Topic No. 1 - Optimal MTU ............................................................................................................. 2
What is MTU? .............................................................................................................................. 2
Optimal MTU size for video......................................................................................................... 2
Conclusion ................................................................................................................................... 2
Topic No. 2 – Comparison between H.264/AVC & WebM/VP8 ..................................................... 3
What is H.264/AVC? .................................................................................................................... 3
What is WebM/VP8? ................................................................................................................... 3
Comparison ................................................................................................................................. 3
Conclusion ................................................................................................................................... 3
Topic No. 3 – Metrics (Subjective & Objective) .............................................................................. 4
What are metrics? ....................................................................................................................... 4
Subjective Metrics ....................................................................................................................... 4
Objective Metrics ........................................................................................................................ 4
Additional Metrics ....................................................................................................................... 5
References ...................................................................................................................................... 6

pg. 1
Thesis assignment 1

Topic No. 1 - Optimal MTU

What is MTU?
A maximum transmission unit (MTU) is the largest size packet or frame, specified in octets (eight-
bit bytes), that can be sent in a packet- or frame-based network such as the Internet.

Optimal MTU size for video

Optimal MTU size depends on many factors such as network type, transfer medium, network
condition/quality etc. We have to devise a trade-off among maximum transmitted information,
reduced response time and less bit error. According to [11], the maximum length of standard
Ethernet frame is 1518 bytes, after being removed the header and CRC, the rest of the maximum
length is 1500 bytes, which is the MTU.
If the application data is divided into a large number of “small fragments”, the actual amount of
data transferred is bigger than the data which is divided into a small number of “large fragments”.
The reason is that each fragment carries additional header information. A larger MTU also means
processing of fewer packets for the same amount of data.
However, longer data packet requires more transmission time, and in the process of
transmission, larger packets are more likely generating errors. Therefore, we should consider
various aspects to choose a suitable MTU value. If the application data is divided into a small
number of “large fragments”, the transmission of delay is longer than the delay of data which is
divided into a number of small fragments. The reason is that during the time when a large
fragment is transmitted, a few small fragments can be transmitted from the source, received by
the next jump, and further forwarded to the network [12]. The transmission delay is reduced, and
then the application response time is reduced. Therefore, there needs longer response time
when splitting same application data waiting to be transported into “large fragments” more than
“large fragments”.
The default setting of the MTU size on Peplink routers is 1440 to accommodate for the different
quality of ISP connections from around the world [13]. The standard set by IEEE802.3 specifies the
MTU of Ethernet is 1500 bytes. It is an interesting thing to note that, as the number of bytes in
the payload increases, the frame rate is decreasing. See that for an MTU of 1500 bytes on
payload, the frame rate has reduced to 812 frames per second. If you increase it above 1500,
frame rate would become less than 812[16]. Hence an optimal selection for MTU is 1500.
According to source [11], a research has been conducted using multiple hops to find optimal MTU.
Using 2 and 4 hops, the conclusion is same. Around 1500 MTU size, the upload response time is
minimum.

Conclusion
The optimal MTU is suggested as 1500 and is hence set standard by IEEE802.3.

pg. 2
Thesis assignment 1

Topic No. 2 – Comparison between H.264/AVC & WebM/VP8

What is H.264/AVC?
H.264 is an industry standard for video compression, the process of converting digital video into
a format that takes up less capacity when it is stored or transmitted. It is an ITU standard for
compressing video based on MPEG-4 and is very popular, especially for high-definition video.
AVC stands for Advanced Video Coding. Actually, it’s identical to H.264 so you can find it as H.264
or H.264/AVC.

What is WebM/VP8?
The WebM/VP8 video codec defines how the original video data is being encoded to achieve a
reasonable file size. VP8 is a so-called lossy codec (like H.264), which means that the compressed
video has a lower quality than the original video. VP8 does do a great job at that though (more
on quality below). [19] WebM is an audio-video format designed to provide a royalty-free, open
video compression format for use with HTML5 video.

Comparison
According to recent survey [4], number of files encoded in H.264 files is in 12 times higher than
number of files encoded in H.265. Another comparison shows that VP8 achieves good video
quality with the basic medium access control techniques of IEEE 802.11, whereas H.264/AVC
benefit significantly from appropriate mapping schemes between encoded video fragments and
IEEE 802.11e Access Categories.[17] Some conclusions drawn from various experiments [21] are:

 Low motion videos like talking heads are easy to compress, so you'll see no real difference.
 In another low motion video with a terrible background for encoding (finely detailed
wallpaper), the VP8 video retains much more detail than H.264.
 Moving to a higher motion video, VP8 holds up fairly well in a martial arts video.
 In higher motion videos, though, H.264 seems superior.
 In this very high motion skateboard video, H.264 also looks clearer where the VP8 video
looks slightly affected.
If you are looking into video calling solution, then VP8 is not a good choice. [20] The world of video
conferencing is revolving around H.264 and moving forward to H.265. The H.264 ecosystem is
larger than that of VP8 and is already in place for a lot of years now. H.264 is rigged with patents
while VP8 is patent free. It is yet to be seen if VP8 will remain patent free or not, but for now, it
is the cheaper alternative.

Conclusion
It has been observed the overall superiority of H.264/AVC in wireless networks but the VP8 codec
strongly gains ground in highly error-prone networks in terms of user satisfaction.[6] Keeping in
view all the facts, H.264 is better than VP8 in many ways but VP8 yet has to make its way. Skype
is using VP8 and it is already partially implemented in browsers and WebRTC was moving fairly
quickly. But H.264/AVC is the need of an hour.

pg. 3
Thesis assignment 1

Topic No. 3 – Metrics (Subjective & Objective)

What are metrics?
Metrics are measures of quantitative assessment commonly used for assessing, comparing, and
tracking performance or production. Generally, a group of metrics will typically be used to build
a system that analysts review on a regular basis to maintain performance assessments and
opinions. Assessment can be subjective or objective.
Objective assessment is done by using some calculations and evaluation methods usually pre-
defined. It is about deriving a possible result based on observations and measurements. In
Subjective assessment, we take input from user, either in the form of questioning right away or
creating surveys and providing multiple options to express the answer.
When defining metrics for QoE assessment there are many subjective factors which might bias
the user satisfaction towards a service, hence increasing the complexity of computing objective
metrics. As a consequence, there are notable research efforts in objectifying such factors in order
to have a measurable value of the user satisfaction in general.
The Mean Opinion Score (MOS) is metric typically used in QoE scenarios as we will detail later in
this section. There is however, a main drawback of using this metric alone, because it only
considers video and audio quality, ignoring other important aspects such as application
responsiveness or degree of interactivity, which are also relevant for a complete QoE assessment
framework. [25]

Subjective Metrics
Subjective metrics measure is to create a scale and regularly assess activity against that scale. If
you were assessing the comments in your customer-facing community, for example, you can
consider a like chart, a unary chart etc. One example is to create a MOS based scale with different
rating options like unary (like or dislike), five star and 100-point slider. In [22], we have observed
users’ preference for 5-star rating method rather than the other two.
There are other methods suggested by [24], and proven to provide more detailed and precise
conclusions than MOS. They include SOS (Standard Deviation of Opinion Score), Discrete 5-point
ACR scale etc.

Objective Metrics
The objective metrics take in account many factors and provide good results. Some of the
objective metrics are discussed below:

 Peak Signal to Noise Ratio (PSNR): it is a basic, yet important metric that assesses the
similarity between two different images. It uses the Full Reference QoE measurement
approach described above. PSNR computes the Mean Square Error (MSE) of each pixel
between the original and received images, represented in dB. Images with more similarity
will result in higher PSNR values.

pg. 4
Thesis assignment 1

 Structural Similarity (SSIM): the main drawback of PSNR is that it does not consider how
human perception works, hence in some cases it cannot detect some human perceptible
video disruptions. To address this shortcoming, SSIM combines luminance, contrast, and
structural similarity of the images to compare the correlation between the original image and
the received one. Similarly, PSNR, SSIM is also based on the Full Reference scheme.
 Video Quality Metric (VQM): improving the approaches described above, VQM detects
human perceivable artifacts on the images, by considering blurring, global noise, and block
and color distortions. This metric also uses the original video, hence using the Full Reference
QoE measurement approach.
 Mean Opinion Score (MOS): MOS was originally devised for audio streams, it combines
delays, perceived jitter at application layer, codec used for the communication, and packet
losses—also at application layer. In the area of video QoE assessment can be considered as a
meta-metric given that it considers values from other metrics to generate the final computed
user perception. The most used extension of MOS was proposed in [26] where the authors
propose a mapping between PSNR and MOS.

Additional Metrics
Some additional factors may also be considered and they also have an impact on user
assessment. These are also known as indirect metrics. Some of them are described as:

 Start-up time: this metric defines the time span since the user queries the system about a
specific content until he/she receives it. A classic example of this is the waiting time
experienced by a user from the time he/she clicks on a video link until the video is actually
reproduced on the user’s screen.
 Response time: this metric is an extension of the start-up time. It is defined as the time span
since the user issues a command or performs an action until it is acknowledged by the system
and an answer arrives at the user, e.g., when the user pauses a video. In video-conferencing
this metric is interpreted as interactivity degree, which determines the fluency of a
conversation among the conference participants.
 Delivery Synchronization: when several users are using a service, the content should be
received at the same time by all the participants. This is critical in on-line gaming where the
users need to react fast to the actions of others.
 Freshness: this metric specifies the time span from content generation until its reception by
the user. It is especially important in live video streaming, where the users want the content
as fast as possible (e.g., to celebrate a goal in a football match). This metric greatly affects
applications performing live streaming in P2P networks (P2PTV).
 Blocking: this metric is closely related the direct metrics because it models the jerkiness of
the video, normally caused by empty buffers on the receiver.

pg. 5
Thesis assignment 1

References
 [1]-TechTarget, Search Networking, Source:
“https://searchnetworking.techtarget.com/definition/maximum-transmission-unit”
 [2]-VCodex, An Overview of H.264 Advanced Video Coding, Source:
https://www.vcodex.com/an-overview-of-h264-advanced-video-coding/
 [3]-Visionary Solutions, Network AV Solutions, Source: http://www.vsicam.com/_faq/what-
is-h-264avc/
 [4]-“https://www.encoding.com/files/2016-Global-Media-Formats-Report.pdf”
 [5]-O. Nawaz, T. N. Minhas, and M. Fiedler, “Optimal MTU for real-time video broadcast with
packet loss #x2014; A QoE perspective”
 [6]-O. Nawaz, T. N. Minhas, and M. Fiedler, “QoE based comparison of H.264/AVC and
WebM/VP8 in an error-prone wireless network”
 [7]-O. Nawaz, T. N. Minhas, S. Khatibi and M. Fiedler, “The Effects of Additional Factors on
Subjective Quality Assessments”
 [8]-O. Nawaz, S. Khatibi and M. Fiedler, “Content Delight, Experience and Mood Effect on
Subjective Mean Opinion Score for Multimedia”
 [9]-T. Wiegand ; G.J. Sullivan ; G. Bjontegaard ; A. Luthra, “The VP8 video codec - overview and
comparison to H.264/AVC”
 [10]-SooYoung Jang ; Mikyong Han, “Optimal MTU Selection for Maximizing Goodput in
Wireless Networks”
 [11]-Yi-lin Guo,Yun Pan,Lei Cai, “OPNET-based analysis of MTU Impact on Application
Performance”
 [12]-H. Zhou, “The Study of OPNET Simulation and Its Application [D]”
 [13]- Peplink, Source: https://forum.peplink.com/t/how-to-determine-the-optimal-mtu-and-
mss-size/7895
 [14]-Network World, MTU Size Issues,
https://www.networkworld.com/article/2224654/mtu-size-issues.html
 [15]-Source: http://homenetworkadmin.com/what-is-mtu-size/
 [16]-Source: “https://community.cisco.com/t5/other-network-architecture/why-the-mtu-
size-is-1500/td-p/105418”
 [17]- Y.Yoon; M.Kim; S.Lee; B.Lee; S.J.Hyun; K.Lee “Performance analysis of H.264/AVC,
H.264/SVC, and VP8 over IEEE 802.11 wireless networks”
 [18]- Encoding.com, Knowledge Base, Source: ”https://help.encoding.com/knowledge-
base/article/webm-vp8-codec/”
 [19]- Tubular Insights, Source: “https://tubularinsights.com/google-vp8-webm/”
 [20]- Source: “https://bloggeek.me/webrtc-h264-vp8/”
 [21]-Streaming Media, Source:
“https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=67266”
 [22]-E.I.Sparkling and S.Sen, “Rating: how difficult is it?”

pg. 6
Thesis assignment 1

 [23]- Deepti Ghadiyaram and Alan C. Bovik, “Massive Online Crowdsourced Study of
Subjective and Objective Picture Quality”
 [24]- T.Hoßfeld, P.E. Heegaard, M.Varela, S.Mo¨ller, “QoE beyond the MOS: an in-depth look
at QoE via better metrics and their relation to MOS”
 [25]- R. Serral-Gracia, E. Cerqueira, M. Curado, M. Yannuzzi, E. Monteiro, X. Masip-Bruin “An
Overview of Quality of Experience Measurement Challenges for Video Applications in IP
Networks”
 [26]- Klaue, J., Rathke, B., Wolisz, A.: EvalVid, “A Framework for Video Transmission and
 Quality Evaluation”

pg. 7