You are on page 1of 152

POLITECNICO DI MILANO

Facoltà di Ingegneria dell’Informazione

Corso di Laurea Magistrale in Ingegneria delle Telecomunicazioni

POLITECNICO DI MILANO Facoltà di Ingegneria dell’Informazione Corso di Laurea Magistrale in Ingegneria delle Telecomunicazioni Transcoding

Transcoding H.264 Video via FFMPEG encoder

Relatore:

Prof. Paolo Giacomazzi

Tesina di Laurea di:

SHABNAM HASHEMIZADEHNAEINI Matr. 754702

Anno Accademico 2014 - 2015

Acknowledgments

I would like to thank my supervisor, Professor Paolo Giacomazzi, for his much appreciated guidance.

I would like to express my sincere appreciation and gratitude to all members of Ies Italia s.r.l for their incessant support. It has been a very graceful experience working with them for the last four years.

iii

Contents

Chapter 1

Introduction

  • 1.1 Interactive multimedia systems

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

1

  • 1.2 iES streaming project .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

2

  • 1.3 Thesis organization

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

3

Chapter 2

Interactive Multimedia delivery system

 
  • 2.1 Acquisition of input contents .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

6

Static files

  • 2.1.1 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

6

Live feed

  • 2.1.2 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

9

  • 2.1.3 Streaming architecture and technologies

.

.

.

.

.

.

14

  • 2.2 Streaming preparation and Streaming protocols

.

.

.

.

.

.

15

  • 2.2.1 Transcode

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

15

  • 2.2.2 Streaming Protocols

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

18

  • 2.3 Streaming Media Distribution

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

25

  • 2.3.1 IP Network

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

26

  • 2.3.2 Delivery Options for Multimedia

.

.

.

.

.

.

.

.

.

.

27

  • 2.3.3 Multimedia Access Networks

 

.

.

.

.

.

.

.

.

.

.

.

31

  • 2.4 Streaming media users

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

34

  • 2.4.1 Major media screen types

.

.

.

.

.

.

.

.

.

.

.

.

.

.

35

  • 2.4.2 Multimedia technology applications

.

.

.

.

.

.

.

.

.

37

Chapter 3

Content Preparation and Staging

 
  • 3.1 Digital Video

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

41

  • 3.2 Transcoding High Efficiency Video Coding

.

.

.

.

.

.

.

.

.

44

  • 3.2.1 General Settings And Concepts

 

.

.

.

.

.

.

.

.

.

.

.

44

  • 3.2.2 Video and playback artifacts

 

.

.

.

.

.

.

.

.

.

.

.

.

56

  • 3.3 Digital Audio

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

60

  • 3.3.1 Uncompressed Audio (PCM, Pulse Coding Modula-

 

tion)

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

60

  • 3.3.2 Compressed Audio Formats

.

.

.

.

.

.

.

.

.

.

.

.

.

61

  • 3.3.3 Audio Containers

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

62

 

v

Chapter 4

General Overview of iES streamer

 

4.1

udpxy

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

66

4.2

FFMPEG

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

69

4.2.1

Main features of x264 library

.

.

.

.

.

.

.

.

.

.

.

.

70

Chapter 5

Experimental evaluation and the specific goals

of this project

 

5.1

iES system .

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

88

5.1.1

iES front-end and back-end systems

.

.

.

.

.

.

.

.

89

5.2

5.1.2

Server and system design and development .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Mycujoo

.

.

.

.

.

.

.

.

.

98

111

5.2.1

Mycujoo front-end system

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

111

5.3

Other streaming based projects

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

128

Chapter 6

Conclusions and future work

List of Figures

  • 1.1 IP Traffic

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

2

  • 2.1 Open Digital Media Value Chain

.

.

.

.

.

.

.

.

.

.

.

.

.

.

5

  • 2.2 Comparison of DAS, SAN and NAS .

.

.

.

.

.

.

.

.

.

.

.

.

8

  • 2.3 DVB-S functional block diagram

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

10

  • 2.4 DVB-T functional block diagram

.

.

.

.

.

.

.

.

.

.

.

.

.

.

12

  • 2.5 A typical streaming system infrastructure

.

.

.

.

.

.

.

.

.

14

  • 2.6 Streamer Server: steps of streaming content preparation

.

15

  • 2.7 A generic compression system

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

15

  • 2.8 OSI Model

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

20

  • 2.9 HTTP over TCP/IP

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

22

  • 2.10 Real-Time Multimedia Traffic

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

25

IP Unicasting

  • 2.11 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

28

IP Multicasting

  • 2.12 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

29

Broadcasting

  • 2.13 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

30

IPTV Solution

  • 2.14 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

32

OTT Solutions

  • 2.15 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

33

Smart Clients

  • 2.16 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

36

  • 2.17 Examples of Set Top Box .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

37

  • 3.1 Human eyes are much less sensitive to color resolution than

 

the brightness resolution

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

42

  • 3.2 Video coding process . .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

43

  • 3.3 H.264 Vs MPEG2 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

44

  • 3.4 Group Of Pictures

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

52

  • 3.5 An example of frame sequence .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

52

  • 3.6 Container Format

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

55

  • 3.7 Buffering Artifact

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

56

  • 3.8 Frame drop artifact [15]

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

57

  • 3.9 Blocking artifact

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

58

3.10

Aliasing artifact [16]

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

58

vii

  • 3.11 Banding artifact [18]

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

59

  • 3.12 Gibbs Effect Artifact [19] .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

59

  • 4.1 IES Streamer

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

65

  • 4.2 FFMPEG directory structure

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

71

  • 4.3 Encoding techniques enabled by profile [39] .

.

.

.

.

.

.

.

.

73

  • 4.4 Diamond motion estimation search pattern .

.

.

.

.

.

.

.

.

80

  • 4.5 hexagon motion estimation search pattern

 

.

.

.

.

.

.

.

.

.

81

  • 5.1 iES Web portal

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

90

  • 5.2 iES Web portal - Web TV

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

91

  • 5.3 iES Web portal - Web Radio .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

91

  • 5.4 iES manager home page

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

92

  • 5.5 iES manager, Control Public screen

.

.

.

.

.

.

.

.

.

.

.

.

.

93

  • 5.6 iES manager, manage content and playlist

.

.

.

.

.

.

.

.

.

93

  • 5.7 iES manager, scheduling playlists

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

94

  • 5.8 iES manager, Audio, video, text and radio messages .

.

.

.

94

  • 5.9 iES manager, Upload contents

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

95

  • 5.10 iES Web portal - Web TV

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

95

  • 5.11 iES cabin - Select desired activity

.

.

.

.

.

.

.

.

.

.

.

.

.

.

96

  • 5.12 iES cabin - VOD

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

97

  • 5.13 iES cabin - Shopping

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

97

  • 5.14 iES Public Screen .

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

98

  • 5.15 UDPXY with seven active clients

.

.

.

.

.

.

.

.

.

.

.

.

.

.

101

  • 5.16 multiple qualities of a video are encoded, chunked into

segments, and requested by the steaming client/player

 

.

.

101

  • 5.17 Multiple output without filter

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

102

  • 5.18 How Domain Name Server works .

.

.

.

.

.

.

.

.

.

.

.

.

.

108

  • 5.19 iES Web portal – Internet access .

.

.

.

.

.

.

.

.

.

.

.

.

.

109

  • 5.20 iES Web portal – Internet purchase

.

.

.

.

.

.

.

.

.

.

.

.

.

110

  • 5.21 iES Payment terminal used by hostess

.

.

.

.

.

.

.

.

.

.

.

110

  • 5.22 iES Payment terminal – sell WiFi

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

111

  • 5.23 Mycujoo

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

112

  • 5.24 Mycujoo home page

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

114

  • 5.25 Mycujoo – Live events

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

115

  • 5.26 Mycujoo – A running live event

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

116

  • 5.27 Mycujoo – TV Information

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

117

  • 5.28 Mycujoo – Channel setting .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

117

  • 5.29 Thunderbolt, IEEE 1394, HDMI, HD-SDI, USB 3

.

.

.

.

.

119

  • 5.30 OBS – Stream configuration

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

120

  • 5.31 OBS – Video configuration

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

121

LIST OF FIGURES

  • 5.32 OBS – Audio configuration

 

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

122

Sportube .

  • 5.33 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

128

Live Reporter

  • 5.34 .

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

129

 

ix

List of Tables

  • 2.1 Major differences between SAN and NAS .

 

.

.

.

.

.

.

.

.

.

9

  • 2.2 Differences between OTT and IPTV

 

.

.

.

.

.

.

.

.

.

.

.

.

34

  • 3.1 Video resolution and bit rate for standard formats .

.

.

.

.

42

  • 3.2 Common Aspect Ratios

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

51

xi

Sommario

Ogni giorno, un numero elevato di video sono caricati online su siti web di

video hosting come Youtube e Vimeo dove le persone possono guardare

film interi immediatamente.

Tuttavia, se un video viene girato in 4k RAW su una videocamera pro-

fessionale e si intende offrirne la visione su Youtube, sarebbe necessario

scalarlo poich`e 30 minuti comporterebbero un costo di circa 63 Gigabytes

di contenuto. Necessita di essere compresso fino ad una risoluzione ammissi-

bile. L’obiettivo `e quello di minimizzare la perdita di qualit`a nell’immagine

mantenendo le dimensioni del file gestibili. Questo processo di riformattare

il contenuto che deve essere trasmesso su Youtube `e chiamato transcoding.

Perci`o per mezzo del transcoding, una codifica digitale viene convertita

in un’altra ed `e necessaria quando un particolare dispositivo target non

supporta il formato o non ha capacit`a di memoria sufficiente per supportare

la grandezza del file o non ha una CPU con potenza sufficiente.

Il codec usato per la compressione in video `e tipicamente H.264, uno

standard che fornisce video ad alta definizione a bit rate sostanzialmente

pi`u bassi. La Libreria x264 `e usata per codificare H.264/MPEG-4 AVC,

sottende alcune delle operazioni di streaming di maggior profilo sul web,

inclusi Youtube, Vimeo a Hulu. Quando compilata con lo strumento

FFmpeg, `e in grado di realizzare compressioni di alta qualit`a a velocit`a

relativamente elevate. FFmpeg `e un importante strumento di encoding

da pi`u di dieci anni.

E ` una potente libreria multi funzione open-source

con un’ampia gamma di comandi da terminale, che pu`o essere utilizzata

efficacemente in combinazione all’utilizzo di altri programmi e a web server

ad altre prestazioni per trasmettere contenuto sul web.

Sulla base di queste promesse, lo scopo di questa tesi `e di presentare progetti

basati sui media con la necessit`a essenziale di effettuare il transcoding e di

studiare il codec video H.264 e lo strumento di codifica FFmpeg.

Parole chiave: codec, transcoder, ffmpeg, h.264

xiii

Abstract

Every day, a large number of videos are uploaded to the video hosting

websites such as Youtube and Vimeo where people can watch entire movies

immediately.

However if a video is shot in 4k RAW on a professional camera and is

intended to be viewed on a website like Youtube, it would need to be

scaled down since thirty minutes would amount to roughly 63 Gigabytes

of content. It needs to be compressed down to the allowable resolution.

The goal is to minimize loss in picture quality while still keeping the file

size manageable. This process of reformatting the content to be streamed

on Youtube is called transcoding. So by transcoding one digital encoding

is converted to another and it is needed when a particular target device

does not support the format or does not have enough storage capacity to

support the file size or sufferes not enough powerful CPU.

The codec used for compression in video is typically H.264, a standard for

providing high definition video at substantially lower bit rates. The x264

Library is used for encoding H.264/MPEG-4 AVC, undergirds some of the

most high profile streaming operations on the web, including YouTube,

Vimeo and Hulu. When compiled with FFmpeg tool, it is capable to

produce high quality compression at relatively high speeds. FFmpeg has

been an important encoding tool for more than a decade. It is a powerful,

multi-purpose open-source library with a wide range of command lines,

which can be effectively utilized in conjuction with programming experience

and high performance web servers to stream content to the web world.

Given these introductions, the purpose of this thesis is to go through media

based projects with essential need of transcoding and to study H.264 video

codec and FFmpeg encodig tool.

Keywords: codec, transcoder, ffmpeg, h.264

xiv

Chapter 1 Introduction

This chapter aims to give an introduction to the context of this thesis,

illustrating the motivations and problems which are behind this work

and to introduce the thesis objectives. The structure of this work is also

reported.

  • 1.1 Interactive multimedia systems

Nowadays, interactive multimedia systems are a reality that affect many

aspects of human life. The day by day development of related softwares

and intelligent components as theoretical aspects also helps the practical

systems, which people encounter daily. There are number of fields where

multimedia projects could be of use, as business, education, entertainment,

home and public places. As you can see in Figure 1.1, in 2016, it is

estimated that 76% of the IP traffic will be about videos [1].

As the demand for multimedia and digital content is constantly grow-

ing, new interactive multimedia systems are been developed. Interactive

multimedia is any computer-delivered electronic system that allows the

user to control, combine and manipulate different types of media, such

as text, sound, video, computer graphics, and animation. It integrates

computer, memory storage, digital (binary) data, telephone, television,

and other information technologies and shifts the user’s role from observer

to participant and are considered the next generation of entertainment

engagement systems.

On the other hand, the explosive growth of the Internet and mobile

computing introduces two main problems in distributed multimedia ap-

plications. The first problem is heterogeneity of client devices and their

network connections. The client devices may vary from desktop PCs,

1

Chapter 1

Chapter 1 Figure 1.1. IP Traffic notebook computers, PDAs to mobile phones, which their capabilities also

Figure 1.1. IP Traffic

notebook computers, PDAs to mobile phones, which their capabilities

also vary a lot, including screen size, color depth and processing power.

Furthermore, they may connect to the Internet via different networks, such

as wired LAN, wireless LAN or wireless WAN.

The second problem is mobility of clients. The clients may be moving

while they are accessing multimedia streams. It may cause a problem

because the network connections may change from time to time, ranging

from a very good network to a congested network. The two problems

described above make it difficult for a multimedia server to provide a

streaming service which is appropriate for every client in every situation.

A solution to the problems above, which is presented in this thesis, is by

converting multimedia streams to the appropriate format on-the-fly. The

converting process is also known as transcoding, which means converting

multimedia streams from one format to another format.

The purpose of the project described in this thesis, is to develop a

prototype multimedia system infrastructure for transcoding multimedia

stream and transmit them via a suitable network to the device of end-user.

  • 1.2 iES streaming project

The project is based on different virtual servers as DB-DNS (Database-

Dynamic Name server) Server, IAC (Internet Access Controller) Server,

Introduction

ADS(Advertisement) Server, Web Portal Server, VoIP Server. However the

core of the system is Streamer Server which is the subject of this thesis.

In fact we replaced costly American brand encoder/transcoder servers

with low ability of remote control, with our iES streamer server. The

streamer server with some light modification is also used for web based

multimedia projects as Sportube Tv and Mycujoo and etc. However to

be brief, this thesis covers just iES system, Sportube Tv and Mycujoo as

practical examples.

Sportube Tv is the sport Web TV which broadcasts Sport Events

streams LIVE and On Demand and you can find more details in http:

Mycujoo, as it is explained in the following Web page https://new.

mycujoo.tv, connects football through video. Unique football TVs will

allow billions of fans to watch, share and discover live football and on-

demand highlights, goals and unique moments.

  • 1.3 Thesis organization

In general, this thesis consists of two main parts, concept part and design

part which are organized in six chapters, including the present one, which

discusses the background as well as the context of this thesis.

Chapter 2 provides an overview of Interactive Multimedia Delivery

Systems, focusing on the acquisition, preparation, distribution and con-

sumption.

Chapter 3 deepens the aspect of content preparation and staging pre-

senting one of the key aspect of an interactive multimedia delivery system,

that is the encoding/transcoding needed for the encapsulation of data

packets.

Chapter 4 introduce IES Italia company and explores design an imple-

mentation of iES multimedia project and iES streamer server in detail.

Chapter 5 presents practical results of the output and efficiency of iES

streamer server.

Chapter 6 gives summary of this thesis as a conclusion and some outlooks

for the future.

Chapter 2 Interactive Multimedia delivery system

Interactive Multimedia delivery system Interactive multimedia is any

computer-delivered electronic system that allows the user to control, com-

bine and manipulate different types of media, such as text, sound, video,

computer graphics, and animation. It integrates computer, memory stor-

age, digital (binary) data, telephone, television, and other information

technologies and shifts the user’s role from observer to participant. They

are considered the next generat