Detailed Test Site Specifications and First Set-Up: Deliverable D5.2

Detailed Test Site specifications
and First set-up

Deliverable D5.2
GLOBAL ITV ID: GLOBALITV-D5.2-TestSiteSpecificationAndFirstSetUp

Version: 21
Deliverable number: D5.2
Authors: Jean-Paul Bernoux (TDF), Fernanda Chocron Miranda, Marianne
Kogut Eliasquevici, Marcos César da Rocha Seruffo, Jane Aparecida
Marques, Carlos Natalino, Maria Ataide Malcher (UFPA).
Contributors: Christian Keimel (IRT), Jaime Mancebo (A- CING), Gabriel Trevisan
(Unicamp).
Internal reviewers: Christian Keimel (IRT)
Work Package: WP5
Task: T5.1
Nature: R – Report
Dissemination: PU – Public
Status: Living
Delivery date: 10.07.2015
Version of
2015-07-10 D5.2 Detailed Test sites specification and First set-up
Version and controls:
Version Date Reason for change Editor

1 2014-02-01 First draft Christian Keimel (IRT).
2 2015-01-12 Table of content JP. Bernoux (TDF).
3 2015-02-13 First contribution JP. Bernoux (TDF).
4 2015-02-19 Contributions from UNESP Fábio Cardoso (UNESP).
5 2015-03-01 Contributions from UFPA Marcos Seruffo, Marianne
Eliasquevici, Fernanda
Chocron, Maria Ataide
Malcher, Jane Marques and
Carlos Natalino (UFPA).
6 2015-03-03 Contributions from TDF JP. Bernoux (TDF).
7 2015-03-05 Contributions from A-CING Jaime Mancebo (A-CING).
8 2015-03-16 Contributions from IRT, Unicamp and Christian Keimel (IRT), Ga-
TDF briel Trevisan (Unicamp),
JP. Bernoux (TDF).
9 2015-05-07 Contributions from TDF JP. Bernoux (TDF).
10 2015-04-08 Contributions and improvements from Marianne Eliasquevici,
UFPA. Fernanda Chocron, Jane
Marques, Carlos Natalino
and Marcos Seruffo (UFPA).
11 2015-05-11 Comments from IRT Christian Keimel (IRT).
12 2015-05-12 Contributions from A-CING Jaime Mancebo (A-CING).
12 2015-05-14 Contributions from UFPA Fernanda Chocron and
Marianne Eliasquevici
(UFPA).
13 2015-05-29 Review from TDF for 6th GLOBAL ITV JP. Bernoux (TDF).
Consortium meeting
14 2015-06-05 Final contributions from A-CING (sub- Jaime Mancebo (A-CING).
jective tests)
15 2015-06-11 Contributions from TDF (First Set-up JP. Bernoux (TDF).
section)
16 2015-06-12 Review from UFPA and format im- Marianne K. Eliasquevici,
provements Jane Marques and Fernanda
Chocron Miranda (UFPA).
17 2015-06-22 Incorporating comments from first inter- Christian Keimel (IRT).
nal review
18 2015-06-23 Final review from TDF JP. Bernoux (TDF).
19 2015-06-26 Final review from UFPA Marianne K. Eliasquevici,
Maria Ataide Malcher,
Fernanda Chocron Miranda
and Marcos Seruffo (UFPA).
JP.Bernoux (TDF), JR Tume
20 2015-07-03 Unicamp contribution
Ruiz (Unicamp).
21 2015-07-10 Final formatting Christian Keimel (IRT)
page ii
Version of
Acknowledgement: The research leading to these results has received funding from the European Un-
ion's Seventh Framework Programme (FP7/2007-2013, call FP7-ICT-2013-10.2) under grant agreement
n° 614087 and from the Brazilian Conselho Nacional de Desenvolvimento Científico e Tecnológico
(CNPq) under grant CHAMADA MCTI/CNPq Nº 13/2012.
Disclaimer: This document does not represent the opinion of the European Community, and the Euro-
pean Community is not responsible for any use that might be made of its content.
This document contains material, which is the copyright of certain GLOBAL ITV consortium parties, and
may not be reproduced or copied without permission. All GLOBAL ITV consortium parties have agreed to
full publication of this document. The commercial use of any information contained in this document may
require a license from the proprietor of that information.
Neither the GLOBAL ITV consortium as a whole, nor a certain party of the GLOBAL ITV consortium
warrant that the information contained in this document is capable of use, nor that use of the information
is free from risk, and does not accept any liability for loss or damage suffered by any person using this
information.
page iii
Version of
Executive Summar y
This document summarises the specification for the test setup and describes how first interoperability
tests will be performed, as part of the Task 5.1 (Coordination and set-up of the test sites) from WP5, in
addition to all other tests planned to be carried out during the development of the project. WP5 is respon-
sible for preparing and organizing the objective and subjective tests. These tests are primarily based on
the objectives and the organisation of the project, the scenarios and functional and interoperability re-
quirements, prototypes of platform specifications and of applications, the role of partners as collaborators
of the tests, dissemination actions and the establishment of GLOBAL ITV Council. All these elements are
important to validate the development of the GLOBAL ITV architecture and to provide essential feedback
for refining the specifications of the platform and the development of applications, if necessary. In this
sense, a detailing of tests needs to be developed, since in deliverable 5.1 (Testing and evaluation plan)
only a map was presented which defines the scope and objectives to be evaluated, the requirements to
be validated, the strategies and resources to be employed in the activities.
Section 1 provides an introduction; Section 2 approaches the functional requirements that will be consid-
ered in the tests proposed by WP5; Section 3 describes the objective measurement test sites specification
from the partners involved with the tests, and explains how the tests will be developed in accordance with
the elements that make up the GLOBAL ITV platform and its derivations (set-top-box, play-out, transmis-
sion and reception aspects of broadcast and broadband); Section 4 presents the subjective measurement
test sites specification proposed by UFPA and A-CING; Lastly Section 5 pictures an example of the test
bed used by TDF, before Section 6 concludes with final remarks.
page iv
Version of
Table of Contents
1 Introduction .................................................................................................................................... 1
2 Functional requirements ............................................................................................................... 2
3 Objective measurement test sites specification ........................................................................ 5

3.1 Generic platform description ....................................................................................................... 5
3.2 TDF platform ............................................................................................................................... 6
3.3 IRT platform ................................................................................................................................ 8
3.4 Unicamp platform ...................................................................................................................... 10
3.5 UFPA platform .......................................................................................................................... 14
4 Subjective measurement test sites specification..................................................................... 18

4.1 UFPA platform .......................................................................................................................... 18
4.2 A-CING platform ....................................................................................................................... 22
5 First set-up ................................................................................................................................... 29
6 Final remarks................................................................................................................................ 32
Glossary ............................................................................................................................................... 33
References ........................................................................................................................................... 36
List of Figures ...................................................................................................................................... 37
List of Tables ....................................................................................................................................... 38
page v
Version of
1 Introduction
The objective of the GLOBAL ITV project is to define an interoperable interactive TV system between
Brazil and Europe. This implies the definition of new mechanisms for application signalling and transpor-
tation to allow the simultaneous broadcasting of Ginga and HbbTV applications. This corresponds to the
coexistence and interoperability phases of the project as defined in WP2 and WP3. WP3 has specified
the technical mechanisms. These mechanisms will be proposed to standardisation if successful. In order
to achieve that goal, a play-out platform is being designed to the GLOBAL ITV specification, and a set-
top-box is being developed.
In WP4, GLOBAL ITV has defined the development of different applications with two main objectives:
firstly, it will allow the study and validation of the performance of the GLOBAL ITV specification and plat-
form. Secondly, it will allow verifying the acceptance and the interest of viewers in hybrid TV services.
WP5 is responsible for the different tests that will confirm the feasibility of the GLOBAL ITV specification
and the performance of the developed play-out and set-top-box. Deliverable D5.1 - Testing and evaluation
plan describes the different tests that will be performed, but not with a deep level of detail. This document,
in turn, describes the different platforms available in the project and the tests that will be performed e.g.
schedule and environments, equipment needed, procedures, etc., taking into account the functional and
interoperability requirements established on WP2.
The tests have been divided in two main categories:
• Objective tests: to check the behaviour of the different GLOBAL ITV components, the GLOBAL
ITV platform and their interoperability with existing platforms and receivers. The tests will validate
the technical feasibility of concepts and specifications and have been divided in three categories:
o Conformance test: to validate the equipment
o Integration test: to validate equipment in an end-to-end platform including network equip-
ment already in operation
o Interoperability test: to validate the platform and application architecture of GLOBAL ITV
with existing receivers.
This will lead to the verification of the requirements listed by WP2
• Subjective tests: will be carried out to evaluate acceptance of the developed applications and
services in the project framework. It is necessary to validate the platform via end-user acceptance
tests, and identify user behaviour regarding the applications and the prototypes. The goal is to
observe and measure user experience at the contact time and platform use, as well as to verify
the operation of its services and applications, even those which have already been tested in tech-
nical simulations performed during its development phase.
This document describes the different platforms available in the project and the tests that will be performed
(different environment of tests, equipment needed, procedures for data collection, etc.), taking into ac-
count the functional and interoperability requirements established in WP2. The proposed tests will be
carried out both in Brazil and in Europe in three specific phases:
Phase 1) First pilot block of tests using playout and STB prototype of the project with HbbTV and
Ginga applications that are currently in operation in Europe and Brazil. The results of these
tests will be reported in the D5.3 – Validation Plan and report of first interoperability tests.
Phase 2) Final objective tests will be carried out using playout, STB and applications of the project if
available. The results of these tests will be reported in the D5.4 – Running test site.
Phase 3) Subjective test with end users will be carried out using playout, STB and apps of the project.
The results of these tests will be reported in the D5.5 – User test results and acceptance
report.
page 1
Version of
2 Functional requirements
According to D2.3 – Functional and interoperability requirements [1], the functional requirements deter-
mine the functions or services that the system should be able to perform and/or provide, i.e., how it should
or should not react to certain commands or situations. Functional requirements for the proposed GLOBAL
ITV system need therefore to consider the relation between system’s technical robustness and its capa-
bilities to support the demands imposed by general scenarios and the discussed use cases.
Although there is an expectation of the system’s perfect operation, criteria of relevance are considered in
order to prioritize what are the main challenges to overcome in terms of future prototyping and testing.
Considering this, the tests proposed by WP5 should check if the specified requirements are being met
and if so, how.
Table 2 presents the Functional Requirements listed by WP2 indicating two levels of priorities: (i) first
approach: it will guarantee the right execution of Ginga-NCL or HbbTV 1.5 applications in the ISDB-Tb
and DVB standards; and (ii) second approach: with lower priority, it allows the GLOBAL ITV system to
execute applications beyond the Ginga and HbbTV iTV platform.
Table 1: Functional and interoperability requirements

Functional require- Priority
No. Description
ment Level
GLOBAL ITV STB should support the execution of Ginga-NCL applications accord-
Ginga-NCL application ing to standard ABNT NBR 15606-2:2011 - Digital terrestrial television – Data coding First ap-
#1
support and transmission specification for digital broadcasting –Part 2: Ginga-NCL for fixed proach
and mobile receivers – XML application language for application coding.
HbbTV 1.x application GLOBAL ITV STB should support the execution of HbbTV 1.5 applications according First ap-
#2
support to ETSI TS 102 796, Rev. 1.2.1 proach
GLOBAL ITV STB should be ready to receive Ginga-NCL application signalling and
Ginga-NCL application
handling commands following the standard ABNT NBR 15606- 3:2012 - Digital ter- First ap-
#3 signalling and handling
restrial television — Data coding and transmission specification for digital broadcast- proach
commands support
ing — Part 3: Data transmission specification.
HbbTV 1.x application
GLOBAL ITV STB should be ready to receive HbbTV 1.5 application signalling and First ap-
#4 signalling and handling
handling commands following ETSI TS 102 809 V1.2.1 proach
commands support
GLOBAL ITV STB should be ready to receive Ginga-NCL application content follow-
Ginga-NCL application ing the standard ABNT NBR 15606-3:2012 - Digital terrestrial television — Data cod- First ap-
#5
receiving ing and transmission specification for digital broadcasting — Part 3: Data transmis- proach
sion specification.
HbbTV 1.x application GLOBAL ITV STB should be ready to receive HbbTV 1.x application content follow- First ap-
#6
receiving ing the ETSI TS 102 796, Rev. 1.2.1 proach
Set-top-box ISDB-Tb GLOBAL ITV STB should be ready for ISDB-Tb system according to standard ABNT First ap-
#7
support NBR 15604:2007 - Digital terrestrial television - Receivers. proach
Set-top-box DVB sup- GLOBAL ITV STB shall support a terrestrial DVB system (DVBT)according to stand- First ap-
#8
port ard ETSI EN 300 7443 proach
GLOBAL ITV STB should provide an interactive channel for Ginga-NCL applications
Ginga-NCL Interactive First ap-
#9 according to standard ABNT NBR 15607-1:2011 - Digital terrestrial television - Inter-
Channel support proach
active channel. Part 1 Protocols, physical interfaces and software interfaces.
HbbTV 1.x Broadband GLOBAL ITV STB should support broadband connections for HbbTV 1.5 applica- First ap-
#10
Support tions according to ETSI TS 102 796, Rev. 1.2.1 proach
GLOBAL ITV Playout should be ready to encapsulate Ginga-NCL and HbbTV 1.5
application metadata, handling commands and content in a MPEG2-TS transport
stream for both ISDB-Tb and DVB systems, according to standards ABNT NBR First ap-
#11 Playout
15606-3:2012 - Digital terrestrial television — Data coding and transmission specifi- proach
cation for digital broadcasting — Part 3: Data transmission specification, and ETSI
TS 102 809 V1.2.1.
GLOBAL ITV Applications should be available in servers in the Internet cloud and
First ap-
#12 Application Availability also transmitted via broadcast by the playout specified in the functional requirement
proach
#11.
HTML5 Native applica- Second
#13 GLOBAL ITV STB should support the execution of HTML5 native applications
tions support approach
Convergence of differ- GLOBAL ITV STB should support different iTV platforms beyond GingaNCL and Second
#14
ent iTV platforms HbbTV 1.x. approach
page 2
Version of
It is important to verify the conformance with these requirements, but we have to keep in mind the capa-
bilities of the GLOBAL ITV demonstrator, consisting of play-out and set-top-box. Therefore it will not be
possible to carry out comprehensive testing, but only to perform tests at a first level that will enable to
validate the GLOBAL ITV concepts in principle.
• Functional Requirement #1 Ginga NCL application support

The requirement states: GLOBAL ITV STB should support the execution of Ginga NCL applica-
tions according to the relevant standards.
This will be covered by the measurement described in Section 3.4. A multiplex including Ginga-
NCL applications will be generated and decoded by the Set-up box.
• Functional Requirement #2 HbbTV1.x application support.

The requirement states: GLOBAL ITV STB should support the execution of HbbTV applications
according to the relevant standards.
Tests will be carried out with representatives of the most common HbbTV applications. TDF will
utilize an application portal that was developed by TDF including VoD, catch-up, quiz, voting, or
with commercial services.
• Functional Requirement #3 Ginga-NCL application and handling command support
The requirement states: GLOBAL ITV STB should be ready to receive Ginga-NCL signaling and
handling commands according to the relevant standards.
Tests described in Section 3.4 include tests with Ginga/ISDB-Tb (and GLOBAL ITV) signaling to
cover this requirement.
• Functional Requirement #4 HbbTV 1.x application and handling command support

The requirement states: GLOBAL ITV STB should be ready to receive HbbTV signaling and han-
dling commands according to the relevant standards.
The measurement described in Section 3.2.2 includes application signaling measurement all
along the broadcasting platform.
• Functional Requirement #5 Ginga-NCL application receiving
This requirement states: the GLOBAL ITV STB should be ready to receive Ginga-NCL according
to the relevant standards.
Tests described in Section 3.4 includes tests with Ginga applications to cover this requirement.
• Functional Requirement #6 HbbTV1.x application receiving

This requirement states: the GLOBAL ITV STB should be ready to receive HbbTV according to
the relevant standards.
End-to-end tests will be carried out with test signals described in Section 3.2.1. It includes HbbTV
applications.
• Functional Requirement #7 ISDB-Tb set-top box support
The requirement states: the GLOBAL ITV STB should be ready to support ISDB-Tb system.
Tests described in Section 3.4 include tests with ISDB-Tb signals to cover this requirement.
• Functional Requirement #8 DVB set-top box support

The requirement states: the GLOBAL ITV STB should be ready to support DVB system.
This requirement will be covered by the end-to-end test carried out with GLOBAL ITV applications.
page 3
Version of
• Functional Requirement #9 Ginga-NCL Interactive Channel support
The requirement states: the GLOBAL ITV STB should provide an interactive channel support
according to the relevant standards. Tests described in Section 3.4 include tests with Ginga ap-
plications to cover this requirement.
• Functional Requirement #10 HbbTV 1; x Broadband support

The requirement states: the GLOBAL ITV STB should support broadband connections for HbbTV
1.5 applications according to ETSI 102 796 rev 1.2.1.
This requirement will be covered by the end-to-end test carried out with test signals described in
Section 3.2.1. They include broadband applications.
• Functional Requirement #11 Playout

The requirement states: GLOBAL ITV play-out should be ready to encapsulate Ginga-NCL and
HbbTV applications metadata, handling commands and content in a MPEG2-TS for both ISDB-
Tb and DVB-T according to the relevant standards.
This will be covered by the end-to-end test GLOBAL ITV specific play-out will be used for that.
• Functional Requirement #12 application availability

The requirement states: GLOBAL ITV applications should be available in servers in the internet
cloud and also transmitted via broadcast by play-out specified in the functional requirement #11.
Tests will be carried out with commercial applications that will be accessible via the open internet.
Some application will have a broadcast component that will be carrouseled.
• Functional Requirement #13 HTML5

The requirement states: GLOBAL ITV should support the execution of HTML5 native applications.
This is a second level priority. If a HTML5 application is available in the project, it will be used to
test this requirement.
During the description of the tests presented in this document it will therefore be mentioned which of the
functional requirements are met, as well as when the tests will be carried out.
page 4
Version of
3 Objective measurement test sites specification
Objective measurements have several goals:

• Validation of the feasibility of GLOBAL ITV concepts: end-to-end platform will be deployed includ-
ing play-out and with the GLOBAL ITV receiver. The validation will be carried out in both interac-
tive systems HbbTV and Ginga corresponding to the coexistence and interoperability phases. An
important point will be the verification that the requirements defined by WP2 are met.
• Validation of the interoperability with the equipment deployed in the network: this condition is
mandatory if GLOBAL ITV concepts should be adopted by the broadcaster and network opera-
tors.
• Non-disturbance of the different receivers. This condition is mandatory to adopt GLOBAL ITV by
different standardisation bodies and local authorities.
Two types of receivers will be available fulfilling GLOBAL ITV specifications:
• one will be equipped with an ISDB-T tuner;
• for the DVB context, a DVB-C STB will be used. Although DTT networks are the main target of
the project, the GLOBAL ITV specification can be applied to DVB T/C/S standards. For practical
implementation issues, it was not possible to implement the GLOBAL ITV stack on a DVB-T re-
ceiver.
3.1 Generic platform description

The following diagram gives the overall description of a traditional interactive TV system. Since DVB and
ISDB standards on one hand, and Ginga and HbbTV standards on the other hand are closely related, the
same pattern can be applied to the European and Brazilian systems.
A multiplex is composed of television services each of them having different components: video, audios,
possibly data services such as teletext and audio description (for blind people), and lastly interactive
services. No components except interactive TV services will be impacted by GLOBAL ITV specifications.
Interactive services have three main components; the data carousel or broadband connection to convey
the data, stream events used to trigger an event and the AIT that is used to signal the application. The
system is being organised by signalling tables such as the PMT.
page 5
Version of
Figure 1: Block diagram of Interactive TV platform
3.2 TDF platform

TDF will integrate a platform that is representative of the platforms in operation on the French DTT net-
works. It is composed of:
• National head-end: it is responsible of the encoding, multiplexing and signalling of services that
have to be broadcast over a country.
• Regional head-end. It receives a “National” multiplex and add local program or substitute a na-
tional services by a local one. Services can include audio, video and data services, but some
cases it is just a component e.g. an HbbTV application. The HbbTV services will include AIT, data
carousels and Stream-event. HbbTV specific insertion required a special equipment under devel-
opment. It will be inserted in the platform if it is available.
• A transmitter including a modulator and a low power amplifier.
• A reception site: TV receiver and broadband networks (consumer internet connection and high
quality network).
The tests and procedures that will be performed are similar to those requested by the French Regulatory
body when a new system is launched on the French DTT networks. TDF will use the experience acquired
during the HbbTV launch in 2012.
This will include:
• DVB services (audio/video and data services) source will be either on-air programs, with high
quality receiver or video files generated and streamed locally.
• Modulation facilities using a multi-standard modulator (DVB-T). Most of the tests carried out by
TDF will be to validate the GLOBALITV concepts in the DVB world, and TDF is mainly concerned
by DTT networks. Tests will be carried out in DVB-T modulation according to the French regula-
tion. But since GLOBAL ITV receiver will be equipped with ISDB-T and DVB-C tuners, the plat-
form will also be able to generate DVB-C signals.
page 6
Version of
• Measurement equipment (multiplex analyser).

• Service platform. This platform is a duplication of HbbTV TDF services. Platforms provided by
partners will be used if necessary for the GLOBAL ITV.
On the receiver side, measurement will be carried out using the DVB-C GLOBAL ITV receiver for the
GLOBAL ITV services management. Although the DTT network is the main type of network targeted, it
was not possible to develop a DVB-T receiver in the framework of the project but it was possible to adapt
a DVB-C STB. DVB-C and DVB-T systems are quite similar concerning signaling and interactivity pro-
cessing. This receiver will be used to validate GLOBAL ITV specification and services in the European
context.
Figure 2: Block diagram of TDF platform
The above diagram presents the platform that will be used by TDF. The standard equipment in operation
on DTT network managed by TDF are in orange. A major constraint will be the interoperability between
the used equipment, in particular if all components are capable to process the GLOBAL ITV compliant
transport stream without any problems.
3.2.1 Test signals

The test signals will be a multiplex of:
• TV programs locally generated with HbbTV services and GLOBAL ITV signaling and services.
The signaling will allow testing the compatibility with different HbbTV services. This last will be
provided by:
o TDF HbbTV services platform (quiz, Catch-up, Video on demand, news) accessed via a
LAN. This platform will also be connected to internet to be accessible to the other part-
ners.
o Commercial services available in France. They will be accessible via the Internet.
o Applications developed by GLOBAL ITV partners in the framework of WP4
page 7
Version of
• Commercials on-air TV programs i.e. on-air programs respecting existing standards (DVB stand-
ard with French regulation profiles, the HbbTV services available in France).
3.2.2 Measurement
3.2.2.1 Integration test
• Signaling and protocol
A major result of GLOBAL ITV is the definition of application signaling. An important task will be
to verify signaling generated in the head-end is valid (unchanged) through the different head-end
equipment including multiplexer and remultiplexer. Different types of equipment (e.g multiplexer
of different manufacturer used by TDF) will be tested. Measurement using DVB and HbbTV pro-
tocol analyzers will be carried out more especially after the remultiplexing process.
Real time and off-line analyzers will be used for that purpose. Equipment specialized in HbbTV
signaling monitoring will be available to the project.
• Application rendering
The performance of GLOBAL ITV receiver will be verified in the European context. Since the
receiver will be equipped with a cable tuner, the RF modulation will be DVB-C. Different interactive
services will be tested using the test signals described section; 2.2.1
3.2.2.2 Interoperability with receivers

DTT is the main means of television distribution in France. Tests will be made with terrestrial receivers.
Commercial TV signals and GLOBAL ITV signals are multiplexed. The RF socket is fed with these signals.
The behavior of the different TV sets is checked during different phases:
• Set-up of the different TV set: are the different DVB services detected and correctly installed? Are
the GLOBAL ITV services detected and correctly installed?
• Access of the different services including HbbTV. Are the different TV services (audiovisual and
data application) working reliably? When zapping is the application life cycle respected? (i.e. the
application stops when it is not declared in the new AIT) Test will be carried out on the different
HbbTV services.
A panel of commercial HbbTV and non-HbbTV and set-top box will be used. Test will also carried out with
TV manufacturers portals (problems were encountered during the original launch of HbbTV in France).
3.2.3 Functional requirements and phase of the test

Table 2: Functional and interoperability requirements and phase of the test (TDF)
Test Approach Phase Requirements
Integration test First First #2, #4, #6, #8, #11
Interoperability with terminals First First #2, #4, #6, #8
Integration test First Second #2, #4, #6, #8,#11
3.3 IRT platform

IRT will assess the non-interference of the GLOBAL ITV platform signalling with existing DVB terminals,
utilising the available terminals, both TV sets with integrated receivers and set-top-boxes, at the IRT
premises.
The focus will therefore be on the non-interference on the terminal side from a user perspective and not
on the influence of the coexistence signalling on head-end components as, for example, in the TDF plat-
form described in the previous section..
page 8
Version of
3.3.1 Test signals

It is currently planned that the test signals will be a multiplex of:
• TV services with HbbTV and Ginga applications utilizing the GLOBAL ITV coexistence approach
for signaling. In particular, these services will contain the applications developed within the pro-
ject.
• Recorded commercial TV services i.e on-air programs services broadcast in Germany via DVB
including HbbTV applications. These services will be recorded and then remultiplexed to include
the test signals.
The play-out will be provided via the GLOBAL ITV play-out prototype. Depending on the progress of the
play-out prototype by the time of the testing campaign, the multiplexing of the GLOBAL ITV compliant
signal with coexisting HbbTV and Ginga applications with existing on-air streams may be done externally,
using existing tools, or if this is not feasible, only the GLOBAL ITV test services will be assessed.
3.3.2 Integration platform

The integration platform consists of the GLOBAL ITV platform, directly outputting a DVB test signal to the
terminal. Hence no potential signal degradation in the transmission path will be considered. The overall
structure of the platform to be used is depicted in Figure 3.
Figure 3: IRT platform
Note that depending on the development progress, the final multiplexing und subsequent modulation will
be done outside of the play-out prototype. Similarly, depending on the final capabilities of the play-out
prototype, DVB-C or DVB-T may be utilized as input for the terminals. The results will, however, still be
indicative for DVB systems in general as all DVB systems use the same mechanisms for HbbTV applica-
tion signalling.
page 9
Version of
3.3.3 Interoperability with terminals

The interoperability assessment will utilise thevariety of terminals available at IRT in combination with
using the play-out to generate a GLOBAL ITV compliant signalling via DVB, allowing an assessment of
the potential interference problems to be encountered with DVB capable devices.
Additionally, the DVB-C version of the set-top-box prototype will also be included in the assessment. Even
though the assessment will be restricted to DVB-C due to the limitations of the set-top-box prototype, the
results will also be indicative for DVB systems in general as all DVB systems use the same mechanisms
for HbbTV application signalling.
3.3.4 Interoperability workshop

IRT regularly organises on the behalf of the HbbTV Association an interoperability workshop for HbbTV,
where HbbTV application developers and HbbTV terminal manufacturers participate to assess the in-
teroperability between a wide range of applications and terminals. In particular, new devices, not yet
available in the markets are usually included in the workshop.
It is planed that the playout of the GLOBAL ITV compliant signalling via DVB-C will be tested in the context
of this workshop, allowing on the one hand for an assessment of the potential interference problems to
be encountered with current and future DVB-C devices, on the other hand to also receive feedback from
manufacturers regarding the coexistence approach for the GLOBAL ITV system.
The assessment of the GLOBAL ITV compliant signalling within the interoperability workshop supple-
ments the interoperability testing using the terminals available at the IRT premises by also including the
newest devices in the assessment.
An interoperability workshop at IRT is scheduled for November 2015 and considering that the progress
with the development of the play-out is in line with the project plan, we expect that the GLOBAL ITV play-
out should be ready for this workshop.
3.3.5 Functional requirements and phase of the test

Table 3: Functional and interoperability requirements and phase of the test (IRT)
Integration test First First #2, #4, #11
Interoperability with terminals First First #2, #4, #6, #8
Test signals First Second #7, #8, #9, #10, #12
Interoperability workshop First Second #4, #6, #8, #10, #11, #12
3.4 Unicamp platform
3.4.1 Platform description

The TS is generated with a software library called OpenCaster on a computer with Debian Linux.
OpenCaster was initially designed to run on Debian as an improvement of justDVB-It, but it can now also
be found in the Ubuntu 14.04 repository.
For transmitting the generated TS, Unicamp is using an ISDB-T licensed DekTec 2111 modulating card,
which is attached to the computer's PCIe bus. The DekTec family has two main pieces of software to
configure and control the transmission: StreamXpress, a GUI that running on Windows, and DtPlay, a
command line based program that runs on Linux and Windows.
Since both OpenCaster and DtPlay work in the same Linux environment, effort is being made to unify
both the generating and the transmitting platforms, but in the meantime, a mix of Debian, Ubuntu 14.04
and sometimes Windows is being used.
There are some limitations regarding DtPlay. It requires the installation of the DekTec LinuxSDK, which
installs drivers and loads the API in the computer. The modulating board also has to have an ISDB-T
license, which is -at the moment- an optional feature. There is also the need to initialize the ISDB-T
page 10
Version of
parameters inside the source code of DtPlay and recompile it if the TS generated with OpenCaster should
be played back.
Figure 4: Block diagram of Unicamp platform
3.4.2 Tables Generation

PSI tables are used to configure the multiplexing of the Packetized Elementary Stream (PES) into a TS.
Some of those tables are described by the MPEG-2 standard and some are described by the modulating
standard (ISDB-Tb or DVB). Unicamp is currently muxing the TS using five of those tables: PAT, PMT,
SDT, NIT and AIT, being all of them defined in the SBTVD.
The PAT lists all the programs inside the same TS, the PMT lists the content of each program, the SDT
will describe the services inside those programs, the NIT will provide technical information about the TS
itself, such as network ID, center frequency, guard interval etc., and finally the AIT is the table that will
describe the applications which are being sent with the TS. Both Ginga and HbbTV applications will be
described in the same AIT with the same PID, though the will differ regarding their service description.
3.4.3 Graphical User Interface

For the purpose of simplifying the integration of the efforts in the generation and transmission of the
transport streams, Unicamp has decided to create a GUI (Graphical User Interface), such that the current
procedures and tests will be performed in a more “user-friendly” way, while the minimal tables generation,
configuration files handling and shell scripts will be executed in background.
The current transmission of transport streams is being carried with Stream Xpress software, on Windows,
but efforts are being made for making the transmission on Linux possible, with DTPlay. As soon as this
page 11
Version of
migration to Linux may be perfectly performed, we will be able to integrate DTPlay into the playout GUI.
So, both the transport stream generation and transmission stages will be done with only one program.
The two following figures summarize the both possibilities for the GUI.
Figure 5: First option – integrating TS generation and transmission stages
Figure 6: Second option – TS generation and transmission stages separated
Obviously, the first option is that one that best suits to our needs, once both stages are performed in only
one platform and operating system. But, the implementation of such feature depends on the success of
the migration of the transmission stage from StreamXpress on Windows to DTPlay on Linux.
3.4.4 Test signal

The open source software multiplexor Open Caster is able to generate DSM-CC object carousels for both
Ginga and HbbTV applications. It is used to multiplex audio, video and data services (Ginga and HbbTV)
into a single Transport Stream. For the set-up of the play-out and measurement with the first version of
the GLOBALITV STB, the multiplex include only one application: Ginga or HbbTV. For the following steps,
it will be possible to generate streams with both type of application and with the relevant signalling.
Tests will be carried out with the TV programs currently broadcast in Brazil to verify the compatibilty of
the set-top box with ISDB-T programs.
page 12
Version of
3.4.5 Measurement
A transport stream analyzer is used to provide the information about the content of the transport stream
such as type of modulation, video, audio.
At the beginning of the project, an open source TS analyzer was used: Snoop Dvb. It displays the infor-
mation of the Program Specific Information (PSI) tables too.
But in order to have more precise measurement, Unicamp is about to buy a commercial analyzer.
Dektec DTA 2111 board is used as modulator. Broadcasting can be done by cable or aerial antenna. Two
different tools can be used depending on the operating system:
• For Windows environment: StreamXpress playout software was used. As explained above, it
offers a graphical GUI for the Dektec platform configuration. The disadvantage software, in our
case, is that it cannot work in the same environment as the multiplexor that requires Linux.
• For Linux environment: DtPlay playout software was used. This software is run through com-
mand lines for the configuration of the broadcasting equipment. But it present the advantage to
run on the same machine as the Opencaster multiplexer.
The main parameters are the modulation configuration and the broadcasting frequency.
3.4.6 Functional requirement
The play-out will be used to test GLOBALITV set-up box in the ISDB-T context. In that purpose,
elementary Ginga and HbbTV applications will be used in a first step. This will be necessary to confirm
the behaviour of GLOBAL ITV set-top box with the specific signalling defined in the project. The different
services will have to be processed and displayed correctly. In a second step test will be carried out with
the different applications developed in the framework of the project.
These tests are essential pre-requistes to the subjective tests
Some tests have been carried out for the play-out development and the interoperability with the STB. It
was observed that the inclusion of HbbTV application does not interfere with the proper transmission of
the transport stream.
Tests have also been carried out with multiplex including one data application (Ginga or HbbTV),
a) Ginga application was decoded and displaid directly by a television commercial. The Ginga application
was clearly visualized without any difficulty.
b) HbbTV application was decoded by GLOBAL ITV STB which supports SBTVD and HbbTV functionality.
The HbbTV application was clearly visualized without any difficulty
Table 4: Functional and interoperability requirements and phase of the test (UNICAMP)

Integration tests First First #1,#2,
#3,#4,#5,#6,#7,#10,#11
Transmission First Second #1, #2,#3,#4,#5,#6,#11
page 13
Version of
3.5 UFPA platform

For the development of the UFPA tests, a platform that integrates the available infrastructure in the insti-
tution laboratories and use of RBA TV (broadcaster located in the northern of Brazil) infrastructure was
designed. The proposal of this integration is to test all the prototypes and perhaps to develop tests with
end-users using the background and infrastructure of the service provider.
The tests proposed using the UFPA platform are divided according to each of platform’s subsystem:
1) Broadcast level (play-out): related to the transmission in order to test the designed play-out to
make feasible the GLOBAL ITV platform. The proposal is to consider:
•Tests involving the European (DVB) and Brazilian (ISDB-Tb) standards and applications de-
veloped for Ginga and for HbbTV in WP4.
• The signal will be analyses, using a spectrum analyser.
• The time needed to set-up the test-bed will be measured. This tracking will provide information
that may help in the development of the business plan, since it will enable too estimate the
resources required for assembling the test-bed.
2) Reception level (STB prototype): related to the reception, during which both objective test goals
with the set-top-box prototype created in the project, as well as subjective tests will be performed.
The equipment, however, will be used mainly for a test focusing on the user experience mapping
during contact with the GLOBAL ITV platform, as described below:
• Flow analysis generated by the application. The application data connection will be recorded
in order to acquire some qualitative and quantitative information about the Internet connection
required for the application, as the data rate required for the application, the data rate deliv-
ered by the network to the application, the delay, and jitter and packet loss. In this way, it will
be possible to characterize the application and analyse its impacts in the telecommunication
infrastructure;
• If possible, an analysis of the hardware requirements for the correct functioning of the plat-
form, measuring CPU usage, RAM, refresh screen frames rate, etc.
3.5.1 Environment 1: Outdoor Broadcast (RBA TV) and reception at laboratory in UFPA
The purpose of this test environment is the broadcast and reception of the digital signal in ISDB-Tb stand-
ards. For this, the transmission equipment and infrastructure of a commercial television channel, the RBA
TV, will be used combined with a set-top box prototype, connected to a TV set hosted in a research lab
at the campus of Federal University of Pará (UFPA), which is 6 km away from the TV channel head office.
In this setup, the data carousel will transport the implemented applications for the Ginga and the HbbTV
developed for both reception and interaction tests in the GLOBAL ITV STB.
From the STB connection in a network and using a protocol analyser, it will be possible to collect all the
data stream generated by the application in the network. From this perspective, the required connection
parameters can be analysed, as well as the limitations imposed by some network technologies.
Furthermore, using a spectrum analyser, various quality parameters for the signal generated by the broad-
caster and the quality of the signal received at the STB, placed at the UFPA laboratory, can be deter-
mined.
3.5.1.1 Procedures for testing

a) Technical visit from UFPA team at the RBA TV, to evaluate which equipment will be available for the
tests and what will be needed.
b) Setting up of equipment:
• Setting-up the playout with the Brazilian standard transmission.
• Insertion of Digital TV applications on data carousel, in both standards, to test the STB of the
GLOBAL ITV.
page 14
Version of
Organizing and testing the prototype STB connection on the TV set placed at UFPA research
•
laboratory.
c) Data collection instruments:
• Developing pre-validating form for data collection to filled during observation of the tests (first
hour), with a researcher at RBA TV and another at UFPA research laboratory.
• Developing and pre-testing script interaction sequence actions with applications that will run dur-
ing the transmission and reception tests.
• Connecting and configuring the computers with the proper monitoring tests to STB software,
based on pre-established metrics (protocol and spectrum analyser).
d) Running integrated tests of the transmission chain (audio, video and application) from broadcasting
and received by the STB:
• ISDB-Tb transmission from the station + Application in Ginga + Application in HbbTV: Test the
ISDB-Tb signal reception and the application in the STB installed at the UFPA laboratory facilities
for a period of 24 hours. At certain times of the transmission and reception will be carried out user
interaction tests with applications from pre-established scripts, to generate a data stream on the
Internet and this, in turn, will be captured and analyzed.
The proposal is that in the end it is possible to compare the results of tests done with the application made
in Ginga and the HTML for HbbTV.
3.5.1.2 Equipment needed

• Computer with two network cards (packet Analyser);
• TV set;
• Playout with two patterns;
• Notebook;
• Signal and spectrum analysers;
• GLOBAL ITV set-top box prototype.
3.5.2 Environment 2: Indoor Broadcast and reception at RBA TV

The purpose of this test environment is the broadcast and reception of the digital signal in the DVB stand-
ard. For this, equipment and the infrastructure of transmission of RBA TV will be used, and the set-top-
box prototype GLOBAL ITV project will be connected to a TV set inside the station. The goal is to perform
tests in a controlled environment, in an attempt to measure the different aspects that impact on the trans-
mission and the DVB standard signal reception within a Brazilian broadcaster.
The indoor environment similar to the outdoor environment at the level of necessary equipment. For this
stage the station's play-out will be configured on the DVB standard, therefore, applications developed for
Ginga and HbbTV are inserted.

a) Technical visit from UFPA team at the RBA TV, to evaluate which equipment will be available for the
tests, and what will be needed.
b) Setting up of equipment:
• Setting the playout with the European standard transmission.
• Insertion of Digital TV applications on data carousel, in both standards, to test the STB of the
GLOBAL ITV.
• Organize and test of the prototype STB connection on the TV set in operation in the TV channel
itself.
page 15
Version of
c) Data collection instruments:

• Developing and pre-testing an unstructured questionnaire for data collection to be filled-in during
the observation of the tests (first hour), with a monitor at RBA TV and another one at the UFPA
research laboratory.
• Developing and pre-testing interaction action script with applications that will run during the trans-
mission and reception tests.
• Connecting and configuring the computers with monitoring tools to the STB software, based on
pre-established metrics (protocol and spectrum analyser).
d) Running integrated tests of transmission (audio, video and application) from the play-out to the re-
ception and by the STB:
• DVB transmission from the station + Application in Ginga + Application in HbbTV (HTML): This
transmission will be at low power, for indoor tests. The aim is to test the DVB signal reception and
the application in the STB on the premises of the TV channel for a period of 24 hours. During that
time, interaction will also occur for the approval testing of the applications from the pre-established
research scripts, in order to generate a data stream over the Internet and this, in turn, will be
captured and analyzed.
The proposal is that at the end of the two levels of transmission and data reception could be possible to
compare the test results done with the application made in Ginga and in HbbTV.

• TV set;
• Signal and spectrum analysers;
• GLOBAL ITV set-top-box prototype.
Table 4: Synthetic table of data collected during the tests (UFPA)

Metrics and information
Tests objectives
will be collected during the tests
Broadcast level (Playout)
Broadcast • Table generation

• Video quality
• Jitter
Evaluating the business plan • Time / investment for deployment of the
tests
Validating technology (architecture) • Analyzing the complete structure of the
assembled system
Reception level (STB)
Analysis of the flow generated over the net- • Throughput required by the app
work (Network analysis) • Throughput experienced by the app
• Delay
• Jitter
• Packet losses
Hardware requirements analysis for the correct • CPU usage
operation of the platform • RAM usage
• Screen frames refresh rate
page 16
Version of
3.5.3 Functional requirements and phases of the test

Table 5: Functional and interoperability requirements and phase of the test (UFPA)

Outdoor Broadcast (RBA) and recep- First First #1, #3, #5, #7, #11
tion (laboratory in UFPA)
Transmission and reception at RBA TV First First #1, #2, #3, #4, #5, #6, #8, #11
page 17
Version of
4 Subjective measurement test sites specification
Several factors must be considered in evaluating and testing the GLOBAL ITV platform, since it is not just
about verifying the platform’s technical operation, but also the levels of acceptance/rejection and approval
that will be established by the end-users sample and other publics of interest.
As presented in D5.1 – Testing and Evaluation plan, the subjective tests include all tests and procedures
to validate the use and approval by the end-users of the platform, and also tests involving people who will
have access to the platform in the future and will therefore have their workflow impacted. Thus the proce-
dures that will be applied in Brazil and in Europe, by UFPA (Brazil) and A-CING (Spain), are listed below:
• Validation and acceptance test.
• Exploitation.
Considering this, four different test environments are defined:

• Environment 1: Transmission, reception and end-user interaction at the UFPA Laboratory.
• Environment 2: Transmission, reception and interaction on end-users at home.
• Environment 3: Demonstration and focus group at the UFPA Laboratory.
• Environment 4: Transmission, reception and end-user interaction at A-CING headquarters.
The public involved in tests will be individual participants selected according to the profiles described in
D2.2 - Use cases and scenarios, and entrepreneurs and professionals from the television broadcasters
and content producers. The sample selection, both in Brazil and in Europe, will be made using:
• Invitation focused in social media.
• Invitation focused during GLOBAL ITV Workshops.
• Invitation focused in universities located in Belém and in Madrid.
• Invitation focused using the mailing of the Council members GLOBAL ITV.
• And possibly using recruitment agencies in Brazil.
It is planned to integrate the efforts of UFPA and A-CING during the tests.
4.1 UFPA platform

The same infrastructure planned for the objective testing in UFPA will be used in the subjective testing,
with the addition of two controlled environments.
Subjective tests are based on a multi-methodological protocol for collecting and analyzing data, in order
to obtain results, which consider the factors that influence the user experience quality. The proposed
protocol, presented in D5.1 - Testing and Evaluation plan, is to detail and guide the test steps in the two
test environments that are defined in the UFPA test platform, involving the participation of end users.
4.1.1 Environment 1: Transmission, reception and end-user interaction at the UFPA Laboratory
Controlled environment to test the transmission, reception and interaction of participating users with the
GLOBAL ITV platform that simulates a living room of the home user with a common television set con-
nected to the set-top box from GLOBAL ITV. This environment will have permanent monitoring by strate-
gically distributed cameras and microphones, and enables the interaction between researchers and focus
group moderators who are working in the room using an in-ear monitor.
The user will receive a guide containing a script with predefined tasks to interact with the platform in a
targeted manner and some directions to exploit the resources freely.
page 18
Version of
To perform this test, the same structure prepared for the objective tests (indoor) will be assembled with a
play-out configured for ISDB-Tb transmission and reception through the STB connected to a television,
however, now at the UFPA laboratory facilities. The STB, in turn, will be connected to a computer for end-
user interaction with software monitoring system (QoS) which will record the screen and the end-user
actions using pre-established metrics, such as the duration of the testing, evaluation and error rate, heat
map, etc.
In addition to monitoring all user actions via audio and video during the test, there will be a researcher in
another room to observe the end-user behaviour through-out the broadcast transmission. The observation
will be systematic, indirect and non-participating, based on a pre-established script.
As mentioned in D5.1 [2], we believe that with this type of test environment it will be possible to create
standards for contextual and system factors, in order to establish benchmarks between the tests per-
formed in the Brazilian and European scenarios. The idea is to ensure that the environments, procedures
and tests format will happen in the most orderly and standardized manner possible both in Brazil and as
in Europe.
By restricting the tests to a controlled environment, the only factor that will make a difference between the
tests conducted is the human factor, related to the sociocultural profile end-user.
a) Sample selection:
• 20 to 30 participants selected by invitation on social media, during GLOBAL ITV workshops in
UFPA and also by invitation in universities located in Belém.
b) Setting up equipment and organization of test site:

• Installation and configuration of camera equipment for transmission and recording during the test.
• Installation of play-out, TV and STB prototype for end-user interaction.
• Configuration and testing the integration of in-ear monitoring of integration between the coordina-
tor of the test that will be in the room and the team of researchers that will monitor the images in
an auxiliary room.
• Definition of the cameras position in order to record of user-end behaviour during testing.
• Preparation of the test scenario in an attempt to simulate a home user’s living room.
• Connecting and configuring the computer with monitoring tests software to the STB from pre-
established metrics.
• Photographing the room with assembled and tested infrastructure.
c) Instruments and data collection methodologies:

• Developing of scripts to pre-test.
• Developing and pre-testing scripts from indirect systematic and non-participant observation.
• Developing and review of terms to be signed by the tests participants, in order that the data can
be collected and used for the project.
• Developing of scripts and acknowledgements to explain the tests to the end-users.
• Definition and preparation of teams that will implement the protocol in Brazil and in Europe.
d) Running test:
• Reception of the participants.
• Collecting of the test participants’ details (signing a participant’s agreement, and filling of socio-
economic survey, delivery of the roadmap of interaction with technology).
• User interaction with the platform and the systematic monitoring (audio and video) and observa-
tion, indirect and non-participating of the research team responsible for testing.
page 19
Version of
• Filling out a questionnaire on the user-platform uses experience.

• Application of in-depth interview based on the questionnaire answers.
• Systematization of data for later analysis.

• Computer with two network cards (packet analyser);
• TV set;
• Play-out with two patterns;
• Notebook;
• GLOBAL ITV set-top box prototype;
• Shotgun microphone (minimum 4);
• Professional or semi-professional video cameras (minimum 2);
• Headset (minimum 2);
• In-ear monitoring.
4.1.2 Environment 2: Transmission, reception and interaction on end-users at home

This environment implements the same testing framework in the users’ home for a short period of time
(one week) to evaluate the quality of the end-user experience in his or her own home. We propose that
part of the testing performed in the laboratory should be also be done in this environment, in order to
observe the platform operation in real-life environments, permeated by different levels of technology, ac-
cess availability etc. The goal of this type of testing is to observe the platform in the end-users’ natural
environment. Although this test can only be carried out in Brazil, it will qualify the results obtained in
laboratory tests in order to understand consumer habits and potential consumer practices related to the
GLOBAL ITV platform.
• Two user’s home selected maybe using recruitment agencies.

• Installation of the TV and STB prototype for end-user interaction.
• Configuration and testing the integration of the in-ear monitoring of integration between the coor-
dinator of the test that will be in the room and the team of researchers that will monitor the images
in an auxiliary room.
• Definition of the cameras position in order to record of end-user behaviour during testing.
• Connecting and configuring the computer with monitoring tests software to the STB from pre-
established metrics.

• Developing assistance and a diary to be filled out by the end-users during the days that the test
is being conducted in the users’ house.
• Developing and pre-testing scripts for indirect systematic and non-participant observation at the
user home.
• Developing and review of compliance terms to be signed by the tests participants, in order that
the data can be collected and used for the project.
• Developing of scripts and acknowledgements to explain the tests for the end-users.
page 20
Version of
d) Running test:
• Installation of the testing infrastructure at the end-users’ home.
• User awareness meeting for the tests and guidelines for filling-in the daily interaction dairy and
assistance.
• Completing a socioeconomic questionnaire.
• Submission of details of the test participants (signature of lease agreement term, completion of
socioeconomic survey, delivery of interaction script with technology).
• User interaction with the platform and systematic observation, indirect and non-participant of the
researchers’ team responsible for testing.
• Systematization of data for later analysis.

• TV set;
• GLOBAL ITV set-top box prototype;
• Microphones shot gun (2 units);
• Professional or semi-professional video camera;
• Headset (1 unit).
4.1.3 Environment 3: Demonstration and focus group at the UFPA Laboratory

The proposal is to establish a controlled environment with permanent monitoring of audio and video of
the focus group, through strategically placed cameras and microphones. A room will be organized in the
working meeting format, for easy viewing and interaction between all participants of the group. Two re-
searchers will be present to conduct the discussion with the sample of viewers selected from different
socioeconomic profiles. Both the image and the audio will be simultaneously transmitted to a distinct
room, where another pair of researchers will be watching the dynamics of interaction among participants
and moderators, and they can get in touch with the focal group moderators, through in-ear monitoring, if
necessary.
In this space, it will still be possible to conduct different types of end-user monitoring/observation, in par-
allel with the test. For this, it will be use the same laboratory of environment 1.
The goal of the focus group is to understand the expectations of the viewers / participants about the
possible uses of GLOBAL ITV platform. For this purpose, it will be made a demonstration of use possibil-
ities of GLOBAL ITV platform through a TV set connected to the STB prototype. The idea is to verify what
audiences expect from an interoperability platform and then it will be possible to compare what they
expect and what the platform can offer (at that stage of development) and what the viewers’ use during
the tests.

• 10 to 12 participants selected by focused invitation to GLOBAL ITV Council members.

• Installation of TV and STB prototype for demonstration during the focus group.
• Configuration and testing the integration of in-ear monitoring of integration between the team of
researchers.
• Definition of the cameras position in the focus group room.
page 21
Version of
• Strategic organization chairs for participants and researchers/moderators focus groups.


• Developing of a script to pre-test focus groups.
• Preparation and review of compliance terms to be signed by the tests participants, in order that
the data can be collected and used for the project.
• Developing of scripts and acknowledgements to explain the tests for the end-users.
• Definition and preparation of teams that will implement the protocol in Brazil and in Europe, in a
online workshop.
d) Running test:
• Reception of the focus group participants.
• Submission of the details for the test participants (signing a participant’s agreement, and filling of
socioeconomic survey).
• Demonstration of the GLOBAL ITV platform, including an example of the interaction set-top box/
applications interaction.
• Carrying out discussions within focus group.
• Systematisation of data for later analysis.

• Computer with two network boards (packet analyser);
• TV set;
• Notebook;
• GLOBAL ITV set-top-box prototype;
• Shotgun microphone (minimum 4);
• Professional or semi-professional video cameras (2 units);
• Headset (2 units);
• In-ear monitoring.
4.2 A-CING platform

As the objective of WP5 is to demonstrate the services and to validate the development carried out in the
framework of the project, a different environment from the ones used in the objective tests is usedby A-
CING.
A-CING is not involved in objective tests, but integrated with UFPA and TDF, will provide an infrastructure
for subjective tests. Note, that the proposal is not to replicate environments already tested in Brazil and
in Europe.
The tests will be carried out with the help of Cellnex, which will kindly provide the facilities for the testing:
equipment, laboratories, etc.
4.2.1 Environment 4: Transmission, reception and end-user interaction

What is intended to be tested in this environment is the difference between commercial developments as
provided today by broadcasters, and GLOBAL ITV developments. Both hardware and software will be
page 22
Version of
tested, in order to provide useful feedback about the overall GLOBAL ITV results. Comparing with existing
hybrid applications currently broadcast on air is a suitable mechanism to get feedback information from
end users about the performance of the GLOBAL ITV system. Both for the development within the project,
but also for the exploitation of the results reasons, it is useful to understand the end users’ expectations,
behavior and assessment of applications. Clearly, there will be a bias due to the difference between well
tested hardware and software already used commercially, and the novel applications running on a proto-
type hardware platform. The proposed testing procedure has therefore being designed to minimize this
bias.
Without this comparison, with tests only focusing on the acceptance of the GLOBAL ITV system it would
be very difficult to find out whether the results are caused by to the basic nature of the used GLOBAL ITV
system or by interactive TV experience in general.
Depending on the applications developed by GLOBAL ITV in WP4, some existing commercial applications
provided by Spanish broadcasters will be selected for the testing. It is intended that the selected HbbTV
applications will be as similar as possible to the apps developed by GLOBAL ITV on WP4.
4.2.2 Procedures for testing

• 10 to 12 participants selected as indicated at the beginning of this chapter.

• The GLOBAL ITV platform test bed set up will be carried out in one specific lab room in Cellnex
facilities. This lab room has all the required network interfaces to test the GLOBAL ITV platform
with the interaction of sampled end-users, including the required equipment for users’ actions and
screen recording. A-CING will be fully able to perform the tests, as the required equipment will be
supplied by other partners from the consortium and Cellnex will provide the lab for the testing.
The typical way to consume interactive TV is provided in the test set-up: a commercial TV with HbbTV
features (or connected to an HbbTV set top box, still to be defined; transparent for testing purposes any-
way) will be installed, connected to the antenna cable to receive the Spanish DVB-T broadcasting signal
and an ADSL line for interacting with broadband:
page 23
Version of
Figure 7: Scenario 1: typical scenario
The high-level description of the A-CING testing play-out system for GLOBAL ITV developments can be
seen in Figure 8.
Figure 8: Scenario 2: GLOBAL ITV Scenario
page 24
Version of
The order for testing Scenario 1/Scenario 2 will be changed to evaluate its influence in the comparison.
Still, the appearance of the services will be different and the user will always know when a commercial
service is being tested and when the GLOBAL ITV development.
c) Instruments and data collection methodologies

• The planned technique for A-CING methodology is Goal, question, metric (GQM). This technique
was promoted in 1984 by Victor Basili [3]. It will allow to plan the tests, to carry out them and to
analyze the results efficiently.
• GQM is divided in three levels:
1. Goals (conceptual level): which are the objectives for testing? What is the info we are willing
to extract from end users?
2. Questions (operational level): which are the questions we are going to make to end users to
reach our goals?
3. Metric (quantitative level): It defines how we are going to analyze those data/answers we
received.
Basili described his six-step GQM process as follows:

Step 1. Develop a set of corporate, division and project business goals and associated measure-
ment goals for productivity and quality.
Step 2. Generate questions (based on models) that define those goals as completely as possible
in a quantifiable way.
Step 3. Specify the measures needed to be collected to answer those questions and track pro-
cess and product conformance to the goals.
Step 4. Develop mechanisms for data collection.
Step 5. Collect, validate and analyze the data in real time to provide feedback to projects for
corrective action.
Step 6. Analyze the data in a post mortem fashion to assess conformance to the goals and to
make recommendations for future improvements.
Applying this methodology within the context of GLOBAL ITV subjective testing we can write the steps as
following:
Step 1. Objectives for testing in Environment 4

• Four objectives have been defined:
1. To find out if GLOBAL ITV hardware and software implementation is free of bugs, adequate
and efficient.
2. To find out if our product features are comparable to similar services existing on the market,
if they are better, or worse, or very similar.
3. To find out if the benefits emerging from interoperable TV technologies are perceived by end
users.
4. To find out which kind of interactive TV services are expected and desired by end users.
Step 2. Questions for each objective

• For Objective 1 (to find out if GLOBAL ITV hardware and software implementation is free of bugs,
adequate and efficient):
1. Have you detected any bug in the GLOBAL ITV service programming, design, workout, etc?
If the answer is YES, please describe it briefly.
2. Regarding the design and the look and feel of GLOBAL ITV service: What do you think about
colors, fonts, size of the letters and boxes, etc?
3. Is the GLOBAL ITV service quick enough for you?
page 25
Version of
4. In general, GLOBAL ITV service is …

5. Did the GLOBAL ITV service satisfy your expectations?
• For Objective 2 (To find out if our product features are comparable to similar services existing on
the market, if they are better, or worse, or very similar)
1. In general terms, which service (GLOBAL ITV or the other one) did you like the most?
2. Do you think both services’ features are similar? Why?
3. In your own opinion, which service has:
• Better design?
• Better latency?
• Better usability?
4. Which service has a better usability?
5. Would you say that Existing service is ease of use? Would you say that GLOBAL ITV is ease
of use?
6. Did you enjoy GLOBAL ITV service? Did you enjoy Existing service?
• For Objective 3. (To find out if the benefits emerging from interoperable TV technologies are
perceived by end users)
1. Do you think it is good, bad or indifferent for you as a TV consumer, that TV technologies
standards would be global, instead of regional as they tend to be today? And for TV industry?
2. Do you think it is good, bad or indifferent for you as a TV consumer being able to experience
interactive TV services not developed in Europe?
• For Objective 4. (To find out which kind of interactive TV services are expected and desired by
end users)
1. Regarding the offered service (not just the app, but the idea of the app), did you find it inter-
esting?
2. Which feature did you enjoy the most in GLOBAL ITV service? And which feature did you
enjoy the least?
3. Same questions that Question 2 about existing service.
4. Sort the following services from 1 – highest interest – to 8 – lowest interest –:
• e-Health;
• e-Administration;
• Other content related with TV programs;
• Links with Social Media;
• Games;
• News;
• More info about TV programs;
• e-Shop.
Step 3. Specify the measures needed to be collected to answer those questions and track process and
product conformance to the goals.
• For Objective 1 (to find out if GLOBAL ITV hardware and software implementation is free of bugs,
adequate and efficient):
1. Have you detected any bug in the GLOBAL ITV service programming, design, workout, etc?
If the answer is YES, please describe it briefly.
No predefined answers: just empty space for users to fill-in and then collecting answers and
grouping them by item.
2. Regarding the design and the look and feel of the GLOBAL ITV service: What do you think
about colors, fonts, size of the letters and boxes, etc.?
page 26
Version of
1. Not appropriated at all

2. Not very appropriated
3. Appropriated.
4. Very appropriated
5. Extremely appropriated.
3. Is the GLOBAL ITV Service quick enough for you?

Yes/no
4. In general, GLOBAL ITV service is …
1. Very poor
2. Poor
3. Fair
4. Good
5. Very good
5. Did GLOBAL ITV service satisfy your expectations? Justify your answer.
No predefined answers: just blanck space for users to fill and then collecting answers and
grouping them by item.
• For Objective 2 (To find out if our product features are comparable to similar services existing on
the market, if they are better, or worse, or very similar)
For questions regarding Objective 2, there are no predefined answers: just empty space for users
to fill-in and then collecting answers and grouping them by item.
• For Objective 3. (To find out if the benefits emerging from interoperable TV technologies are
perceived by end users)
For questions regarding Objective 3, there are no predefined answers: just empty space for users
to fill-in and then collecting answers and grouping them by item.
• For Objective 4. (To find out which kind of interactive TV services are expected and desired by
end users)
No predefined answers for first questions: just empty space for users to fill-in and then collecting
answers and grouping them by item.
1. Sort out the following services from 1 – highest interest – to 8 – lowest interest –:
The answer is the sorted-out list.
Step 4. Develop mechanisms for data collection.

• A questionnaire will be filled by the user with useful data about his/her profile, according to the
profiles described in D2.2 Use cases and scenarios.
• The subject will be invited to watch/interact with one service provided in Scenario 1 in a limited
time (around ten minutes). Then, the same time and the same invitation will be made for Scenario
2. After that, a questionnaire will be filled by him/her in order to quantify the comparison, the
differences, the QoE, etc.
• The same performance described in the point above will be develop for the second service de-
veloped in WP4.
The scene will be monitored by A-CING researchers and also recorded, in order to extract information
also from people’s behavior, questions, reactions, etc.
After the tests are done, the analysis of all the information gathered in questionnaires, videos, research-
ers’ experience, etc. will be done and the final results of the tests in this platform will be released.
page 27
Version of
Step 5. Collect, validate and analyse the data in real time to provide feedback to projects for corrective
action.
Step 6. Analyse the data in a post mortem fashion to assess conformance to the goals and to make
recommendations for future improvements.
d) Running test:
1. Installation of the tests infrastructure.
2. User awareness meeting for the tests and guidelines for filling-in the daily interaction dairy and
assistance.
3. Reception of the focus group participants.
4. Submission of the details for the test participants (signing a participant’s agreement, and filling of
socioeconomic survey).
5. User interaction with the platform.
6. Completing the questionnaires described in item c (above).
7. Systematization of data for later analysis.

• For scenario 1:
• TV set.
• Video/audio recording equipment.
• For scenario 2:
• Personal computer (PC) with minimal hardware configuration defined by DekTec modulator to
be used in the test, running Ubuntu Linux 12.04
• OpenCaster SW as required for the generation of signal multiplexing.
• RF11 antenna cable for connecting modulator in the PC with a set-top box.
• HDMI cable for connection between the STB and the TV screen.
• Ethernet cable for the connection of the GLOBAL ITV STB. This connection is required if it is
intended to test interactive applications or if applications will be downloaded by means of broad-
band network.
• DekTec modulator extension for the PC (PCIe, PCI, USB), with DVB-C output.
• GLOBAL ITV STB with incorporated DVB-C tuner.
• Carousel objects and audio/video TS or recorded signal multiplex.
• TV set.
• Audio/video recording equipment.
page 28
Version of
5 First set-up
Although the development of the demonstrator has not been finished yet, the first equipment is already
available:
• Play-out: first version following GLOBAL ITV specification is available
• GLOBAL ITV set-up box implementing HbbTV stacks
This enables some partners to carry out preliminary integration testing. This is necessary to validate
GLOBAL ITV choices regarding application signaling, the project development and the play-out interop-
erability.
This section gives as an example the test bed used by TDF, but other partners, such as UNICAMP or
IRT, involved in play-out development and/or interoperability testing have similar test-beds.
As a first step, TDF has generated and recorded multiplex of programs that can be used for different tests.
The streams are generated in 3 steps:
A – Generation of GLOBAL ITV multiplex

The first step is to generate a GLOBAL ITV program. Transport streams files are generated and can be
recorded. It consists of:
• Audio-video components: in our case, the source is a Brazilian TV program recorded in Belem.
• Data services: an equipment (Vizion’R) is able to encapsulate HbbTV and Ginga application into
DSM-CC carousels to broadcast them. At present, only HbbTV services are broadcast. The Ginga
application is signalled but not broadcast. This will be done as soon as the receiver will implement
Ginga NCL stacks
• Signalling: Vizion’R equipment multiplex this signal with data services and generates the required
signalling. This includes DVB-SI signalling in accordance with the French regulation and the sig-
nalling specified by the project to enable simultaneous broadcasting of HbbTV and Ginga appli-
cation.
PAT
à PMT service S
à Video component
sample.ts à Audio component
PAT
à PMT service S
HbbTV broadcast à Video component
application files à Audio component
à AIT Ginga
à AIT HbbTV
config.properties output.ts à Ginga DSM-CC Object Carousel
à HbbTV DSM-CC Object Carousel
Ginga application files
Figure 9 GLOBALITV multiplex generation
B- Multi-program multiplex generation

To perform tests in a more representative environment, TDF generates a multi-program multiplex. It in-
cludes:
• The GLOBAL ITV multiplex previously described. The recorded TS is played in loop using DVB
player tool.
page 29
Version of
• Commercial programs currently broadcast in France. They include HbbTV services.
These programs are multiplexed using a standard network multiplexor. It generates a Transport Stream
that can be recorded. It is necessary to make replicable tests in standardised condition i.e. to guarantee
that the content or the generated signals are always identical.
Possibly the output of the multiplexor can be modulated to feed TV receivers.
PAT
à PMT service S
à Video component
à Audio component
à AIT Ginga
à AIT HbbTV
à Ginga DSM-CC Object Carousel
multiplex.ts à HbbTV DSM-CC Object Carousel
Recorded à PMT service X
GLOBALITV à Video component
multiplex <
à Audio component
à AIT HbbTV
à HbbTV DSM-CC Object Carousel
Stream à HbbTV DSM-CC Stream Events
Recording à etc.
Live channels Multiplexor
DVB-T
Modulator
Figure 10 Multi-program Multiplex generation
C – Multiplex play-out
The recorded multiplex can be played-out to feed modulators at different standards;
• DVB-C and ISDB-T to test GLOBAL ITV receivers.
• DVB-T to test interoperability with existing TV receivers.
Recorded
Multi-program
multiplex
Dektec DVB-C & ISDB-T GLOBALITV
modulator receiver
DVB-T
Modulator
Figure 11 : Multiplex play-out
page 30
Version of
First GLOBAL ITV compliant signals have already been generated using this described first set-up and
will be tested in a first interoperability test in the next step, using the procedures and platforms described
in the previous sections.
Depending on these first results, adjustments may be made both to the specification of the test sites,
leading to the final test side specification to be described in D5.4, but also to the overall GLOBAL ITV
specification and prototypes, if unexpected issues arise during the testing of the first setup.
page 31
Version of
6 Final remar ks
In this document, considering the current stage of development of the GLOBAL ITV platform, we provided
a specification for the test setup and the description about how the first interoperability tests will be per-
formed, including the subjective to be performed with end users. We described the different platforms
available in the project and the tests that will be carried out i.e. environments, equipment needed, proce-
dures, etc., taking into account the functional and interoperability requirements established in Work Pack-
age 2 (WP2).
In addition, the three specific phases during which the proposed tests will be conducted both in Brazil and
in Europe were described: (i) first and pilot block of tests using playout and STB prototype of the project
with HbbTV and Ginga applications, that are currently in operation in Europe and Brazil; (ii) final objective
tests will be carried out using the play-out, STB and applications developed in the project; and (iii) sub-
jective tests with end users will also be carried out using the play-out, STB and applications developed in
the project. In each of these phases, reports will be generated as a basis for new WP5 deliverables.
page 32
Version of
Glossar y
ABNT Associação Brasileira de Normas Técnicas (Brazilian Association of Technicals

Standards)
ADSL Asymmetric Digital Subscriber Line
AIT Application Information Table
API Application Programming Interface
CA Consortium Agreement
CoA Coordination Agreement
CPU Central Processing Unit
DoW Description of Work, Annex-1 of Project Proposal
DSM-CC Digital Storage Media Command and Control
DTT Digital Terrestrial Television
DVB C/T/S Digital Video Broadcasting Cable/Terrestrial/Satellite
EC European Commission
ES Elementary Stream
ETSI European Telecommunication Standards Institute
ETSI EN European Telecommunication Standards Institute European Norm
ETSI TS European Telecommunication Standards Institute Technical Specification
GUI Graphical User Interface
HbbTV Hybrid Broadcast Broadband TeleVision
HDMI High Definition multimedia Interface
HTML Hyper text Mark-up Language
ID Identifier
IPR Intellectual Property Rights
ISDB-T Integrated Services Digital Broadcasting-Terrestrial
iTV Interactive TeleVision
LAN Local Area Network
MPEG Moving Picture Expert Group
MPEG2 TS Moving Picture Expert Group 2 Transport Stream
NBR Norma Brasileira
NCL Nested Context Language
NDA Non-disclosure agreement
NIT Network InformationTable
PAT Program Association Table
PCI/PCIe Peripheral Component Interconnect/ Peripheral Component Interconnect Express
PES Packetized Elementary Stream
PMT Program Map Table
PO Project Officer
page 33
Version of
PSI Program Specific Information

QA Quality Assurance
QoS Quality of Service
R&D Research and Development
RAM Random Access Memory
RF Radio Frequency
SBTVD Sistema Brasileiro de Televisão Digital (Brazilian Digital Television System)
SDT Service Description Table
STB Set Top Box
TS Transport Stream
TV TeleVision
URL Uniform Resource Locator
USB Universal Serial Bus
WP Work Package
XML eXtensible Mark-up Language
Dx.y Deliverablex.y
Partner Acronyms
Acronym Name
IRT Institut fuer Rundfunktechnik GmbH, DE
A-CING Aqua-Consult Ingenieros, S.L., ES
FHG Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V., DE
TDF TDF, FR
RETEVISION Retevisión – AbertisTelecom, ES
SYMELAR Symelar Innovación SLU, ES
EBU European Broadcasting Union, CH
BAND TV Rede Bandeirantes de Televisão, BR
HXD HXD Interactive Television, BR
LSI-TEC Associação do Laboratório de Sistemas Integráveis Tecnológico, BR
UCB Universidade Católica de Brasília, BR
UFABC Universidade Federal do ABC, BR
UFPA Universidade Federal do Pará, BR
UNESP Universidade Estadual Paulista “Júlio de Mesquita Filho” , BR
UNICAMP Universidade Estadual de Campinas, BR
USP Universidade de São Paulo, BR
W3C World Wide Web Consortium at GEIE ERCIM, FR
page 34
Version of
Acronym Name
IRT Institut fuer Rundfunktechnik GmbH, DE
A-CING Aqua-Consult Ingenieros, S.L., ES
FHG Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V., DE
TDF TDF, FR
RETEVISION Retevisión – AbertisTelecom, ES
SYMELAR Symelar Innovación SLU, ES
EBU European Broadcasting Union, CH
RBA TV Rede Brasil Amazônia de Televisão, BR
TARA TARA Systems, DE
page 35
Version of
References
[1] GLOBAL ITV, D2.3 – Functional and Interoperability requirements, 2014.

[2] GLOBAL ITV, D5.1 - Testing and evaluation plan, 2014.
[3] Basili, Victor R.; Caldiera, Gianluigi; Rombach, H. Dieter, “The Goal Question Metric Paradigm”, En-
cyclopedia of Software Engineering (Marciniak, J.J., editor), Volume 1, John Wiley & Sons, 1994,
pp. 578-583.
page 36
Version of
List of Figures
Figure 1: Block diagram of Interactive TV platform .................................................................................... 6

Figure 2: Block diagram of TDF platform ................................................................................................... 7
Figure 3: IRT platform ................................................................................................................................. 9
Figure 4: Block diagram of Unicamp platform .......................................................................................... 11
Figure 5: First option – integrating TS generation and transmission stages ............................................ 12
Figure 6: Second option – TS generation and transmission stages separated ....................................... 12
Figure 7: Scenario 1: typical scenario ...................................................................................................... 24
Figure 8: Scenario 2: GLOBAL ITV Scenario ........................................................................................... 24
Figure 9 GLOBALITV multiplex generation .............................................................................................. 29
Figure 10 Multi-program Multiplex generation .......................................................................................... 30
Figure 11 : Multiplex play-out ................................................................................................................... 30
page 37
Version of
List of Tables
Table 1: Functional and interoperability requirements ............................................................................... 2

Table 2: Functional and interoperability requirements and phase of the test (TDF) ................................. 8
Table 3: Functional and interoperability requirements and phase of the test (IRT) ................................. 10
Table 4: Synthetic table of data collected during the tests (UFPA) ......................................................... 16
Table 5: Functional and interoperability requirements and phase of the test (UFPA) ............................. 17
page 38

Detailed Test Site Specifications and First Set-Up: Deliverable D5.2

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Detailed Test Site Specifications and First Set-Up: Deliverable D5.2

Uploaded by

Copyright:

Available Formats

Detailed Test Site specifications

and First set-up

GLOBAL ITV ID: GLOBALITV-D5.2-TestSiteSpecificationAndFirstSetUp

Version and controls:

Version Date Reason for change Editor

2 Functional requirements ............................................................................................................... 2

3 Objective measurement test sites specification ........................................................................ 5

4 Subjective measurement test sites specification..................................................................... 18

5 First set-up ................................................................................................................................... 29

List of Figures ...................................................................................................................................... 37

List of Tables ....................................................................................................................................... 38

Table 1: Functional and interoperability requirements

• Functional Requirement #1 Ginga NCL application support

• Functional Requirement #2 HbbTV1.x application support.

• Functional Requirement #3 Ginga-NCL application and handling command support

• Functional Requirement #4 HbbTV 1.x application and handling command support

• Functional Requirement #5 Ginga-NCL application receiving

• Functional Requirement #6 HbbTV1.x application receiving

• Functional Requirement #7 ISDB-Tb set-top box support

• Functional Requirement #8 DVB set-top box support

• Functional Requirement #9 Ginga-NCL Interactive Channel support

• Functional Requirement #10 HbbTV 1; x Broadband support

• Functional Requirement #11 Playout

• Functional Requirement #12 application availability

• Functional Requirement #13 HTML5

3 Objective measurement test sites specification

Objective measurements have several goals:

3.1 Generic platform description

Figure 1: Block diagram of Interactive TV platform

3.2 TDF platform

• Measurement equipment (multiplex analyser).

Figure 2: Block diagram of TDF platform

3.2.1 Test signals

3.2.2.2 Interoperability with receivers

3.2.3 Functional requirements and phase of the test

3.3 IRT platform

3.3.1 Test signals

3.3.2 Integration platform

Figure 3: IRT platform

3.3.3 Interoperability with terminals

3.3.4 Interoperability workshop

3.3.5 Functional requirements and phase of the test

3.4 Unicamp platform

3.4.1 Platform description

Figure 4: Block diagram of Unicamp platform

3.4.2 Tables Generation

3.4.3 Graphical User Interface

Figure 5: First option – integrating TS generation and transmission stages

Figure 6: Second option – TS generation and transmission stages separated

3.4.4 Test signal

3.4.6 Functional requirement

These tests are essential pre-requistes to the subjective tests

Test Approach Phase Requirements

3.5 UFPA platform

3.5.1.1 Procedures for testing

3.5.1.2 Equipment needed

3.5.2 Environment 2: Indoor Broadcast and reception at RBA TV

3.5.2.1 Procedures for testing

c) Data collection instruments:

3.5.2.2 Equipment needed

Table 4: Synthetic table of data collected during the tests (UFPA)

Broadcast • Table generation

3.5.3 Functional requirements and phases of the test