Apollon - Eparticipation Re-Adjusted Pilot Techniques and Methodology

DELIVERABLE

Project Acronym: APOLLON
Grant Agreement 250516

number:
Project Title: Advanced Pilots of Living Labs Operating in

Networks
D.5.4 Re-adjusted pilot techniques and methodology
Authors:
Tanguy Coenen, Bram Lievens (IBBT)

Claudio Vandi (UP8)
Dave Carter (MDDA)
Project co-funded by the European Commission within the ICT Policy Support Programme
Dissemination Level
P Public X
C Confidential, only for members of the consortium and the Commission Services

Apollon – Deliverable 5.4
Table of Contents
1. Introduction ................................................................................................................ 3
2. Intial pilot description ............................................................................................. 5
2.1 Paris pilot ........................................................................................................................... 5
2.2 Manchester pilot .............................................................................................................. 5
2.3 Brussels pilot .................................................................................................................... 5
3. Pilot re-‐adjustments ................................................................................................. 6
3.1 Paris pilot ........................................................................................................................... 6
3.1.1 Use scenario ................................................................................................................................ 6
3.1.2 Pilot set-‐up ................................................................................................................................... 9
1.1.2 Technology adjustments .................................................................................................... 13
3.1.3 User involvement ................................................................................................................... 16
3.2 Manchester pilot ............................................................................................................ 17
3.2.1 Use scenario ............................................................................................................................. 17
3.2.2 Pilot set-‐up ................................................................................................................................ 18
3.3 Brussels pilot .................................................................................................................. 22
3.3.1 Use scenario ............................................................................................................................. 22
3.3.2 Pilot set-‐up ................................................................................................................................ 25
4. Evaluation tracking ................................................................................................ 33
4.1 Transfer Living Labs evaluate SME technology based on user feedback in
cross-‐border settings .............................................................................................................. 33
4.2 SME evaluation of the collaboration with Living Labs in the cross-‐border
experiments ............................................................................................................................... 34

The information in this document is provided as is and no guarantee or warranty
is given that the information is fit for any particular purpose. The user thereof
uses the information at its sole risk and liability.

Statement of originality:
This deliverable contains original unpublished work except where clearly
indicated otherwise. Acknowledgement of previously published material and of
the work of others has been made through appropriate citation, quotation or both.
2
1. Introduction
The main issues addressed by APOLLON (Advanced Pilots Of Living Labs
Operating in Networks) are the present lack of Living Lab harmonisation and
collaboration, and the serious difficulties of SMEs in engaging in cross-‐border
innovation.
APOLLON will demonstrate the positive impacts of cross-‐border domain-‐specific
Living Lab networks, by setting up an advanced pilot composed of 4 thematically
focused European-‐wide Living Lab experiments. SMEs are enabled to take part in
cross-‐border Living Lab experiments beyond their home markets, and are
supported by large industrial companies, academic centres and other
stakeholders.
The APOLLON pilot aims to share and to harmonise the Living Lab approaches
and platforms between exemplary European networks as well as the subsequent
evaluation results and the set up of sustainable domain-‐specific networks on a
European and global level.
APOLLON addresses 4 major domains in which ICT products and services
innovation may benefit most from cross-‐border Living Lab networking. These
are:
• Homecare and Independent Living
• Energy Efficiency
• eManufacturing
• eParticipation
The project consortium of the domain 4 is composed of:
Issy Media ( France), Université de Paris VIII (France), IBBT (Belgium),
Manchester City Council ( United-‐Kingdom), 3D2+1 (France), Navidis (France),
and People’s Voice Media (France)
The objectives of Work Package 5 are the following:

1 The name of this SME has changed to Virdual. However, for the sake of
conformity with the other documents of the Apollon project, we will keep
referring to them as 3D2+.
3
• Sharing and comparing technologies/ methodologies in order to
understand to which local, regional, national results can be extended to
other contexts and which common technology/methodology can be built
for generalization.
• Adapting technologies/methodologies to the European context.
• Integrating technologies/methodologies to overcome fragmentation of
services
• Promoting citizens’ innovation to eMedia participation in Europe and
evaluating if cross-‐border user testing can help existing projects to open
to the European audience.
In this report we describe how the technologies and the methodologies have to
be adapted in order to carry out the pilot. The different pilot descriptions within
the media and e-‐participation workpackage were the subject of D5.3. In this
report, we document how and why some methodologies have been withheld and
others rejected.
In other words, this deliverable describes in more detail the way in which the
existing technologies will be integrated and how they need to be adapted in
order to do so. In addition, this document also describes the adjustments from a
living lab research point of view. Concerning living labs, we will not only focus on
the actors involved and the context, but also on research activities related to user
evaluation.
The structure of this document is as follows. In section 2, the description of the
initial pilots, as described in Deliverable 5.3, is re-‐stated. If changes to the pilot
description have been made, this is mentioned, but the actual changes are
detailed in section 3. This section provides detailed descriptions on the different
pilots. Finally, in section 4, a number of ways to follow up on evaluations of both
SME collaboration and user experience are described.
4
2. Intial pilot description

2.1 Paris pilot
The French pilot is identical to what was described in deliverable 5.3. As
described there, the pilot involves the aggregation of Media technologies using
3D multimedia, context aware mobile applications and Community Reporting,
within the framework of the Digital Fort project in Issy Les Moulineaux.
All WP5 partner's technologies will be involved: 3D2+ provides ‘Real and More’,
Navidis provides ‘Issy 3D’ and ‘Urbadeus’, IBBT provides ‘Air Graffiti’, and People
Voice Media provides ‘Community Reports’. 2
The main change with regard to the initial set-‐up (described in deliverable 5.3) is
a more elaborate and detailed scenario for the pilot as well as the decision at
which level the different technologies will be integrated in the pilot. Between a
fully functional integration and a simulated integration we opted to adopt an
intermediate solution that we call the aggregation approach. In this approach,
partners' technologies are not fully integrated on a common platform, but still
exchange data and are mutually accessible.
2.2 Manchester pilot

As outlined previously in Deliverable 5.3, the Manchester pilot will use the
refurbishment of Manchester Central Library and a public art trail in city centre
Manchester, managed by the Manchester City Galleries department, to focus
activities related to the testing and evaluation of products from IBBT, Navidis
and Peoples Voice Media and the evaluation of Living Labs processes and
methodologies created by Work Package 1. In addition to the scenario set out in
D5.3, the Manchester pilot will also involve a project being run by Greater
Manchester County Records Office to engage citizens in local history archives.
2.3 Brussels pilot

The initial pilot in Brussels, as described in Delverable 5.3, revolved around the
campus of the future project. The aim was to experiment with e-‐participation
regarding the future of the university as an institution and the campus as a

2 For a full description of these technologies see deliverable 5.3
5
physical social space. Therefore, various components would have been deployed
at the university campus, like an open Wifi network and access to the open data
sources from the University. The purpose of the Brussels pilots as initially
envisioned in APOLLON was to conduct an open envisioning exercise based on
and using this existing infrastructure. This would have been done in
collaboration with as many stakeholders as possible. The foreseen outcome was
to create a computer-‐assisted methodology that, by using future scenarios
wrapped in a context-‐aware application, helps to define possible ways to record
scenarios for change of a certain entity involving the users of that entity.
However, due to the fact that some of the initial technologies and core services
on the University level, related to this smart-‐university concept, are not yet
implemented, we are forced to refocus the initial pilot-‐scenario and setup. This
will be detailed in section 3.3.
3. Pilot re-adjustments
3.1 Paris pilot
3.1.1 Use scenario

The Use scenario for the French pilot is built around the City of Issy Les
Moulineaux and the Digital Fort new district. The Digital Fort in Issy-‐les-‐
Moulineaux is a new Green and Hi-‐Tech district that is being built in the place of
the old Issy Fort (constructed between 1840-‐1845 and nearly destroyed in the
war of 1870).
The first objective is to encourage the citizens to get involved in the life of their
city, to make them discover and increase their awareness on the rich history of
their town. They will not get involved in delicate political issues (for example,
expressing their anger regarding the works at the construction site), but they
will be able to express themselves and have an active role in their town, via a
series of innovative media technologies. Using a 3D interactive documentary on
the history of the Fort and a 3D interactive cartography of Issy, they will discover
the city's history. Using two context-‐aware mobile application for creating and
viewing geolocalized content they will participate in an Urban Game, become
6
History Explorers and thus contribute to a shared representation of the city.
Through Community Reports, they will become Community Reporters and create
User Generated Content.
The pilot aims to test the aggregation of various new media technologies for
involving and engaging the citizens during the construction works and to help
them discover and re-‐appropriate the Fort's history and become a part of its
future.
The realization of the pilot will involve two steps:

• In a first step, the Awareness phase, citizens of Issy-‐les-‐Moulineaux will
first be invited to discover the History of the Fort through a cross-‐media
Documentary on the Fort and a 3D interactive plan of the City.
• In a second step, citizens will act as History Explorers and Community
Reporters and will be involved in an Urban Game and Community
Reports.

Figure 1: Illustration of the Paris pilot scenario
3.1.1.1 First Phase : raising awareness

To engage with citizens, it is important to raise awareness on the object of their
participation activity and to provoke the citizens' curiosity on the proposed
subject. A group of citizens will be invited to participate in a workshop in which
they will discover the history of the Fort in an innovative way using 3D
technologies.
In this phase, the following technologies will be used to create awareness and
provoke the citizens' curiosity:
• Real and More will be used to provoke the citizen’s curiosity about the
Fort and its history.
• Issy 3D will be used as the entry level for the documentary to help
citizens visualize the Fort on the City plan.
Using these technologies, a mediator will explain the history of the Fort and its
role in the city. Citizens will be invited to actively participate in the presentation
by contributing their knowledge and memories and by interacting with the 3D
environment.
7
During this workshop, citizens will be motivated to take an active role in the
project and become History Explorers and Community Reporters in the second
phase. At the end of the workshop, citizens will be asked to express their opinion
about the technologies they have been using.
3.1.1.2 Second Phase: User Involvement through an Urban Game and

Community Reports
In this second phase, citizens will take a more active role and participate as
History Explorers and Community Reporters.
During two Urban Games, the citizens will be able to discover and contribute to
the history of their city in a vivid, interactive and playful way. Using Urbadeus
and Air Graffiti, they will follow a discovery trail, an Urban Game on the history
of Issy. Citizens will have to find a number of hints and clues that will take them
through time and legend, back to the War of 1870 and in emblematic places and
streets around the city that still show traces of the old times.
The starting point of the game will be Real and More through which
users/players will be introduced to the game scenario. With the help of an
iPhone or another Smartphone, users will be able to find POI (points of interest)
in the surrounding neighborhoods of the Fort of Issy (the interior of the Fort will
be closed to public access until 2013). The POI will offer access to information on
the history of the Fort and its surroundings before and during the 1870 war.
The available resources in the POI will be:

• Engravings presenting the Fort during the war
• Engravings of the surroundings
• 3D images based on the digital reconstruction of the Fort
• Videos of the 360 ° panorama realized from the digital reconstruction of
the Fort
• On-‐site additional information in the form of small texts
These POI will be the starting point of the urban game, which will take users
from one historical point to another, leading them finally to the coveted treasure.
Throughout the trail, users will have to solve various riddles and puzzles and
they will have to report constantly on their journey via the Urbadeus platform.
8
This will be done by “capturing reality” with their mobile phones through
emoticons, photos, videos or sounds.
Using Air Graffiti, 2D barcodes (tags) will be placed on certain historical
buildings that users will have to find by solving the various riddles along the
way. The QR codes will link to online content and information about the
building/object they are placed on.
During the game, citizens will have to snap photos and take videos that will be
uploaded to Issy 3D, thus creating a shared cartography of the City History.
During and after the game, players will also have to produce reportage materials
using Community Reports methods. They will have to comment on what they do
and what they see around them as they progress in the game.
After the game, content produced by the citizens will be accessible both on Issy
3D through Urbadeus and on mobile through Air Graffiti. Starting from the
contents produced during the game, a Community Report will be realized to keep
track of the events and to build a shared representation of the City's past and
future.
3.1.2 Pilot set-up
3.1.2.1 Technologies
In this pilot we will use all the technologies of the SME’s involved in WP5. We
will now elaborate the role of each SME in the pilot.
3D2+
3D2+ provides Real and More, a 3D documentary on the history of the Fort of Issy
Les Moulineaux and 3D interactive reconstruction of the Fort. The documentary
proposes a guided visit of the place enriched with documents such as video
sequences, engravings, sounds and pictures. The 3D reconstruction of the place
allows the user, in the form of an avatar or a subjective camera, to go around and
make a tour as one pleases in a free visit. Real and More will be used to raise
citizen’s interest and make them aware of the place of the Fort in Issy's history.
Navidis

9
Naividis provides Issy 3D, a 3D cartography of Issy that allows a user to travel in
space and time. Issy 3D will be used as the entry level for the documentary to
help citizens visualize the Fort on the City plan.
Navidis also provides Urbadeus, allowing citizens to post with a smart phone
into the 3D Model of the city additional points of information for sharing city
experience and providing relevant feedback. An example would be to build a
memory of the city or to raise any issues that will be monitored by the technical
service of the city. Urbadeus will be used to geo-‐localize content from the 3D2+
product Documentary and enrich it with geo-‐localized user-‐generated content.
IBBT
IBBT’s Air Graffiti is a context-‐aware urban mobile service that allows users to
describe, experience and discover objects and locations. The application allows
for dynamic in-‐ & outdoor trails and uses GPS, QR-‐codes, RFID-‐tags and short -‐
urls.
Air Graffiti will be used to distribute context-‐aware content (geo-‐localized, 2D-‐
barcodes) regarding the Fort’s history around the city. The Fort played an
important role in the Franco-‐Prussian War of 1870-‐71 and references to that era
are still to be found across other parts of Issy and the wider Parisian area (e.g.
museums in Paris).
People's Voice Media

People's Voice Media will provide Community Reporters programmes, which are
an innovative way for people to build confidence in the use of social media tools,
learn new skills and tell a story about themselves or their communities.
Community Reporters Programmes will be used to engage with Community
Reporters and creating Community Reports and work with and train Issy
Media’s staff as well as recruit members of the public to act as community
reporters. They will create and upload content on the social media (e.g. YouTube,
Flickr). The content will be geo-‐localized using AirGrafitti and projected via
Navidis’s Urbadeus onto the Issy 3D plan.
10
3.1.2.2 Non-SME Actors
We aim at involving Citizens, Schools and local public authorities. There is not a
specific age range or profile. The Urban Game scenario will be adapted to satisfy
different user groups (kids-‐adults).
An important actor for the pilot that wasn't involved in previous pilot set up is
the City Museum, which provided a detailed list of 9 Points of Interest that bear
some traces of the history of the City related to the Fort. These will be used as a
base for both Community Reports and the Urban Game.
3.1.2.3 User research

Users are at the core of the pilot since they will act both as History Explorers and
Community Reporters. During the pilot we will have two ways of collecting users
feedback :
1. During the pilot, as citizens use the technologies, a Living Lab
representative will be with them to collect their feedback on two
dimensions: usability and acceptability. This will be done through
questionnaires using both Likert scales and open ended questions. These
questionnaires will be designed at the beginning of the pilot (by M14 of
Apollon).
2. Towards the end of the pilot (M22 of Apollon) we will organize a Co-‐
innovation workshop to record and analyze users feedback on the
technologies they have been using. UP8 will be responsible for this task
that will be carried out using a methodology that has been used widely
before APOLLON in national projects in which we brought together the
creator of the technology and some representatives of the target user
groups: the Eneide project about New Technologies for Education
(Observus workshops with teachers), the RANUTER project about Digital
Terrestrial Radio (workshops with drivers), the Tablets for seniors
project (workshop with senior citizens). These workshops are intended to
bring together technology providers (SMEs) and end users and guide
them through a participative innovation workshop in which:
a. Users can express their needs, provide new ideas for technology
development and test and highlight what they like or don't like.
11
b. SMEs can collect input on what is needed to improve acceptability,
to make the learning curve less steep, to adapt their tools to a
group of users and to extend the potentials of their tools.
The workshops will be conducted along the following lines :
1. Participants gather in the same room
2. Each participant introduces himself
3. During a first phase, the SME’s make a quick presentation of the proposed
service
4. During the second phase, users are invited to generate ideas, focusing on
two aspects :
a. On the product side, starting from existing services, possible new
usages of these services are investigated. The general question is
“What would you like to do with this service ?”
b. On the users side, starting from existing needs new services
(inside the product) to satisfy user needs are focused upon.

Different methods exist to boost the idea production process, like free
brainstorming, guidance by images, by scenarios, …. Yet in order to record ideas,
what we propose is for each participant to write down his idea on a post-‐it note.
Ideas should be formulated clearly and include a verb (ex: use the service to do
X).
The mediator (LL) then takes the post-‐its and organizes them on a wall, creating
groups associated to “events” in the brainstorming (e.g. ideas generated while
discussing a given service, or presenting an image).
At the end of the brainstorm, each group of idea is commented together, trying to
identify the best ideas and increase the coherence of what has been proposed.
Finally, the results are analysed. Recurrent ideas are the most requested from
users. But these are not always the most relevant for the SME. In order to have
the most relevant ideas we need to organize the ideas along categories
associated to services and/or along a typology of ideas (e.g. ideas for innovation
/ ideas for business opportunities).
12
3.1.2.4 Timeline

1. End of M12: Identify POI in Issy related to the History of the Fort (done)
2. End of M13: Adapt the Urban Game to include these POI
3. 15th of December (M14)– People’s Voice Media will train Issy Média’s staff
to use their technology and become Community Reporters
4. 12th of December (M14) – Real and More will be presented at the Cité des
Sciences et de l’Industrie in Paris
5. 2nd half of M15 – Test the exchange of data between the apps
6. M16: Preparation and Communication for the first Urban Game
7. M17: Urban Game and Community Report during the “Fête de l'Internet”
8. M18: First evaluation including user feedback and SMEs (what we
integrated and how it worked, what are the new perspectives)
9. M19: Preparation and Communication for the second Urban Game
10. M20: Second Urban Game and Community Report during “Futur en Seine”
Festival
11. M21: second evaluation including user feedback and SMEs (what we
integrated and how it worked, what are the new perspectives)
12. M22: Final evaluation
1.1.2 Technology adjustments

The best scenario would have been all technologies providing APIs for easily
exchanging data with each other. Since this is the case only for Air Graffiti, we
decided to adopt an Aggregation Approach that avoids a functional integration of
technologies but still allows them to exchange data. Table 1explains what will be
exchanged.
Table 1 : What and how -‐ data exchanged across technologies
To Pilot Real and Urbadeus Air G raffiti Community

M ore Reports
From
Real and What: Inside What: Inside What: Inside

Real and
M ore More is used Real and Real and Real and

to explore More users More users More users
13
the history of can access can access can access
the Issy Fort, content content content
provoke created by created by created
curiosity and using using Air during
engage with Urbadeus Graffiti Community
citizens. How: How: Reports
Hyperlinks Hyperlinks How:
Embedded Embedded Hyperlinks
in the 3D in the 3D Embedded in
Documentary Documentary the 3D
Documentar
y
Urbadeus is What: From What: From What:

Urbadeus
and Issy used to Issy 3D and Urbadeus Urbadeus is
3D create Urbadeus users can see used during
geolocalized users can geolocalized Community
content access Real content that Reports to
during the and More has been produce and
Urban Game How: through produced geolocalize
and geolocalized using Air contents
Community hyperlinks, Graffiti
How: already
Reports embedded in How: supported
Issy 3D and through Air

Urbadeus Graffiti API Furthermore
Issy 3D is
Urbadeus What other
used to
retrieves the kinds of User
visualized
geolocalized Generated
Urbadeus and
content contents
other
produced in produced
applications'
Issy during
content on a
Community
3D map of
Reports can
Issy
be
geolocalized
14
in Urbadeus

How through
geolocalized
hyperlinks
embedded in
Issy 3D
Air Graffiti is What: From What: From

Air G raffiti
used to Air Grafitti, Air Graffiti

produce users can see users can see
geolocalized Videos, contents
content and Pictures and produced
access it on Audio created during
mobile. with Community
Urbadeus Reports
How: through How: already

a script supported
embedded in
Urbadeus

Community Real and Urbadeus is Air Graffiti is

Community
Reports Reports More is used used as a tool used as a tool
Programmes to build for producing for
are used to scenarios for and geolocalizing
train trainers Community geolocalizing content
in Issy and to Reports content during
involve during Community

citizens in Community Reports

content Reports
creation.

Based on their current state, the following adjustments will need to be carried
out on the involved technologies :
15
• Real and More : To give access to other content inside the 3D
reconstruction of the Fort, 3D2+ will have to design a solution to
visualize and access hyperlink from within the 3d environment.
• Urbadeus : To exchange content with other apps in Urbadeus, Navidis will
have to develop a script that will make available daily contents for Air
Graffiti.
• Air Grafitti : To exchange content with other apps, Air Graffiti will use its
API.
• People’s Voice Media : To adapt its Community Report training to a
French audience. This involves translating supporting material into
French before the course (with support from french Llabs) and having a
translator (French LL staff) during the course for those people that have
problems understanding English. People’s Voice Media will also have to
adapt to a new target population and will have to choose appropriate
technologies, avoiding Social Media platforms that are not localized in
French.
3.1.3 User involvement

Citizens of Issy will have a core role. They will be asked to participate as History
Explorer and produce content to enrich the different technologies. At the same
time, they will be asked to express their opinions on the technologies they are
using (as described in “User Research”) and to participate to the city life through
innovative eParticipation technologies. From this point of view, Urbadeus will
have a particularly important role of giving citizens the opportunity to express
their opinion by attributing an emotional tag to the content they create during
the urban game.
The inhabitants of Issy are generally proud of their city’s image as one of the
most advanced cities in France and proud of being an active part of it. Thus, the
majority of Issy’s citizens will voluntarily participate in the experimentation of
new services and products.
All these activities will be broadcasted on the city’s local Web TV channel
(www.issy.tv) and relayed on the city’s local magazine, local social network
16
(www.i-‐folio.fr), and on the city’s Web page (www.issy.com), a cornerstone of
Issy’s national dissemination activities.
Issy-‐les-‐Moulineaux Facebook and Twitter pages will also be used to raise
awareness on the Virtual Museum within the framework of cross-‐border Apollon
experiment.
3.2 Manchester pilot
3.2.1 Use scenario

The Manchester pilot will work in partnership with three existing projects in the
city to test and evaluate the Air Graffiti application from IBBT, the Urbadeus
application from Navidis and the Community Reporters model from Peoples
Voice Media. In relation to this, evaluation of Living Labs processes and
methodologies will also take place.
Below is an overview of the three existing projects that will be used in the
Manchester Pilot:
• Manchester Central Library closed in June 2010 for three years of

refurbishment and renovation and there is a need to keep citizens
informed of the progress of the refurbishment work. The use of QR codes
located at key locations around the renovation site and at other libraries
around the city will be used to link to online resources where citizens can
find out more about the renovation work and leave comments and
questions.
• Manchester Galleries is the Decoding Art project
(http://www.manchestergalleries.org/decodingart/), which has fixed QR
codes to twenty public art works in the city. Each QR code links to a web
page about that art work, and the page will include text about the work
and an MP3 audio clip of the text.
• The Greater Manchester County Records Office is running a ‘Big Society’
project funded by the Museums, Libraries and Archives Council
(http://www.mla.gov.uk/) to explore how to engage with citizens who
would not normally access their local archives. Images from the local
images collection of Manchester (http://images.manchester.gov.uk/) will
17
be printed on posters with QR codes and placed at the locations shown in
each photograph to encourage citizens to leave stories, comments and
other information about that location. Events will take place in local
libraries to make citizens aware of the posters and the QR codes.
Walking tours and evaluation sessions will be set up with expert users and
citizens to test and evaluate how Air Graffiti from IBBT and Urbadeus from
Navidis can be used to engage citizens in both the Central Library refurbishment
and the Decoding Art project.
User testing will evaluate subjects including usability of the applications,
language / cultural differences in the application’s interfaces, ideas for future
development and general feedback.
In addition, evaluation will also take place of the preparation work required to
enable the applications from IBBT and Navidis to be used in Manchester, such as
time required to prepare 3D model data of Manchester Central Library for use in
Urbadeus.
Peoples Voice Media will promote the Central Library, Decoding Art and Big
Society projects through their network of Community Reporters, and will work
with Manchester Central Library staff to clarify if and how the Community
Reporters model can be used by a city administration as alternative channel of
communication to citizens. For example, Community Reports will be given access
to inside the Central Library building and blog about the progress of the
renovation work, carry out interviews with key people involved in the
renovation project which will go online, and similar. Community Reports will
also be invited to be involved in the user testing sessions of the Air Graffiti and
Urbadeus applications.
3.2.2 Pilot set-up

The Manchester pilot will involve three phases:
• Preparation of applications for use in Manchester.

• User testing sessions.
• Evaluation of Living Labs processes and methodologies.
18
3.2.2.1 Preparation of applications for use in Manchester
In order to carry out successful testing of the Air Graffiti and Urbadeus
applications during user testing sessions – e.g. walking tours around public art
sites tagged with QR codes using Air Graffiti – time will need to be spent to
prepare them for use.
In the case of Urbadeus, this will involve gaining access to the 3D City Model of
Manchester produced for the city by Arup. Informal agreements have already
been set with Arup to access the data and Manchester is currently working
through the administration processes to hand over 3D data to Navidis for input
into Urbadeus.
In the case of Air Graffiti, Manchester will liaise with IBBT to ensure that the
application is ready for public use.
Timeline: November – December 2010.
3.2.2.2 User testing sessions.

The user testing sessions will consist of the following:
• Expert reviews. MDDA staff will carry out one-‐to-‐one meetings with local
experts with interests in areas of QR codes, e-‐participation / social media
and 3D applications to carry out half-‐day, in-‐depth evaluations of the Air
Graffiti and Urbadeus tools, feeding back findings from the evaluations to
IBBT and Navidis in order to support development of the applications.
These sessions will also be used to enable IBBT and Navidis to gain
knowledge of local business contacts and networks that may be useful to
expand into new markets. By proxy, MDDA will also test Living Labs
processes for cross-‐border knowledge sharing. The expert reviews will
focus on usability, market readiness, language / localisation / culture
differences, scenarios for testing the products with users in the target
market, software / hardware requirements.
• Testing with citizens. A maximum of three user testing sessions will take
place with members of the public. One user testing session will evaluate
how using QR codes and location aware comments can engage citizens in
the Manchester Central Library renovation work. Air Graffiti will be used
19
as the application to engage with citizens using QR codes and Urbadeus
will be used as the application to engage using location aware comments.
A second user testing session using both Air Graffiti and Urbadeus will be
carried out to evaluate how they can augment and support the public art
tour provided by Manchester City Galleries. A third user testing session
using Air Graffiti and Urbadeus will evaluate how they can support access
to local history archives through the use of QR codes and location aware
comments based around posters showing old images of Manchester at
various locations around the city.
Timeline:
• Expert reviews: M14-‐M15.

• User testing session 1: Manchester Galleries is the Decoding Art project
(Walking tour of public art works provided by Manchester City Galleries).
M16-‐M17, dependent on the weather.
• User testing session 2: Manchester Central Library. M17-‐M20, mainly
dependent on when the 3D city model of Manchester can be integrated
into Urbadeus.
• User testing session 3: Greater Manchester County Records Office ‘Big
Society’ project (local images of Manchester). M15-‐M17. This is
dependent on Greater Manchester County Records Office providing local
images for use on posters.

The following adjustments will need to be carried out in order to test the
products in the Manchester scenario.
• Ensuring that English is provided as a language in the user interfaces of
both Urbadeus and Air Graffiti. Air Graffiti already provides an interface
in English. Work will need to carried out with Navidis and MDDA to test
an English language interface for Urbadeus;
• Ensuring that the 3D city model of Manchester can be integrated into
Urbadeus. Agreements are already in place to access the 3D city model,
20
and Manchester is currently going through administrative exercises to
release the data.
The Manchester team will be working closely with Peoples Voice Media and
other local partners to ensure that there are adequate opportunities for feedback
built into this part of the planned local collaborative working to enable any
further adjustments that are required to be made efficiently and effectively.

In order to involve users (experts and citizens), Manchester will carry out the
following activities:
1. Decoding Art public art works tour. To involve citizens, Manchester

Galleries and MDDA will put out a call for interested citizens to take part
in a ‘walking tour’ that will test the Air Graffiti and Urbadeus applications.
The call will go out via the Manchester Galleries website, Twitter and
Facebook pages. Additionally, a call will go out via Peoples Voice Media to
ask emerging and existing Community Reporters to be involved in the
session. Users will be invited to meet at MDDA’s offices, given an
overview of the APOLLON project and the Air Graffiti and Urbadeus
applications, then taken on a walking tour where they will play with
applications. During the tour, comments from citizens will be recorded via
audio and video (if consent is provided) and text. When the tour is
finished the users will be taken back to MDDA’s offices and asked to
complete a simple evaluation form asking for their views on the
applications, ideas on how they could be used in e-‐Participation, any
technical / language / cultural issues that arose, and similar. Following
the session a summary report will be prepared and sent back to IBBT and
Navidis.
2. Manchester Central Library. This session will follow the model of (1)
above, apart from that the call for citizens will go out via Manchester
Libraries website, Twitter and Facebook pages, and also via the printed
Full Volume newsletter. Users will be taken to the Central Library
21
renovation site and asked to evaluate the Air Graffiti and Urbadeus
applications.
3. Greater Manchester County Records Office ‘Big Society’ project.
Again, this session will follow the model of (1) above. As with (2) the call
for users will go out via Manchester Libraries website, Twitter and
Facebook and also via Peoples Voice Media’s website. The user testing
session will start at a local library in a suburb of Manchester, and users
will be taken on a walking tour of locations in the area where posters of
old images of Manchester with QR codes on them have been placed. They
will use Air Graffiti to test leaving memories, comments and other
information about that place. They will also be asked to test how
Urbadeus can support information related to the images.
4. For expert reviewers, MDDA will approach local experts from
organisations and businesses with knowledge and experience of the areas
relating to the Air Graffiti and Urbadeus applications – QR codes, 3D
modelling and social media. These will include organisations such as the
Manchester Digital Laboratory, the Centre for Construction Innovation,
Social Media Manchester and Arup.
5. Additionally, for user testing, MDDA will approach citizens who have been
involved in the DEHEMS home energy monitoring Living Labs project and
ask them to be involved in the user testing session for APOLLON. Those
citizens will have experience of previous Living Labs processes and useful
information may be gathered from them by comparing their experiences
of previous Living Labs processes with current thinking.
3.3 Brussels pilot
3.3.1 Use scenario
3.3.1.1 Concept
The working title for the pilot is “Museum Quest”. Within this pilot we will
implement a serious game to increase the involvement of youth with the exhibits
in a museum. This is done by providing a game-‐based motivation to absorb
information and by steering the player towards noticing certain important
22
aspects of the exposition. The innovation in the presented pilot case lies in the
linking of a real and a virtual environment and in doing this by means of a game
concept. This will be achieved by integrating the existing virtual technology of
3D2+ with the Internet Of Things technology (Air Grafitti) of IBBT.
In terms of added value for a museum, the pilot offers two use cases. The first
one is a group of young people playing a game in a museum. This can be done in
the context of a school visit or a visit with the family. The second use case
involves non-‐player visitors that can access content related to exhibits on their
smartphones. This content is used for both use case one and two. In use case two,
non-‐gaming visitors can also add comments to the exhibits that they visit in the
museum.
Both use cases can be combined, to offer visiting families a more appealing visit
to a museum. The children then play the game, while the parents engage in a
more exhaustive visit along the lines of use case two.
3.3.1.2 Scenario
In this section, both use cases are discussed in a more hands-‐on way, to provide a
concrete impression of how the pilot will operate. This scenario will function as a
guideline for the integration of the various technologies.
Use case 1: the Museum Quest game

• The users have or loan smartphones on which they can see a badly kept
3D museum room in 2030 in which certain exhibits are missing.
• They are told that, strangely, all the stolen exhibits were exposed in the
museum in the year 2011. However, much information has been lost over
the years regarding the stolen objects. The user is asked to help the police
identify the stolen objects by going back in time, meaning to move in the
real museum in which they are currently located, and identify the missing
exhibits.
• Each of the exhibits is marked by a question mark in the 3D world. The
user can walk around in the world and click on the question marks.
Clicking on the question mark produces an overview of the remaining
23
information regarding the exhibit. The users are asked to go find the
matching works in the museum.
• Some or all of the pieces in the real museum have been marked with a QR
code. When scanning the code, the user is presented with content related
to the exhibit and an identification code. If the user believes they have
identified a missing object, they can enter the code at the location of the
missing exhibit in the 3D world. If it matches, they get points.
• A Twitter account is set up for the game. Key events that take place in the
game, like players successfully matching a missing exhibit to a real
exhibit, are logged to this twitter account. The twitter account can then be
embedded in the website of the museum to provide an overview of the
activity of the game. This could intrigue visitors of the website and
motivate them to visit the museum. In addition, the twitter account can be
visualized in the heads up display of the 3D application, to give players an
overview of what other players are doing in the game.
Use case 2: participation of museum visitor in museum exhibition
• Before entering the museum, visitors are made aware of the fact that they
can access exhibit-‐related content using their smartphones. However,
they are told that they need a scanner application to do so. For example
OptiScan for iOS costs 1.99$.
• QR codes have been placed close to exhibits. Scanning the QR code using
the scanner opens a browser window that automatically links to the Air
Grafitti page on which the content can be accessed. This can be text,
images, video or audio.
• Visitors are also offered an interface to post comments on the exhibit
about which they are viewing content. They also have access to a
Facebook “like” button, which will automatically post the exhibit to their
friend network on Facebook. Using the like button increases the exposure
of the museum and its exhibitions to the public.
24
3.3.2 Pilot set-up
3.3.2.1 Motivation for pilot reorientation

As the remainder of this deliverable will elaborate, the pilot that will be carried
out in Brussels is substantially different from the pilot that was described in
deliverable 5.3 and that has been summarized in section 2.3. The reason for this
reorientation is threefold:
A first reason is that there are a number of hindrances related to the
implementation of the campus of the future scenario. The main obstacle was the
slow progress in completing all the foreseen infrastructure to establish a
campus-‐wide living lab representing the university of the future. Especially the
usage of a student-‐card featuring an RFID chip and the linking of different
databases containing student and personnel information proved very
cumbersome. The university is an environment with a flat organizational
structure, in which decisions need to be taken by many instances before they are
actually implemented. Unfortunately, we could not wait for the decision making
process on this to be completed before starting to roll-‐out of the Apollon pilot.
Secondly, discussions with the various involved SME’s made it clear that the
business case for the campus of the future was not (yet) within their current and
future business roadmap. This is why we are switching to a more culture-‐
oriented pilot that features more overlap with the pilots that will be conducted
in Paris and Manchester. The SME’s are interested in this type of application,
making the new pilot definition more of a win-‐win situation for all involved
parties.
A final reason why we adjusted the pilot is the fact that a museum partner was
looking to experiment with a similar technology set-‐up and application as the
type of pilot we are offering. Within the role of the Living Lab as a broker for
innovation between SMEs there was an interesting opportunity for a synergetic
cooperation and by so to offer a possible opportunity for each of the
stakeholders involved.
25
3.3.2.2 Non-technical set-up
The pilot involves collaboration between different stakeholders, each with a
specific responsibility:
• The local Living Lab: The responsibility of this actors does not only
include a good integration of the various technologies, but also that this is
in-‐line with the needs and requirements of the hosting partner, i.e. the
museum. Next to that, they also are responsible for involving the users in
the development and evaluation process
• The transfer Living Lab: as they already have experience with one of the
basic technologies that will be part of the integrated pilot, their role is to
provide the necessary input with regard to user experiences on that
technology, based on the research they have conducted so far.
• Technology providers: The technology providers involved, 3D2+ and
IBBT, are responsible for integrating their technologies. They will work
together in developing the ‘Museum Quest’ concept. They are also
responsible for describing the necessary technology set-‐up and guide
both the Living Lab and museum in the actual set-‐up and deployment.
Next to the integration, they will provide the necessary monitoring
instruments to allow the logging, needed for the evaluation.
• The hosting partner: The application that is the subject of the pilot will be
tested and evaluated in a real life environment. In practice this will be a
museum setting. The museum acts as a full partner. This mean that they
will not only use the application in their daily operations, but also that
they will provide the necessary access to their premises. In addition, this
actors will be responsible for the addition of the exhibit-‐related content
that is relevant to the scenario.
26
3.3.2.3 Technical set-up
Use Case diagram

Figure 2: use case diagram for Brussels pilot
As can be seen in Figure 2, both gamers and non-‐gamer visitors can use the
functionality that will be developed in this pilot. The non-‐gaming museum visitor
will be able to scan QR codes and receive content on the related exhibit. In
addition to this, the gamer will use this content to match the missing exhibits in
the 3D world to the exhibits in the real museum. Also, the gamer will be able to
monitor the progress of the other teams that are playing the game, through a
visualization of the game’s twitter log. A similar visualization will be available on
the museum’s website, in order to draw new players to Museum Quest and thus
to the museum.
27
In terms of user-‐generated content, both the gamer and the non-‐gamer will be
able to use a “like” button to post the exhibit content to their social network in
Facebook. Non-‐gaming visitors will also have the ability to write comments on
the exhibits. Although perfectly feasible from a technical point of view, we do not
believe writing comments on an exhibit is something one would do while playing
the game. This is why we are leaving it out of the use case for the Museum Quest
game.
Someone who has access to the appropriate content and can oversee its quality
will add the content to Air Grafitti. In Figure 2, this role is called the “Museum
content provider”. Most museums already have content on their exhibits for
publication in e.g. printed catalogues and web sites, so adding the content will be
mainly a question of selecting and formatting the appropriate content. In
addition to administering the content, the museum content provider will be able
to print a QR code via Air Grafitti that can be placed close to the exhibit it is
related to. Depending of the time frame of the pilot, these QR codes can be
printed or engraved on a durable medium, or not.
3D2+ will develop the 3D world. The world will look like a museum that has
been badly kept. The world does not have to match the actual museum to match
the game concept. Air Grafitti will be used to provide content on the exhibits, and
generate QR codes that can be placed close to the exhibits in the real museum.
28
Technical architecture

Figure 3: software and hardware deployment in the integrated scenario
As can be seen in Figure 3, this scenario foresees that all the functionality of the
game can be used on one smartphone device and in the same application. The 3D
application, built in the Shiva 3D engine technology3 used by 3D2+, will be
ported to a code base that can be executed on smartphones using either iOS or
Android as operating system.
3D2+ will also integrate a QR scanner in the 3D application, so the user does not
need to leave the application when scanning a QR code and displaying the
resulting content.

3 http://www.stonetrip.com/
29
Once a QR code has been scanned in the application, the content that is hosted on
the Air Grafitti server will be read using the Air Grafitti API. This API can be
called upon through REST GET requests and produces XML data that will need to
be parsed by the application in order to be displayed in the 3D world, or in a
dialogue window on top of the application’s heads up display.
When certain events occur in the 3D application, the Twitter API4 will be invoked
and specific information will be passed in XML format. This Twitter log is then
visualized in the 3D application of other players and on the museum’s web site.
When gamers or non-‐gaming visitors use the “like” button, data is sent in JSON
format to Facebook’s Open Graph API5.
All data streams that will need to pass over internet are requests to web
platforms that produce text, images or videos that are optimized for the web.
Therefore, we expect the application to be able to read and display exhibit
content using a HSDPA connection which is readily available from Belgian mobile
carriers. Should the architectural structure of the museum prove to greatly
impair data transmissions, or should the responsiveness of the application prove
to be too low using HSDPA, we will need to look for alternatives, like installing a
temporary Wi-‐Fi network in the museum.
3.3.2.4 Timing
A number of different tasks will need to be undertaken in order to complete this
pilot. They are the following:
1. Scenario co-‐design: adaptation of the game scenario, together with the
museum and the implementation partners: 3D2+ and IBBT. In this task,
more research needs to be done on interface design and technical aspects,
like the exporting of Shiva 3D applications to mobile operating systems
and the integration in Shiva 3D of a QR scanning function. (M13-‐M15)
2. Implementation & testing: the actual creation of the application,
integration of 3D2+ and IBBT technologies and testing of the resulting
application. (M14-‐M20)

4 http://apiwiki.twitter.com/
5 http://developers.facebook.com/docs/api
30
3. Experiment & user tests: exposing users to the actual application and
tesing the effect of the application. (M18-‐M24)
3.3.3.1 3D2+
3D2+ will use the KidNet platform for this pilot. KidNet is a virtual community
for children in which each user has its own login. In order to decrease the
budget, we will use existing KidNet graphics in the pilot.. Several existing KidNet
funtionalities will be left out. For example, there is no need in the pilot to talk
with one’s friends, personalize one’s avatar or one’s island. This means the
interface of the program will have to suit the requirements of the pilot.
The following elements need to be developed to suit the pilot’s game design:
o Text explaining the game
o The design of the 3D world in the future with the missing exhibits.
o Support for the game mechanics that are part of the pilot :
§ Players can match real-‐world exhibits with missing exhibits

in the virtual world.
§ Each time the player matches a missing exhibit, this exhibit
appears in the virtual world.
o Visualisation of Twitter log in heads-‐up display
o Integration of Facebook “like” button in heads-‐up display
o Single signon, allowing the players to log in to the 3D application
and in the same stretch, with Air Graffiti.
3.3.3.2 IBBT
The Air Grafitti web-‐based application will be used by non-‐gaming visitors to
access content on exhibits in the museum, but currently has a distinct look and
feel that does not necessarily match the style of a museum. As the site can be
skinned, a new look and feel should be applied to more closely match the style of
the museum in which the pilot will be deployed.
Also, the second use case described in 3.3.1.2 mentions the possibility to “like”
content related to a certain exhibit. This like button needs to be added to the Air
Grafitti platform.
31
Another thing that needs to be taken care of is the printing of QR codes that are
large enough to be displayed in a museum. In the current version of Air Grafitti,
the generated QR codes are non-‐printable.

The user involvement, within the pilot has the objective to, on the hand, steer the
integration process and, on the other hand, evaluate the pilot. Therefore, we plan
to involve the users during the pilot phase on three levels:
1. Firstly, the end-‐users will be involved in the shaping process. This is a
two-‐step approach. The first step is an observation phase that will
investigate how people are currently visiting the museum and looking at
the various exhibits. This will be done through ethnographic observations
in-‐situ in which we will focus on how people browse the museum, how
they interact with the artworks, the interaction between them,… In a
second phase, the integration of the various technologies into the specific
Museum Quest game will be co-‐designed with users during iterative
sessions.. Here, existing results of user research from the different core-‐
technologies used (eg. 3D technology 3D2+ and Air Grafitti) will act as a
reference point. This will be done during the integration and set-‐up
phase. The co-‐design sessions will be organized with the specific target
groups (students, teachers, families) and will focus on the requirements
of each group. During these sessions, we will start with some key-‐
concepts (based on the existing technology, the initial scenario and the
results of the observation) that will be checked with these users and
transferred in the concrete design of the applications.(M14-‐M15)
2. Secondly, the users will be involved in the evaluation process. During the
deployment phase of the pilot in a real-‐life setting, we foresee a
permanent evaluation exercise with the ‘end-‐users’ of the Museum Quest.
This is done through an ex-‐ante and post-‐ evaluation exercise performed
by every user that will test the application in-‐situ. (M16-‐M20) During the
ex-‐ante phase we will make a thorough profile of the users with regard to
their ICT and game use as well as their museum experiences. During the
evaluation, a permanent logging will keep track of their activities on the
32
application level. At the end of this phase, an ex-‐post questionnaire will be
conducted to evaluate the users’ experience with regard to the
application.
3. Thirdly, we will involve the stakeholders themselves as users within the
general post-‐evaluation process. This will done through some interactive
workshops in which all stakeholders (museum, 3D2+ , AirGrafitti and LL)
are involved. (M20-‐M24) The goal of these workshops is not only to
evaluate the application and service as such but also to investigate the
process of the integration and the cross-‐border Living Lab exercise. Here,
we will use the templates from WP1. Part of this research will also focus
on how this has helped the SMEs in their business development activities.
4. Evaluation tracking
The evaluation of the pilots will be carried out following the questionnaires and
templates provided by WP1 in D1.3.
WP5 Living Labs are responsible for collecting the necessary data to evaluate the
pilots accordingly to WP1 requirements. In particular, Living Labs will be
responsible for collecting and analyzing user feedback during the “piloting
activity both in local and cross-‐border settings, and to collect and analyze SME
feedback about the benefits and problems of collaborating with a cross border
network of Living Labs.
4.1 Transfer Living Labs evaluate SME technology based on user

feedback in cross-border settings
The transfer living labs will be responsible for evaluating the technologies they
are piloting, according to the feedback they receive from normal users, experts
and local stakeholders. Living Labs will track two kinds of user data: qualitative
and quantitative. Qualitative evaluation will be done through the analysis of
users feedbacks during the piloting activities. Since each pilot foresees different
levels of user engagement, each living lab will have to define its methodology for
collecting these feedbacks. One example is the co-‐innovation workshops adopted
for the French Pilot. Living Labs will exchange best practices to achieve the best
33
coherence in the evaluation method in order to obtain comparable data.
Common templates for collecting user feedback will be defined by the end of
January 2011 (M15).
In terms of quantitative evaluation, each SME will record its technology logs during
the piloting activities in order to monitor how many users use the applications and
how much content they produce. The local living labs will make sure these logs are
active and up-to- date. The transfer living lab is responsible for analyzing the logs for
its pilot along with qualitative data.
These data will be collected during the pilot. The final evaluation will be completed
by the end of July 2011 (M21).
4.2 SME evaluation of the collaboration with Living Labs in the

cross-border experiments
In order to collect SME feedback about their collaboration with living labs in
cross-‐border settings, each participating SME will be interviewed two times :
once by the local living lab and once by the transfer living lab. The local living lab
will interview the SME about its cross-‐border experience and how they
collaborated with the transfer living lab. The goal is to identify the problems
encountered and the role of the transfer living lab in supporting the SME. The
transfer living lab will interview the SME about its collaboration with the local
living lab. The goal is to evaluate how the local living lab supported the SMEs in
preparing and conducting the cross border activity.
Furthermore, in order to keep track of new contacts and business opportunities
that SMEs discovered thanks to the cross-‐border activity in APOLLON, each
Living Lab will draw a graph, representing the connections (projects, partners,
users) that local SMEs had at the beginning of the cross-‐border pilot They will
keep this graph updated with new connections during the piloting activity.
34

Apollon - Eparticipation Re-Adjusted Pilot Techniques and Methodology

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Apollon - Eparticipation Re-Adjusted Pilot Techniques and Methodology

Uploaded by

Copyright:

Available Formats

Project Acronym: APOLLON

Grant Agreement 250516

Project Title: Advanced Pilots of Living Labs Operating in

D.5.4 Re-adjusted pilot techniques and methodology

Tanguy Coenen, Bram Lievens (IBBT)

2. Intial pilot description

2.2 Manchester pilot

2.3 Brussels pilot

3.1.1 Use scenario

The realization of the pilot will involve two steps:

3.1.1.1 First Phase : raising awareness

3.1.1.2 Second Phase: User Involvement through an Urban Game and

The available resources in the POI will be:

3.1.2 Pilot set-up

People's Voice Media

3.1.2.3 User research

1.1.2 Technology adjustments

Table 1 : What and how -­‐ data exchanged across technologies

To Pilot Real and Urbadeus Air G raffiti Community

Real and What: Inside What: Inside What: Inside

Urbadeus is What: From What: From What:

Air Graffiti is What: From What: From

How: through How: already

Community Real and Urbadeus is Air Graffiti is

3.1.3 User involvement

3.2 Manchester pilot

3.2.1 Use scenario

• Manchester Central Library closed in June 2010 for three years of

3.2.2 Pilot set-up

• Preparation of applications for use in Manchester.

Timeline: November – December 2010.

3.2.2.2 User testing sessions.

• Expert reviews: M14-­‐M15.

3.2.3 Technology adjustments

3.2.4 User involvement

1. Decoding Art public art works tour. To involve citizens, Manchester

3.3 Brussels pilot

3.3.1 Use scenario

Use case 1: the Museum Quest game

3.3.2.1 Motivation for pilot reorientation

Use Case diagram

3.3.3 Technology adjustments

§ Players can match real-­‐world exhibits with missing exhibits

3.3.4 User involvement

4.1 Transfer Living Labs evaluate SME technology based on user

4.2 SME evaluation of the collaboration with Living Labs in the

You might also like

Table 1 : What and how -‐ data exchanged across technologies

• Expert reviews: M14-‐M15.

§ Players can match real-‐world exhibits with missing exhibits