JID: ESWA

ARTICLE IN PRESS [m5G;November 2, 2016;11:3]

Expert Systems With Applications 0 0 0 (2016) 1–14

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

A recommendation approach for consuming linked open data

Jonice Oliveira1,∗, Carla Delgado, Ana Carolina Assaife
Universidade Federal do Rio de Janeiro, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: Most of linked open data (LOD) applications focus on the search and visualization of information, not ef-
Received 6 April 2016 ficiently using the links among objects in different data sources and the semantics of their relations. This
Revised 16 October 2016
work aims to create a LOD-consuming approach that uses recommendation techniques based on items’
Accepted 16 October 2016
description, their relations, users’ interests and social network. The proposed approach was instantiated
Available online xxx
by an application that uses movie related LOD. The results obtained in our experiments were promis-
Keywords: ing: accuracy of the recommendations generated was equal or better, compared to other recommender
Information retrieval algorithms used in conventional (not LOD) scenario.
Linked open data © 2016 Elsevier Ltd. All rights reserved.
Recommender systems

1. Introduction Identifier) (Berners-Lee, Fielding, & Masinter, 1998). What makes

The World Wide Web (WWW, or Web for short) changed the
way people access and share knowledge (Berners-Lee, Cailliau,
Groff, & Pollermann, 1992). Barriers in document publication and
sharing have been considerably reduced, causing the global infor-
mation space to grow. In spite of the efforts to improve data con-
sumption, the data published in the Web still form knowledge is-
lands. Also, document search is still based on keywords, hetero-
geneous data models and data types, and no pattern to resource
identification has been established. As documents were primarily
made to be understood by humans, document integration used to
be a manual process. This makes the computational processing and
information inference on published documents very difficult.
Because of the difficulties in automatically extracting mean-
ing from documents, the Web evolved to the so called “Web of
Data”, incorporating mechanisms for a better semantic representa-
tion and integration of Web publications (Bizer, Heath, & Berners-
Lee, 2009). The concept of Linked Open Data (LOD) establishes the
principles for data publication and data relation, and motivates the
creation of links with associated meaning among data from sev-
eral sources (Bizer, Cyganiak, & Heath, 2007). In LOD, the infor-
mation is represented in RDF (Resource Description Framework)
triple graphs (Klyne & Carroll, 2004) where world elements are
represented by unique identifiers called URIs (Uniform Resource
∗
Corresponding author.
E-mail addresses: jonice@dcc.ufrj.br, jonice@gmail.com
carla@dcc.ufrj.br (C. Delgado), ana.assaife@ppgi.ufrj.br (A.C. Assaife).
1
Authors would like to thank CNPq, CAPES and FAPERJ for partially supporting
this research work.
LOD particularly interesting is the huge volume of structured data
available, which can be consumed by humans or artificial agents
(computer programs) and also, the links among items of different
sorts, as for example, links among person data bases to movies’
data bases.
With this mobilization from the community and with the grow-
ing volume of published data, research was conducted and appli-
cations developed to improve the means to consume and explore
the Web of Data. According to Bizer et al. (2009), the main advan-
tage that linked data brings to the user is the possibility to access
related or combined data from distributed and even heterogeneous
data sources. Also, according to the same authors, using the Web
as a unique global database has a couple of challenges, as for ex-
ample the construction of applications that ease the user interac-
tion with the data. This way, new contributions to the fields of data
navigation, data search and data consumption is welcome.
The support for users that consume information published as
LOD is still very limited: such applications are in majority focused
in the search and visualization of the retrieved information. They
are able to show related vocabulary, exhibit the data in facets, but
they do not suggest related information to catch the user inter-
est (Franz, Koch, Dividino, & Staab, 2010). It is still a challenge to
make the consumption of LOD attractive to the end user. Our goal
is to tackle the problem of easing the consumption of LOD to the
end user, as well as support the construction of applications that
would motivate the user to consume LOD. We believe that this is
an important step towards using the web as a unique global data
(J. Oliveira),
repository to serve all sorts of end users’ applications.
After studying the related works on the field of end users’
LOD consumption and navigation, we noticed that recommenda-

http://dx.doi.org/10.1016/j.eswa.2016.10.037
0957-4174/© 2016 Elsevier Ltd. All rights reserved.

Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
JID: ESWA
ARTICLE IN PRESS
2 J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14
tion mechanisms are applied to ease and motivate the consump-
tion of such data (Heitmann & Hayes, 2010; Di Noia, Cantador, &
Ostuni, 2014; Musto, Lops, Basile, de Gemmis, & Semeraro, 2016, to
cite some). In this scenario of recommendation systems using LOD,
previous works used to combine LOD with other closed systems, as
for instance Facebook, MySpace and LastFM (Mirizzi, Noia, Ragone,
Ostuni, & Sciascio, 2012; Heitmann & Hayes, 2010; Passant, 2010),
envisioning the improvement of the recommendations by the use
of more structured and semantically richer data about the user.
The improvement of recommendations using only LOD data – on
a way to work only with open data on a self-sustainable LOD uni-
verse, and not requiring user information – was an open problem.
The presented approach is based on this challenge: working with
the recommendation of LOD using one or more datasets – and only
LOD datasets – enabling the end users to navigate, by the recom-
mendations, in LOD.
This paper describes a recommendation approach to LOD, using
one or more datasets. This is the main contribution, considering
that previous works are limited to the use of only one dataset. The
approach here presented is based on three recommendation types:
(1) Recommendation of resources of the same type of the input re-
source, considering that the input resource is a resource that
interests the user, or that the user likes;
(2) Recommendation of social networks, that recommends people
from the same domain as the input resource;
(3) Recommendation of resources of any type that are related to
the input resource, using the textual description of the re-
sources to find related items. This empowers the identification
of related resources from different types. For instance, when
the user searches for a movie, it is possible to recommend peo-
ple, books, soundtracks or any other resource that the algo-
rithm considers similar to the movie given as start point for
the search.
As we said previously, our goal is to tackle the problem of eas-
ing the consumption of LOD to the end user. Consequently, we
would like to analyze how our approach could be feasible and ef-
ficient from the user’s point-of-view. An experiment, divided in
two stages, was conducted to validate this work. Firstly, we con-
duct a quasi-experiment, aimed at evaluating the end user percep-
tion of the recommendation technique. In this stage, a group used
our system and we collected information about the execution and
their perception. These results were used in the second phase of
the evaluation, which was a quantitative off-line analysis. In this
stage, we measured a set of recommendation metrics for the pro-
posed algorithms (RICCI et al., 2011), such as: accuracy, prediction
accuracy by ranking, novelty and utility. All the analysis based on
the user’s feedback and the four metrics were positive and, con-
sequently, we could affirm that this approach brings contributions
for Consuming Linked Open Data scenario.
This paper is organized as follows. Section 2 presents the works
related to the use of LOD in recommendations. Our proposal is
detailed in section 3. In Section 4 our proposal is applied to the
movie scenario, and the validation experiments are described in
Section 5, as well as the results obtained. Conclusions and future
work come in Section 6.
2. Related work
Considerations on how to take advantage of LOD have been
raised since real LOD bases became available (Heath, 2008). Due
the absence of works in the area, Passant (2010) proposed a set
of measures to compute the semantic distance in LOD in or-
der to measure similarity between two resources, and discussed
how such measures could be used in applications, such as re-
sources’ recommendations. The measures proposed were based on
Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
[m5G;November 2, 2016;11:3]
the abundance of links among the resources and do not consider
the ontology hierarchy describing this data. Passant conducted an
experiment to decide which of the distances (direct, indirect, bal-
anced and combined distances) would be more appropriate for a
recommender system. The combined balanced distance was the
one performing best.
Later, the same author (Passant & Decker, 2010; Passant, 2010)
applied the created methodology to develop a system to recom-
mend songs called dbrec (Passant, 2010). The output of this system
was validated by making a comparison to Last.fm, a music recom-
mender system already established. As we will see later, the main
difference to our work is the restriction of the items to be com-
pared. In (Passant, 2010) just resources that have the properties
dbpedia:Band and/or dbpedia:MusicalArtist are taken into account.
In our approach, broader criteria are used (correlated properties,
similarities in descriptions for a resource). Another aspect observed
in this related work is that it just uses one database, thus not ben-
efiting from the links found in several bases and also DBPedia.
Still in the music domain, the Seevl application is worth men-
tioning (Passant, 2011). It is a Chrome plugin that provides mu-
sic recommendations, biographies and complementary information
to youtube videos. Once extracted and converted to RDF, the data
about artists is translated into a common vocabulary using the Mu-
sic Ontology Raimond et al. (2007), forming a graph of musical en-
tities like artists, bands, genres, labels, and others. Seevl also pro-
vides combined search of properties, making it possible to search
“all artists and bands of rock genre that played in a particular fes-
tival”. In spite of presenting the used architecture and the steps
in the process, this work did not bring any information about the
implementation of the recommendation algorithms used. But as it
is from the same author of the previous works described in this
section, and using data from the same domain, we believe that
Seevl is a mature application developed after other more theoreti-
cal works from the same author.
Also in the field of media files, MORE (Mirizzi et al., 2012) is
an application developed to complement Facebook by recommend-
ing movies. MORE uses the data from bases of triplets in the do-
main, and compute movie similarities. The special thing about this
application is that besides looking at the content, it also consid-
ers the user’s Facebook profile. By using as input information the
movies that the user marked as interesting in his or her Face-
book profile, MORE intends to overcome the “cold start” problem,
i.e. having some information even about users that have not yet
rated any movie. The main contribution of this work is that it pro-
vides a semantic recommendation algorithm, called Semantic Vec-
tor Space Model. The algorithm is an adaptation from the Vector
Space Model (Salton, Wong, & Yang, 1975). In spite of exploring
the same domain as our work (movies), this work is not strongly
related to ours as it was not our goal to use user profile informa-
tion, as this information is not always available. Our goal was to
generate recommendations from RDF databases.
Other project worth mentioning is RKBExplorer.com (Glaser,
Millard, & Jaffri, 2008). It is a semantic web application that aims
to present unified versions from a considerable number of differ-
ent data sources. Up to now, the application works in the domain
of people and publications in Computer Science. The data about
people, publications and institutions is in RDF, and have been pro-
vided by the project partners in ReSIST (Glaser & Millard, 2007).
This application provides an interface where the user specifies two
resources of his or her interest and the type of the relation be-
tween them, and gets as an output the properties that link these
resources. This application requires that the user knows both the
input resources and the type of each resource. This is a limitation
in the sense that there is no way to surprise the user recommend-
ing novel items. In our approach the user does not need to know
anything about the resources nor the datasets.
JID: ESWA
ARTICLE IN PRESS
J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14
Fig. 1. Architecture of the proposed recommendation system.
Heitmann & Hayes, 2010 proposed an architecture for open rec-
ommender systems based on collaborative filtering. Their work fo-
cuses on using collaborative filtering to exploit simple, binary con-
nections between users and items. They report very good results
on using LOD bases to mitigate the “cold start” problem, a known
problem of collaborative filtering recommenders which means that
for users that did not yet rate many things, or items that were not
yet sufficiently rated, it is very hard to produce recommendations.
Basically the LOD resources are used to solve the data acquisition
problem, filling in the gaps when user data is missing for a spe-
cific domain. But different from our approach, the LOD bases are
not the source of the items being recommended.
Ostuni, Di Noia, Di Sciascio, & Mirizzi, 2013 present SPrank (Se-
mantic Path-based ranking), a hybrid recommendation algorithm
to compute top-N item recommendations from implicit feedback,
that combines ontological knowledge extracted from DBpedia with
collaborative user preferences in a graph-based setting. Sprank’s
strength is to address the top-N item recommendation task and
at the same time, deal with implicit feedback datasets. The main
idea is exploring paths in a semantic graph in order to find items
that are related in some sense to the ones the user is interested
in. From the analysis of these paths, path-based features (extracted
from DBpedia) are extracted and a learning algorithm is applied to
obtain a ranking function able to recommend the most relevant
items to the user. The whole proposal of Sprank is very different
from what we are proposing, but two meaningful differences from
our system to Sprank is that it uses just one source of LOD (DB-
pedia), and that it sticks to recommending items of the same type
(in the paper, they mention recommendation of either movies or
songs, but not a combination of both).
In 2014, the ESWC 2014 Challenge on Linked Open Data-
enabled Recommender Systems was launched. The primary goal of
the Challenge was both to create a link between the Semantic Web
and the Recommender Systems communities and also to show
how Linked Open Data and semantic technologies can boost the
creation of a new breed of knowledge-enabled and content-based
recommender systems (Di Noia et al., 2014). The challenge focused
on the particular scenario of book recommendation, and had three
tasks: rating prediction in cold-start situations, top-N recommen-
dations from binary user feedback, and diversity in content-based
recommendations. Though our example aims at the movie sce-
nario, we also addressed some of the challenges in our experi-
ments, and our results converge to the ones that emerged from an
analysis of the different approaches proposed by the participants
of the ESWC 2014 challenge: the best performing techniques, with
respect to the provided dataset, for rating prediction and top-N
Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
[m5G;November 2, 2016;11:3]
3
recommendations use an ensemble of several different recommen-
dation methods, while post-processing results are very effective in
increasing the diversity of the recommendation list. Also, we be-
lieve that our work goes beyond the tasks proposed, as we explore
the recommendation of items that are not only movies, but belong
to the universe of movies. This way, we invest on improving met-
rics such as diversity, novelty and serendipity.
Piao and Breslin (2016) proposed various distance measures, on
top of the basic concept of Linked Data Semantic Distance (LDSD)
(Passant, 2010), for calculating Linked Data semantic distance be-
tween resources that can be used in a LOD-enabled recommender
system. Their results show that the performance can be signif-
icantly improved beyond the usage of LDSD based metrics. We
did not use Piao and Breslin (2016) measures in our system, as
we wanted to stick to the basics, so we could better compare it
to other systems. But this opens possibilities for further improve-
ments that we plan to investigate.
Musto et al. (2016) reports the implementation of a graph-
based recommendation methodology based on the personalized
PageRank algorithm. The graph used represents both user and
item’s as nodes from a bipartite graph, and the edges between
users’ nodes and items’ nodes represent the feedback information
on how the item interests the user. This information is used to im-
plement the collaborative filtering approach, which is not the goal
of our work. But the proposal from Musto et al. (2016) and ours
have in common the usage of LOD. The results presented in Musto
et al. (2016) confirm that knowledge coming from the LOD cloud
can positively impact on the recommendation algorithm.
3. A model for recommendations using linked open data
According to Burke (2002), traditional recommender systems’
architecture has three basic components: i) the data provided by
the user (for which the system should find something related),
ii) the data the system already has about other items or other
users or both and iii) the recommendation algorithm. As pointed
by Heitmann and Hayes (2010), when using LOD as input data, this
general architecture should be extended with the creation of an
abstraction layer for accessing the data. We found it necessary to
detail even more the data abstraction layer, by the creation of a
data treatment layer. This was important because LOD are mainly
raw data and when more than one base is used, it is essential to
work the data so that interlinking data across the bases is pos-
sible. So adaptions were made to Heitmann and Hayes’ proposal.
Fig. 1 shows the architecture used, with the phases to be followed
and their relations.
JID: ESWA
ARTICLE IN PRESS [m5G;November 2, 2016;11:3]

4 J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14

The first phase of the architecture is data treatment, which has three subtasks:

(1) data access

(2) data interlinking
(3) treated data storage

The goal of data treatment is to leave the data ready to be consumed by the recommendation system. The domain and the datasets to
be used have to be chosen before this phase begins.
To explore the possibility of using several triplet datasets, it is necessary to build an access layer to the data. The goal of the access
layer is to make any available data retrievable as a triple by a query. So, after the data is extracted from the data cloud, it is treated and
stored locally. But it is not enough to store the data, this data also needs to be integrated. This is an important phase as each dataset has
its own ontology describing its terms, and not always the vocabulary adopted is the same. Besides, each base has an URI to identify an
element that rarely has the same URI in other base, even if semantically both bases would mean the same element. Because of that, a
mapping is made during the data treatment phase so that data in one base is associated to equivalent data in another base.
In order to create a semantic link between data items, it is necessary to specify the source and the destination bases, as well as the
possible link types (sameAs, differentFrom, equivalentProperty, inverseOf). The similarity measure between the items is then used to define
the link creation. There are different approaches to measure similarity between two data item, as for example the string distance of values
of properties. This is the one that was used. Also, the threshold to define when similarity is high can be chosen according to the specific
domain. A high level procedure of the data interlinking procedure follows:

The data interlinking phase leaves the data ready to be used by the recommendation algorithms. At this point the open recommenda-
tion system turns, with some adaptations, into a closed recommendation system. The treated data, the recommendation algorithms, and
the input URI become the three system’s basic components. The input URI corresponds to the resource based on which a recommendation
should be computed when compared to the data that was prepared and stored in previous phases. Providing these two inputs needed by
recommendation algorithms makes it possible to recommend to the user items from the integrated datasets. So next we explain how the
recommendations are made. This is done by the recommendation module of our architecture.
The recommendation module is divided in three parts: recommendation of resources from the same type of the input resource, recom-
mendation of people based on social networks and recommendation of resources based on textual description. According to the desired
recommendation type, the system switches among three different algorithms, as seen in the pseudo-code below.

The recommendation of resources from the same type of the input resource consists in providing to the user recommendations of re-
sources from the same type, and belonging to the same dataset, as the resource (URI) given as input. This is the simplest recommendation,
as only one information source is used. The user provides a URI of a resource from a specific type, belonging to a dataset of his or her
choice. This URI feeds the recommendation module, that compares the input with all other resources from the same type and from the
same dataset. This way, the output is a set of resources from the same dataset as the input, ordered by similarity. The guidelines to this
recommendation process can be seen in Procedure Rec-ResourcesSameType, given below.

The second recommendation proposed is the recommendation of the social network around a resource, in other words, recommenda-
tion based on the network of people around the input resource. Considering an input resource (i) from a dataset source (parameter ds)
chosen by the user, the mapping (previously prepared at the interlinking phase) finds the corresponding resource in other LOD datasets
(parameter dd). Our main intention in this approach is to be independent of domain, which means, it could be applied in several scenar-
ios. For this, we based our search on the property “type” (in our example, as we are looking for movies, we will search by all items which
have the property type defined as “movie”).

Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
JID: ESWA
ARTICLE IN PRESS [m5G;November 2, 2016;11:3]

J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14 5

The next step is to identify all people related to the corresponding resource. To find them, we search by resources having the type
“person”. For each person, we will search by other people related to him/her because they are considered collaborators, creating a (social)
network connected to the original resource. In our example, for each movie, we will search by resources linked to it, which have the
type property with value “person”. Resources of type “person” related to resources of type “movie” could represent actors, directors,
screenwriters, and others. After that, we will identify the social network related to that movie, which is people which worked or have
other relationships with each person related with this movie. People with the biggest number of collaborations with the people related to
the input resource compose the output of this recommendation. The threshold, that is, the degree of relationships is a variable defined by
the user. This step can be described as:

The third and last type of recommendation developed is recommendation of resources using textual descriptions. Here again more than
one dataset is used as data source for the recommendation, and the resource mapping among datasets is important. The user provides
an input resource from a specific dataset (URI), then corresponding resources to the input URI are found in other datasets and similar
resources are computed considering the analysis of texts found in the description properties of the resources. The resources’ description
texts that are more similar to the description text of the input resource form the output set of this recommendation. Here, resources of
any type can be recommended (not necessarily resources from the same type as the input resource will be recommended). The procedure
Rec-Description illustrates this activity.

To clarify the use of this approach, in next section we describe an example, detailing all the steps of this architecture.

4. Application example: movies domain

This section details the architecture components Data Access layer, Data Interlinking layer, and Recommendation Algorithms, contex-
tualized in the chosen domain: movies. We discuss in detail how each recommendation type is computed, in order to highlight relevant
implementation aspects and tools used.

4.1. Data access layer

As discussed before, this layer prepares the data for the next layers. The goal is to extract and load the data so that it can be used
by the Recommendation Algorithms. Two data sources were selected to generate recommendations: Linked Movie Database and DBPedia.
Both sources allow remote access through SPARQL Endpoints (Prud’hommeaux et al., 2007). Even though the developed algorithms can be
easily translated to SPARQL language, this approach has shown several limitations: it is highly dependent on the servers’ availability, and
the amount of data accessed during the execution of the recommendation algorithms requires a very large number of requisitions to the
endpoints, turning the process very slow. Some public endpoints as DBPedia limit the number of replies to each query, forcing the use of
nested subqueries to obtain the data needed (Passant & Decker, 2010).
Because of these restrictions, we followed the approach of some related works (Mirizzi et al., 2012; Passant, 2010) and opted for the
local storage of the data. Both selected datasets provide up to date dumps, so recommendations’ computation can be done locally. The
Linked Movie Database provides a unique dump that was completely used. DBPedia on the other hand provides segmented files, from
which the following were used:

Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
JID: ESWA
ARTICLE IN PRESS
6 J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14
– Ontology infobox types: This file contains triplets in the form
<rdf object:object type class>. This data allows to identify, for
example, if the resource corresponds to a person, to a city, to a
movie, and so on. For the recommendation of persons, it is nec-
essary to query this group of triplets in order to filter resources
of the desired type.
– Ontology infobox properties: This is the biggest file, considered
the center of the dataset. The resource properties used by the
recommendation algorithms are in there.
– Short abstracts: This is an essential file for the computation of
the text based recommendation. There lay the abstracts that
are extracted from Wikipedia articles. The objects from these
triplets are abstracts that describe the resources in texts up to
500 characters.
– Links to LinkedMDB: This file holds the equivalence links from
DBPedia to the Linked Movie Database. It contains triplets in-
forming that a specific resource in DBPedia corresponds to (rep-
resents the same object as) another resource in Linked Movie
Database, even if the URIs from both resources are not the
same.
Being the data ready and locally stored, the next step is to
construct a data interlinking mechanism so that the different data
sources can be cross referenced.
4.2. Data interlinking
Two from the three recommendation algorithms proposed in
this work use data from more than one data source. If the chosen
data sources do not have a common data schema, a tool to map re-
sources from one source to the other is necessary. In other words,
we need a mechanism to find out if a resource in one dataset is
equivalent to a resource in another dataset.
In the movies domain, we should obtain the equivalence among
resources that represent the same movie in LinkedMovieDB and
DBPedia. LinkedMovieDB has some links to DBPedia, but in spite
of that the number of relations owl:SameAs among movies in both
datasets is still small in relation to the total number of movies. As
shown in Fig. 2, the movie “Billy Elliot” is an example of those that
do not have a property owl:SameAs that points to DBpedia. The
same movie is represented by different URIs in each datasource,
as can be seen in Fig. 3, and there is no connection among them.
As described in Procedure Interlinking in Section 2, we need to
measure the similarity between items and link them. We used Silk
(Volz, Bizer, Gaedke, & Kobilarov, 2009) to do that, and a similar-
ity relation of the kind owl:SameAs was created, as represented by
the dotted line in Fig. 3.
A configuration file must be provided to Silk, describing the
type of link that should be created, the data sources that should be
used, how the data will be accessed, and some other parameters.
Fig. 4 shows the part of this configuration file which deals with the
data access to the movie recommendation system. In this part, we
have to specify the addresses of the access points where will be
connected and used. In this example, DBPedia’s and LinkedMDB’s
access points are configured.
The type of the new links (that will be created) must also
be specified. For this movie recommender, a link of the type
owl:SameAs should be created among equal items from different
datasets. The source resources are from LinkedMDB and are re-
stricted to the type http://data.linkedmdb.org/resource/movie/film.
These resources will be linked to destination resources from DBPe-
dia limited to the type http://dbpedia.org/ontology/Film. This con-
figuration is illustrated in Fig. 5.
To define how the source and destination resources should
be compared, another file configuration is needed. Fig. 6 illus-
trates this process, where the URIs of the different datasets will
Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
[m5G;November 2, 2016;11:3]
be compared using their labels (represented by the property rdf-
schema:label). In this step, we used the Jaro metric (Volz et al.,
2009), provided by Silk because its usage very common. It is a
measure of similarity between two strings.
It is important to emphasize that all the configuration process
(represented in Figs. 4–6) is manually specified.
Silk execution (with the aforementioned configurations and set-
tings) results in a set of RDF triplets. The subjects are the source
resources from LinkedMDB, the properties created are owl:SameAs
and the objects correspond to destination resources from DBPedia.
Fig. 7 shows the generated result for the movie “Billy Elliot”. Using
this mapping, the recommendation algorithms that use more than
one dataset can, from a given resource in LinkedMDB, consider re-
sources from DBPedia.
After Data Interlinking, the generated triplets are stored locally
together with the data extracted from the data sources. The re-
sulting dataset is then ready to be accessed and consumed by the
recommendation algorithms.
4.3. Recommendation of resources from the same type of the input
resource: movies
In this case, when the input of the recommendation algorithm
has “movie” as its type property, the recommended output re-
sources will necessarily have this same type property. Besides, all
output URIs come from the same data source as the input resource.
From the three recommendation types developed in this work, this
is the only one that can be found in the literature (Mirizzi et al.,
2012; Passant & Decker, 2010; Passant, 2010). As described in Pro-
cedure Rec-ResourcesSameType in Section 2, we need to measure
the similarity between items. We opted for the use of the Linked
Data Semantic Distance (LDSD) (Passant, 2010), more specifically
the Balanced Combined Distance (LDSDcw). An output file is gener-
ated after the computation of the recommendation using LDSDcw.
This file has the URI of the input resource, the URI of the movie
being compared, the similarity value found and the recommenda-
tion type (F, from film).
4.4. Recommendation of the social network around a resource
The movie “Across the Universe”1 will be used to explain the
steps of the algorithm used to recommend people.
The idea behind this process is to recommend the social net-
work in which the input resource is inserted. For our example, we
consider as input resource the movie “Across the Universe”. The
first step is to discover the corresponding URI of the movie in DB-
Pedia, using the mapping done with the Silk tool, previously ex-
plained. For the next steps, the algorithm uses the URI from DB-
Pedia, http://dbpedia.org/resource/Across_the_Universe_(film). This
movie is related to different resources, by several properties. The
next step is to find out which of these properties are linked to re-
sources of the type “person”, as for example actors, writers and
movie directors, which we consider the “collaborators” regarding
the movie. After identifying all people related to the movie, for
each person found, the algorithm searches for the movies in which
the person took part (navigating by linking properties of interest).
This process is then repeated for all people related to the movie
“Across the Universe”, and the movies found are gathered in a list.
With the generated list of movies, a new list is generated with
all people that worked or are somehow related to these movies (so,
the collaborators of these movies). The persons that appear more
times in the list are the ones to be recommended. The output of
1
corresponding URI in LinkedMDB is http://data.linkedmdb.org/resource/film/
6345
JID: ESWA
ARTICLE IN PRESS [m5G;November 2, 2016;11:3]

J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14 7

Fig. 2. Properties of the movie Billy Elliot in LinkedMDB.

the algorithm is a ranked list of persons. We say that the list is

ranked by the most number of collaborations. The social network
recommendation delivers an output text file containing the URI of
the input resource, the corresponding URI for each recommended
person, the rank number computed by the algorithm, and the rec-
ommendation type (P, from people).

4.5. Recommendation from resources using textual description

When instantiating this recommendation type to the movie do-

Fig. 3. Representation schema of the movie Billy Elliot in LinkedMDB and DBPedia.
main, the descriptions or synopsis of the movie resources were
used as the texts. DBPedia offers two types of text descriptions:
short ones, limited to 500 characters, and complete ones, of free
size. Small texts were chosen because of text indexing limitations.

Fig. 4. Data source configuration in Silk.

Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
JID: ESWA
ARTICLE IN PRESS
8 J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14
Fig. 5. Configuration of the type of link, source and target in Silk.
Fig. 6. Configuration of the type of comparison in Silk.
Fig. 7. Silk result for the movie “Billy Eliot”.
The main goal of this recommendation type is to recommend
resources of any type to the user, based on how similar their text
description is in relation to the text description of the input re-
source. The proposed method gets as input a resource from Linked-
MovieDB. Using the mapping built by the integration layer, the cor-
responding resource in DBPedia is found and the algorithm uses
the URI from DBPedia for the next steps. It is important to re-
member that short descriptions from DBPedia resources (among
other data) were locally stored during the Data Integration phase.
Through the analysis of these short descriptions, the recommenda-
tion algorithm delivers 10 resources considered similar to the input
resource.
As seen in Procedure Rec-Description in Section 2, this recom-
mendation makes use of text mining techniques to do the recog-
nition of most relevant terms from description texts. For this, the
Lucene library2 was used to compute the similarity search among
the texts. The first step is the creation of an inverted index (Frakes
and Baeza-Yates, 1992) to locate terms from the description texts
in the local copy of DBPedia data. Lucene’s StandardAnalyzer li-
brary was used to process the short descriptions and define key
words that appear in the index. Among other tasks, this library re-
moves stop words and converts all letters to capitals.
After locating the short description of the input resource, it is
possible to query the index in order to obtain the resources that
have associated texts’ most similar to the one of the input re-
source. The output of the algorithm is a text file containing: the
URI of the input resource, the URIs of the recommended resources,
the similarity score and the recommendation type (A, for all).
2 3
Apache Lucene, http://lucene.apache.org/
Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
[m5G;November 2, 2016;11:3]
5. Experiments and evaluation
This section details the experiment that we conducted to eval-
uate our approach. Specifically, it is important to find out its feasi-
bility and its efficacy from the user’s point-of-view.
At first, a quasi- experiment was conducted to let a group of
users validate the application. At second, users’ feedback obtained
in the quasi-experiment was used to analyze: accuracy, prediction
accuracy by ranking, novelty and utility.
For the quasi experiment 50 famous movies were selected and,
for each of them, the three recommendation types were used. This
experiment generated 150 text files, 3 for each movie, with the
recommendations of other movies, people and resources. In order
that users could browse the data, receive and evaluate recommen-
dations, a graphical interface was also implemented.
The experiment was done from the perspective of a recom-
mender systems’ user, so the participants should evaluate if the
provided recommendations were useful, in relation to the movie
provided as input. The desired user profile for a participant was
someone familiarized with the use of recommender systems and
that had an interest for movies. The experiment participants were
35 Computer Science undergraduate students from UFRJ3 from the
3rd year or further. Participants received a two hour talk about the
motivation of the application and the general idea of the work, be-
sides instructions on how to use the system. They were asked to
choose one to five movies that they had already seen and liked,
and then to evaluate the recommendations generated when giv-
ing each of these movies as input. Participants’ impressions on the
recommendations were collect by asking them to answer a ques-
tionnaire, where for each recommendation they had to choose one
among each of the following options:
– The user knows the recommended item and finds it interesting;
– The user knows the recommended item and finds it uninterest-
ing;
– The user does not know the recommended item, but is inter-
ested in knowing it; or
– The user does not know the recommended item and is not in-
terested in knowing about it.
A binary scale was adopted: either the user is interested or not.
In total, 97 questionnaires were answered. All participants
started at the same lab and at the same time, the last participants
left the lab in 2 h time. The accuracy of the recommendations was
used to estimate how close the recommendations generated were
to the real preferences of the users. This criterion was also used
to compare the three recommendation types among them, and to
investigate which recommendation type most surprised the user.
The null hypothesis of this experiment is: “The system is not
able to provide good quality recommendations”. This means that
the system cannot recommend items that might interest the user.
The alternative hypothesis is: “The system is able to provide
good quality recommendations”. A “good quality recommendation”
brings correct results that will give some benefit for the user (new
acquisitions, knowledge, entertainment, etc). Some of the proper-
ties are traded-off, the most obvious example perhaps is the de-
cline in accuracy when other properties (e.g. diversity) are im-
proved. Based on RICCI et al. (2011), we consider as the main prop-
erties of a “good quality recommendation”:
Accuracy average – it is one of the most important dimen-
sions to be analyzed. It was obtained for each of the three types of
recommendation. The computation of this metric can be done by
dividing the number of correct recommendations (items that the
user was interested in) by the total of recommendations provided.
Universidade Federal do Rio de Janeiro
JID: ESWA
ARTICLE IN PRESS
J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14
Here, we compare our results with Passant (2011), which is a state
of the art. In case the results show that accuracy of the proposed
algorithms is higher or equal to this reference, our alternative hy-
pothesis will be accepted.
• Prediction accuracy by ranking – It may predict user opinions
over items. In our case, we are interested not only in whether
the system properly predicts the ratings of these items, but
rather whether the system properly predicts that the user will
add these items to the queue (for a future use). We calculate
it by correlations between algorithm ranking and user prefer-
ence ordering. If all the correlations have positive values and
all of them higher than 0.70, our alternative hypothesis will be
accepted. In statistics, the correlation coefficient (r) measures
the strength and direction of a linear relationship between two
variables on a scatterplot. The value of r is always between +1
and –1. In Rumsey (2016) a correlation of +0.70 is considered
a strong uphill (positive) linear relationship. That is the reason
that we chose this value.
• Novelty – Some recommender systems produce recommenda-
tions that are highly accurate and have reasonable coverage—
and yet that are useless for practical purposes (Herlocker,
2004). We need new dimensions for analyzing recommender
systems that consider the “nonobviousness” of the recommen-
dation. One such dimension is novelty (Herlocker, 2004). Novel
recommendations are recommendations for items that the user
did not know about (RICCI et al., 2011), i.e. the discovery of new
items. Designing metrics to measure novelty is difficult, because
novelty is a measure of the degree to which the recommenda-
tions are presenting items that are both attractive to users and
surprising to them. In fact, the usual methods for measuring
quality are directly antithetical to novelty. We considered a suc-
cessful score when 1/3 of recommended items are new (higher
than 33%) and in this case, our alternative hypothesis will be
accepted. To analyze this metric, we will use as a reference the
studies of Zhang (2013) and Vargas (2011). Zhang (2013) com-
pared different novelty metrics in two larger datasets of movies
(MoviesLen and Netflix), using different recommender meth-
ods: random, topN popularity recommendation, probability of
items being liked (PL), probability of items being liked and dis-
similarity (LD) and probability of items being liked and dissim-
ilarity and satisfaction (LDS). In this solid study, the novelty
score were extremely small in all the methods (<= 0.0431). The
second article (Vargas 2011) presented a formal framework for
the definition of novelty and diversity metrics that unifies and
generalizes several state of the art metrics. Based on the com-
bination of ground elements, the authors defined a set of meth-
ods to measure novelty and diversity. In their approach, the
metrics related to novelty (EPC and EPD) of a relevance-aware
variant of the content-based algorithm (remembering that our
approach is also a content-based one) were lower than 33%.
Consequently, we used this score to be our threshold.
• Utility – Based on the participants’ feedback about the rec-
ommendations generated by each of the three algorithms. In
this case, we will analyze the rate of the answers “I do not
know the item, but I am interested in knowing it” related to
all unknown items by the user. We consider that the most use-
ful type of recommendation is the one that shows the user an
item that he is interested in and did not know about it. In a
study presented in McNee et al. (2002) several recommenda-
tion algorithms were evaluated regarding user perceived util-
ity of the recommended items. The percentage of “useful’ rec-
ommended items reported in this study ranged from approx-
imately 25% to 60%, depending on the recommendation algo-
rithm used, but both known and unknown recommended items
were classified as useful in case the user perceived them as in-
Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
[m5G;November 2, 2016;11:3]
9
teresting. Another work that investigated the recommendation
of interesting items (items that the user is interested in) was
Duan et al. (2012). In Duan et al. (2012), the number of un-
known items in the top-10 list ranged (roughly) from 20% to
70% considering different algorithms, whereas the percentage
of interesting items (among known and unknown items) ranged
from 20% to 40%. The evaluation results reported in this work
show the percentage of the recommended items which were
unknown to the user, and also the percentage of the recom-
mended items which were interesting to the user, but not the
percentage of items which are both unknown and interesting
at the same time. None of the related works cited in Section
2 showed evaluation experiments to assess the utility of the
recommendations in a way that could be compared to our re-
sults. Considering the results reported in the aforementioned
studies and that the order of items that match the user’s pref-
erence is extremely subjective and varies from person to per-
son, we believe that the utility metric can be improved with a
personalized recommendation. Knowing that this work did not
include any data from the user’s profile in the calculation of the
recommendations, we considered a successful score when 50%
of new recommended items are useful. In this case (>= 0.50)
our alternative hypothesis will be accepted.
In this evaluation, we will consider that the statement “the sys-
tem is able to provide good quality recommendations’ is true when
all the properties were validated.
5.1. Accuracy average
In our desire to reject the null hypothesis and prove the alter-
native hypotheses, the accuracy of the three types of recommen-
dation were computed. Considering that the algorithm for recom-
mending resources of the same type as the input resource is the
same used to recommend songs in Passant (2011), accuracy from
the other two types of recommendation will be compared to the
values previously obtained in the music domain. In case the re-
sults show that accuracy of the proposed algorithms are higher or
equal to this reference, our alternative hypotheses is accepted.
For each movie selected by the user, each recommendation type
provided in general 10 recommended items. In total, the recom-
mendations generated by 970 input movies were evaluated. This
should have generated 970 recommended items for each recom-
mendation type, but as can be seen in Table 1 the line “Total rec-
ommendations” is lower than that. This happens because in spite
of all the care that was used to select and prepare the datasets,
some links were broken and in some rare cases the algorithm
could not find 10 related items to recommend.
In order to compute the accuracy average for each recommen-
dation type, we first computed the accuracy per user, type, and
movie. This means each recommendation feedback of each type,
for each user and each input was individually considered. For ex-
ample, assume that a user has seen and liked movie “ET The Extra-
Terrestrial’ and that this user is going to evaluate recommendations
given for this input movie. The feedback from the user for the first
type of recommendation regarding this input can be seen in Table
2.
The computation of this metric can be done by dividing the
number of correct recommendations by the total of recommen-
dations provided, so it is necessary to define what we consider a
“correct” recommendation. We assumed that a recommendation is
correct when the user classifies it as “I have seen and liked it” or “I
have not seen but would be interested”. For the example in Table
2 the total recommendation given for the input “ET The Extra Ter-
restrial” is ten. From this set, seven are considered correct, as seven
are the number of “X” in the two categories that denote user in-
JID: ESWA
ARTICLE IN PRESS [m5G;November 2, 2016;11:3]

10 J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14

Table 1
Quasi-experiment participants’ feedback regarding the recommendations generated by each of the three algorithms.

Classification / Type Movie Recommendation People Recommendation Resource recommendation

I know the item and find it interesting 398 352 443

I know the item and find it uninteresting 80 44 59
I do not know the item, but I am interested in knowing it 224 321 264
I do not know the item and I have no interest in knowing about it 199 232 173
Total recommendations 960 949 939

Table 2
User feedback for movie recommendations given “ET The Extra-Terrestrial” as input.

Movie / Classification I have seen and liked it I have seen and I have not seen, but I have not seen and
did not like it would be interested I am not interested

Close Encounters of the Third Kind X

Twilight Zone: The Movie X
Jurassic Park X
Schindler’s List x
Hook x
The Lost World: Jurassic Park x
A.I x
Always x
Empire of the Sun x
War of the Worlds x

terest. This way, accuracy for the recommendations in the example Table 3
Number of positive, negative and non-existing correlations between algo-
is 0.7, what means that 7 out of 10 suggestions interested the user.
rithm ranking and user preference.
The same computation was done for the 97 movie inputs,
and the corresponding feedback was obtained for each user, and Correlation / Recommendation type Movies People Resources
grouped by the type of recommendation. After the individual cal- r > 0 (positive) 81 75 86
culation, accuracy average was obtained for each of the three types r < 0 (negative) 16 22 11
of recommendation. r = 0 (none) 0 0 0

– Recommendation of resources of the same type (Movies): 0.69
– Recommendation of people: 0.70
– Recommendation of resources of any type (based on the short
description of the resources): 0.75
As Seelv is a mature application and represents a state of the
art, we decided to use this work to compare with ours. In this case,
we re-implemented its algorithm and used in the same dataset of
our experiment. The accuracy of Seelv and our recommendation of
resources of the same type (Movies) were the same: 0,69, but the
other algorithms of our approaches were better: Recommendation
of people: 0.70, Recommendation of resources of any type (based
on the short description of the resources): 0.75 . Consequently, we
considered our results (based on accuracy) positive.
5.2. Prediction accuracy by ranking
Another investigation was to check if there is a relation among
the items’ rank generated by the algorithm and the real users’ pref-
erence among the items. In the feedback questionnaire participants
were asked to reorder the recommended items according to their
preferences. This data was used to compute the Spearman coef-
ficient, in order to measure the strength and direction (positive
or negative) of the correlation among the two variables provided:
the recommendation rank or the user’s preference ordering. Spear-
man coefficient (rs) values are in the interval [−1, 1], where rs > 0
means positive relation among rank and preferences, rs < 0 means
negative relation, and rs = 0 means no relation among them. Table
3 shows the number of evaluations for each category.
The numbers in Table 3 show that mainly the algorithm ranking
and the real user preferences relation go together (are positively
related). Consequently, our alternative hypothesis is accepted.
5.3. Novelty
Another metric that can be extracted from the feedback is the
discovery of new items. This is relevant as one of the goals of our
approach was to provide novel, serendipitous items to the user. We
propose a simple metric to evaluate this: the discovery index, to be
computed dividing the number of recommended items unknown
to the user, by the total number of provided recommendations.
This is done individually for each recommendation type. For this
metric, an item in considered novel when the feedback received
for it was “I have not seen but would be interested” and “I have
not seen and am not interested”. In the social network recommen-
dation, the categories for feedback were “I do not know but I am
interested in his or her work” and “I do not know and am not in-
terested in his or her work” . In the recommendation of resources
from any type, classification options were: “I do not know the item,
but would be interested” and “I do not know the item and have no
interest”.
The recommendation of people was the one with the higher
discovery index, bringing novel items to the user in 58% of the rec-
ommendations. The other two recommendation types had similar
results: 47% of novel items. It is important to mention that this
index does not take into account if the user liked the item.
5.4. Utility
In this case, we had 53% (movies), 58% (people) and 60% (re-
sources) as new and useful items. Based on the results obtained
for each property, we could consider that our approach is able to
provide good quality recommendations.
6. System interface
A simple Web interface was developed to allow end users to
make use of our recommendation system. The interface language

Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
JID: ESWA
ARTICLE IN PRESS
J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14
Fig. 8. Selection of the input resource.
is Portuguese-Brazilian because this project received support from
the Brazilian government.
During the experiment, all the users evaluated our approach us-
ing this interface. Fifty films were pre-processed, and its recom-
mendations already calculated and stored in an ad-hoc and offline
process. The pre-processing was done because the algorithms are
complex and could be uncomfortable to the user to wait for the
processing time.
The first screen of the interface (Fig. 8) corresponds to the in-
put selection feature. Through it, the user selects the film that will
be the seed of the recommendation process. We opted for the use
of the name of the film obtained by the label property, rather than
the URI for easy recognition of each film, making the use of triplet
data friendlier. In Fig. 8 the film Spider-Man was selected. Still
on the first screen, two disabled checkbox are presented with the
name of the datasets that are used in the recommendation. This is
done only to inform what data sources are being used.
After selecting the movie, the system displays the second
screen, were the three types of recommendations are seen. On the
recommendation of films (Fig. 9), ten films with the highest degree
of similarity are suggested to the user. To facilitate the visualiza-
tion, a bar graph is displayed with the names of the recommended
films. If the user wants to view the properties of the recommended
films, the list above the graph has links to the source datasets.
On the recommendation of people (Fig. 10), the interface dis-
plays a list with the ten persons with strongest connections to the
input resource. A graph is also displayed, representing the connec-
tions between the people resources and the film. In the list of rec-
ommended persons, a prominent number in bold and in paren-
theses is the number of collaborations (links) between the input
resource and that person. This number can also be found as the
weight shown in the graph edges.
The last type of recommendation (Fig. 11) shows a list of re-
sources without type restriction. Therefore, although the input re-
source is a movie, characters are suggested, people and comic
books. In addition to the list, the interface displays, highlighted,
the summary of the film that served as the input to the recom-
mendations.
7. Conclusions and future work
This work tackled the difficulties in consuming LOD. Recom-
mendation techniques to present the user serendipitous informa-
Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
[m5G;November 2, 2016;11:3]
11
tion that should catch his or her interest were proposed and eval-
uated. Also, our work aimed to situate the user in the growing LOD
cloud.
The movie domain was chosen to instantiate an example appli-
cation, given the difficulties to build a general and domain-free ap-
plication. Two datasets were used: LinkedMovieDatabase and DB-
Pedia. Three recommendation types were implemented: one that
recommends items of the same type as the input item, a sec-
ond one that recommends people given an item, and a third one
that recommends items of any type, given an input item. Building
an application that was restricted to the movie domain is justi-
fied by current technical limitations, such as low responsiveness of
the SPARQL endpoints, poor quality of public datasets, no existing
mapping among the ontologies behind the datasets.
The proposed approach brings a new light to LOD consumption
in relevant aspects. It uses more than one dataset (two were used
in our application, but more could be used), it is able to recom-
mend items of different types, and it is able to recommend items
that have few properties in one dataset, as it can use other datasets
and a mapping among them to search for further properties of an
item. Together, these features allow the user to reach the real ben-
efits from LOD. A first-necessary step is the data treatment. The
treatment stage can be applied to any dataset. For this, the dataset
should be available on the Internet; then imported and stored on
a local base; with available metadata. Respecting these conditions,
it is possible to recommend data using any datasets. Consequently,
if we use a good number of descriptors, better the recommenda-
tions will be. We can deal with several datasets, but the bigger the
number of datasets, the bigger is the complexity. During the Data
Interlinking Process, our approach analyzes two datasets per inter-
action. Consequently, we can have a performance problem in this
stage, since all possible pairs of datasets will be considered and
processed. This can be considered a limitation of this work and
performance tests using more than 2 datasets will be made in the
future.
The presented evaluation results are per se a contribution, as
most works found in the literature say very few about what to ex-
pect from recommendation in LOD. Here, we would like to high-
light a limitation of this work. A sample of 35 people could not
be expressive. In our case, we had two options: 1) use a bigger
sample, which could have people with different profiles and con-
sequently be a heterogeneous group or 2) use a small sample, with
a specific profile. We decided by the second option because we be-
JID: ESWA
ARTICLE IN PRESS [m5G;November 2, 2016;11:3]

12 J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14

Fig. 9. Recommendation of movies.

Fig. 10. Recommendation of people.

Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
JID: ESWA
ARTICLE IN PRESS
J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14
Fig. 11. Recommendation of resources.
lieve that a balanced group formed by skilled people could evalu-
ate better this approach than a bigger and heterogeneous group.
For a better analysis, we needed people with a high expertise in
web systems, experience in the recommender systems, highly mo-
tivated with the scenario (to complete all the tasks and compare
the results carefully) and with enough knowledge in LOD (all of
them received training in LOD). It is hard to get a big group with
all these characteristics. We believe that this restriction does not
affect the quality of the work, because most works found in the
literature say very few about what to expect from recommenda-
tion in LOD and very few evaluate their solutions with a group of
users.
This approach can be extended to other domains. In the current
stage of the work, we cannot confirm if this solution will present
good results for other domains. As future work, more experiments
with datasets from different domains should be conducted.
This work was thought as a way to foster the usage of LOD.
Our long term goal is that recommendation can be incorporated
in RDF browsers, in order to exhibit for a searched resource not
only resource properties but also suggestions of other resources
that might interest the user, contextualizing the searched resource.
The reverse path sounds also promising: using LOD to boost rec-
ommendations of existing recommenders. A next step in this di-
rection is to take the user profile in to account to make recom-
mendations, personalizing the output.
References
Berners-Lee, T., Cailliau, R., Groff, J.-F., & Pollermann, B. (1992). World-wide web:
The information universe. Electronic Networking: Research, Applications and Pol-
icy, 1(2), 74–82.
Berners-Lee, T., Fielding, R., & Masinter, L. (1998). Uniform resource identifiers (uri):
Generic syntax.
Bizer, C., Cyganiak, R., & Heath, T. (2007) How to publish linked data on the web.
http://www4.wiwiss.fu-berlin.de/bizer/pub/LinkedDataTutorial/.
Bizer, C., Heath, T., & Berners-Lee, T. (2009). Linked data - the story so far. Interna-
tional Journal on Semantic Web and Information Systems, 5(3), 1–22.
Burke, R. D. (2002). Hybrid recommender systems: Survey and experiments. User
Modelling and User-Adapted Interaction, 12(4), 331–370.
Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037
[m5G;November 2, 2016;11:3]
13
Di Noia, T., Cantador, I., & Ostuni, V. C. (2014). Linked open data-enabled recommender
systems: ESWC 2014 challenge on book recommendation (pp. 129–143). Springer
International Publishing. doi:10.1007/978- 3- 319- 12024- 9_17.
Duan, J., Prasad, S., Huang, J., Tan, P., Chawla, S., Ho, C. K., et al. (2012). Discovering
unknown but interesting items on personal social network. Advances in knowl-
edge discovery and data mining: 16th Pacific-Asia conference, PAKD, Kuala Lumpur,
Malaysia, May 29 – June 1, 2012, proceedings, part II. Springer Berlin Heidelberg.
doi:10.1007/978- 3- 642- 30220- 6_13: 145- 156, 2012.
Frakes, W. B., & Baeza-Yates, R. (Eds.). (1992). Information retrieval: Data structures
and algorithms. Upper Saddle River, USA: Prentice-Hall, Inc..
Franz, T., Koch, J., Dividino, R. Q., & Staab, S. (2010). Lena-tr: Browsing linked open
data along knowledge-aspects. In AAAI spring symposium: Linked data meets ar-
tificial intelligence (pp. 1–10).
Glaser, H., & Millard, I. (2007). Rkb explorer: Application and infrastructure. In Se-
mantic Web Challenge (pp. 111–118).
Glaser, H., Millard, I., & Jaffri, A. (2008). RKBExplorer.com: A knowledge driven in-
frastructure for linked data providers. In Proceedings of the 5th European seman-
tic web conference on the semantic web: Research and applications (pp. 797–801).
Berlin, Heidelberg: Springer-Verlag.
Heath, T. (2008). How will we interact with the web of data? IEEE Internet Comput-
ing, 12(5), 88–91.
Heitmann, B., & Hayes, C. (2010). Using linked data to build open, collaborative rec-
ommender systems. In AAAI spring symposium 2010: Linked data meets artificial
intelligence (pp. 76–81).
Herlocker, JonathanL. (2004). Evaluating collaborative filtering recommender sys-
tems. ACM Transactions on Information Systems (TOIS), 22, 5–53.
Klyne, G., & Carroll, J. J. (2004). Resource description framework (RDF): Con-
cepts and abstract syntax, World WideWeb consortium, recommendation REC-rdf-
concepts-20040210 http://www.w3.org/TR/2004/REC- rdf- concepts- 20040210.
McNee, S. M., Albert, I., Cosley, D., Gopalkrishnan, P., Lam, S. K., Rashid, A. M., et al.
(2002). On the recommending of citations for research papers. In Proceedings
of the 2002 ACM conference on computer supported cooperative work (CSCW
’02) (pp. 116–125). doi:10.1145/587078.587096.
Mirizzi, R., Noia, T. D., Ragone, A., Ostuni, V. C., & Sciascio, E. D. (2012). Movie rec-
ommendation with dbpedia. In IIR (pp. 101–112).
Musto, C., Lops, P., Basile, P., de Gemmis, M., & Semeraro, G. (2016). Semantics-aware
graph-based recommender systems exploiting linked open data. In Proceedings
of the 2016 conference on user modeling adaptation and personalization (UMAP
’16) (pp. 229–237). doi:10.1145/2930238.2930249.
Ostuni, V., Di Noia, T., Di Sciascio, E., & Mirizzi, R. (2013). Top-N recommendations
from implicit feedback leveraging linked open data. In Proceedings of the 7th
ACM conference on Recommender systems (RecSys ’13) (pp. 85–92). doi:10.1145/
2507157.2507172.
Passant, A. (2010). Measuring semantic distance on linking data and using it for re-
sources recommendations. In AAAI spring symposium: Linked data meets artificial
intelligence (pp. 93–98).
JID: ESWA
ARTICLE IN PRESS [m5G;November 2, 2016;11:3]

14 J. Oliveira et al. / Expert Systems With Applications 000 (2016) 1–14

Passant, A. (2011). Seevl - mining music connections to bring context and discovery
to the music you like. Semantic web challenge 2011 winner, ISWC.
Passant, A., & Decker, S. (2010). Hey! ho! let’s go! explanatory music recommenda-
tions with dbrec. Extended semantic web conference. Berlin Heidelberg: Springer.
Piao, G., & Breslin, J. (2016). Measuring semantic distance for linked open data-
enabled recommender systems. In Proceedings of the 31st annual ACM symposium
on applied computing (SAC ’16) (pp. 315–320). doi:10.1145/2851613.2851839.
Prud’hommeaux, E., Seaborne, A., Raimond, Y., Abdallah, S., Sandler, M., & Gias-
son, F. (2007). The music ontology. In ISMIR 2007: 8th international conference
on music information retrieval (pp. 417–422).
Raimond, Y., Abdallah, S., Sandler, M., & Giasson, F. (2007). The Music Ontology.
In Proceedings of the 8th International Conference on Music Information Retrieval
(pp. 417–422). Vienna, Austria.
Ricci, F. (2011). Recommender system handbook (p. 842). New York, NY: Springer-Ver-
lag. ISBN-13:978-0387858197.
Rumsey, D. J. (2016). How to interpret a correlation coefficient r. Statistics For Dum-
mies ISBN: 978-1-119-29352-1.
Salton, G., Wong, A., & Yang, C. S. (1975). A vector space model for automatic in-
dexing. Communications of the ACM, 18(11), 613–620.
Vargas, S., & Castells, P. (2011). Rank and relevance in novelty and diversity metrics
for recommender systems. In Proceedings of the fifth ACM conference on recom-
mender systems.
Volz, J., Bizer, C., Gaedke, M., & Kobilarov, G. (2009). Silk - A link discovery frame-
work for the web of data. In InLDOW (pp. 1–8).
Zhang, L. (2013). The definition of novelty in recommendation system. Journal of
Engineering Science and Technology Review, 6., 141–145.

Please cite this article as: J. Oliveira et al., A recommendation approach for consuming linked open data, Expert Systems With Applica-
tions (2016), http://dx.doi.org/10.1016/j.eswa.2016.10.037