Entity Content Recommendation Judgment Guidelines

1 Task Overview
The goal of this task is to evaluate if a web page has good quality and interesting content for reading in a recommendation/feed app
focused on a particular entity.

Once you log in to UHRS and have launched the HitApp, you will see a screen similar to this example below:

The HitApp shows two windows side-by-side:

• On the left is a web page. It includes the link of this web page.
• On the right is a pane with an information card for entity (People, Place, Organization, Event, etc.)
You need to judge how good the web page on the left is to recommend to the user for the entity on the right.

2 Judgment Instructions
2.1 Process Steps

1. Click the web page link and read the web page on the new opened window.
2. Read the entity information on the right.
3. Determine how good the web page is to recommend to the user for the entity.
4. Consider these 2 main factors in judging how good web page on the left is to recommend to the user for the
entity.
 a. Relevance:
i. Primary entity: The web page is mainly talking about this entity.
ii. Related entity: The web page is not mainly talking about this entity. But it is related, such as
family members, competing companies, author and their book, singer and their song, CEO and their
company, company and its product.
b. Interesting:
i. In-depth. For example, interviews or reviews.
ii. Recent significant events. For example, popular news or a new released product.
iii. Long term bullet points. If you were summarizing everything important about this
person/company/entity's lifetime or recent few years, this topic is one of the top ones.
5. Choose the first option below which applies:
a. Recommendable The content on this page is relevant to the entity and people would still be interested in reading
it today. Examples: News that is still relevant today, in-depth articles, interviews, informative blog posts, good
quality picture galleries/slide shows, informative videos, travel guides (but not just lists of hotels), historical
accounts. Biographies should be judged as reference if they are more fact-based, and recommendable if they are
more story focused and therefore more interesting.
b. Web Page Not readable: Error or empty page without content.
c. Wrong Entity: Not related to or very loosely related to the entity.
d. Controversial content, such as adult, violence, hate, discrimination or strong political sensitive bias, conspiracy
theories. Example: “15 hot photos of X”.
e. Low quality
i. Not enough informative content: Very little readable/viewable informative content about the entity. A
single paragraph of content is generally not enough to recommend unless it’s very interesting. Examples:
Sign-in page, contact page. Hub/list page with aggregation of information or links. Such as a product
listing page, search results, listing of news articles.
ii. Providing informative content is not primary goal (advertising, download page, coupon, forum, etc.)
These may have some paragraphs of background information, but the focus of the page is not on
presenting content to the user, rather some other goal such as selling or discussion. The page has more
ads than content or otherwise seems very unreputable. Page content is fiction or otherwise not true. Page
content is comedy/parody.
f. Out of date, no longer interesting. The focus of the page is some previous time-sensitive event which is no longer
important. If the article mentions something out of date but gives a lot of background information which is still
interesting today, then it is still recommendable.
i. Announcing upcoming event that already passed: Concert announcement, Black Friday events.
ii. Content no longer interesting: weather report from 2 weeks ago, company earnings report from 5
months ago. Article about an old version of a product. Article from the middle of a series of events when
more recent events are more important: story about wedding preparations when wedding itself already
happened, story is about someone being arrested while their trial has already ended.
g. Reference page (includes Wikipedia-like pages, dictionary, Yelp, local branch info pages). Almost all the content is
encyclopedic or expert-oriented. Articles covering background information are okay, as long as they are not
written in a dry, facts-only reference style. If it looks like it came from Wikipedia, Britannica, some academic
journal, or you need to be an expert to understand it mark it as Reference.
h. Not recommendable for Other reason This page doesn’t fall into the other categories, but someone would be very
unlikely to want to read/view this page to better understand the entity.
6. Click “Submit”.

3 Rating Examples
 Rating Examples
Left Web Page
(content URL)
Recommend https://www.inc.com/bill-snyder/netflix- Reed Hastings
able founder-reed-hastings-make-few-decisions.html
https://www.investopedia.com/articles/professi Tim Cook
onals/081715/how-does-tim-cooks-
management-style-differ-steve-jobs.asp
https://www.investopedia.com/university/warr
en-buffett-biography/warren-buffett-success-
story.asp
https://studytiger.com/essay-the-signalman-by- The Signalman
charles-dickens-essay/
Entity Information Card Reason
Relevant and
interesting.
Wilmot Reed Hastings Jr. is an
American entrepreneur and
philanthropist. He is the co-founder,
Chairman and CEO of Netflix and serves
on the boards of Facebook and a
number of non-profit organizations. A
former member of the California State
Board of Education, Hastings is an
advocate for education reform through
charter schools
Relevant and
Interesting.
Timothy Donald Cook is an American
business executive and industrial
engineer. Cook is the Chief Executive
Officer of Apple Inc., who previously
served as the company's Chief
Operating Officer, under its founder
Steve Jobs.
Warren Buffett Relevant and
interesting.
Warren Edward Buffett is an
American business magnate,
investor, and philanthropist who
serves as the chairman and CEO of
Berkshire Hathaway. He is
considered one of the most
Relevant,
and not
The Signalman is a 1976 BBC disqualified.
television adaptation of "The
Signal-Man", an 1866 short story
by Charles Dickens. The story was
adapted by Andrew Davies as the
BBC's sixth Ghost Story for
Christmas, with Denholm Elliott
starring as the signalman and
Bernard Lloyd as the traveller, an
unnamed character who acts as a
 plot device in place of the short
story's narrator.

Low Quality https://www.tripadvisor.com/Attractions- Feucherolles List page

g2205474-Activities-
Feucherolles is a commune in the
Feucherolles_Yvelines_Ile_de_France.html
Yvelines department in the Île-de-
France in north-central France.
Along with Crespières, Chavenay
and Saint-Nom-la-Bretèche
Feucherolles is twinned with
Rösrath, Germany.

https://finance.yahoo.com/news/buy-microsoft- Microsoft Stale news.

msft-ahead-earnings-185406241.html
Microsoft Corporation is an American
multinational corporation
headquartered in Redmond,
Washington, that develops,
manufactures, licenses, supports and
sells computer software, consumer
electronics and personal computers
Out of Date and services

http://realjewnews.com/?p=156 Talmud Conspiracy

The Talmud is the central text of
Rabbinic Judaism and the primary
source of Jewish religious law and
theology. Until the advent of
modernity, in nearly all Jewish
communities, the Talmud was the
centerpiece of Jewish cultural life and
was foundational to "all Jewish thought
Controversi and aspirations", serving also as "the
al guide for the daily life" of Jews.

http://www.cookingwiththewolfman.com/news Cooking with the Wolfman No real

contact.html content
Cooking With the Wolfman is a cooking
series first produced for the Aboriginal about entity.
Peoples Television Network, an
aboriginal television network in
Canada. Created, executive produced
and hosted by chef David Wolfman and
also executive produced by Larry
Pasemko, this series combines
Not enough traditional North American Native
content cuisine with modern dishes.
 https://explorable.com/chameleon-effect Chameleon
Chameleon was a record label formed Not the same
by producer, music entrepreneur and Chameleon.
former Capitol Records A&R executive
Stephen Powers, in association with
Bob Marin, Managing Director of
alternative rock importer Sounds True,
Wrong and Richard Foos, co-founder of Rhino
entity Records.

http://natmedtalk.com/showthread.php? Padmanabhaswamy Temple Forum page

t=25980
Padmanabhaswamy Temple is located
in Thiruvananthapuram, Kerala, India.
The temple is built in an intricate
fusion of the indigenous Kerala style
and the Tamil style of architecture
associated with the temples located in
the neighbouring state of Tamil Nadu,
featuring high walls, and a 16th-
Low Quality century Gopuram.

https://earnthenecklace.com/lindsay-lohan-net-worth Lindsay Lohan Article is

focused on
Lindsay Dee Lohan is an American
actress, businesswoman, fashion just
designer and singer. Born and raised in presenting
New York, Lohan was signed to Ford facts, not
Models as a child. Having appeared as telling an
a regular on Another World at age 10, interesting
her breakthrough came in Disney story.
Pictures' The Parent Trap. The film's
success led to appearances in a string
of their televised films and the
Reference theatrical Freaky Friday.

https://en.wikipedia.org/wiki/HBO HBO Article is

focused on
Home Box Office is an American
premium cable and satellite television just
network of Home Box Office, Inc.. presenting
Programming featured on the network facts, not
consists primarily of theatrically telling an
released motion pictures and original interesting
television series, along with made-for- story.
cable movies and documentaries,
boxing matches, and occasional
Reference comedy and concert specials.
4 FAQ
a) Q: The page sometimes fails to load or content is blocked with a popup/overlay.

IF the left web page fails to load or content is blocked with popup/overlay, please open the page via content link given and
continue judging.

b) Q: I am using Internet Explorer 11 and I see the following message pop up when I open the HitApp, what should I do?

A: Click “Options for this site” button, and then click “Always allow” in the drop-down menu.

If see the “Are you sure you want to leave this page” dialog afterwards, click “Leave this page”, and the page will be
refreshed.

c) Q: I am using chrome and I see the alert icon in the address bar, what should I do?

A: Click the red alert button, and then click “Load unsafe script” in the message pop up.

--You are ready judge now---