QC Result Quality v1.

0 Guidelines

Terms to Understand

Query – a question, especially one addressed to a search engine.

Keyword – a word or concept of significance.

Label – the answer selected for each question within the Hitapp.

Result Page – the webpage given in the hitapp to be evaluated.

Quality – the standard of something as measured against others of a similar kind.

Credibility – the quality of being trusted and believed in.

Source – the person or group from which the information is obtained.

Authority – possessing the knowledge to speak about a topic.

Accountability – the condition of being responsible for something.

Reputation – the general opinion of a person or company.

Purpose – the reason for which a website was created.

Bias – prejudice in favor of or against something.

Integrity – the quality of being honest and having strong moral principles.

Tone – the general character or attitude of the content, for example, serious, sarcastic, or humorous.

SPAM site – a site with irrelevant content intended to attract users, clickbait.

Moderation – the process of reviewing information to manage the quality of content.

Harm – causing or likely to cause physical or psychological distress or damage to a person.

Promote – further the progress of, encourage.

Aggregator – a website that collects related items of content and displays them as links.

Content Farm – a website that contains large quantities of content, typically low quality, from other
sites, generated solely to ensure that it appears high in search results.

Misinformation/Disinformation – false or inaccurate information deliberately intended to deceive.

UGC/User Generated Content – content created and posted by a user, typically found on social media
or forum sites.

Task Overview

When a user issues a query on a search engine, they have an expectation that the results returned to
them will be of high quality and credibility. Your task will be to evaluate websites based on 3 factors,
Integrity, Page Quality, and Authority, that will be used to determine the overall quality and credibility of
the page.

Steps for Judging

Understand the Result Page – Read through the main content of the Result Page and determine what
the purpose of the page is. Determine the source of the content and take notice of how the page is
designed. Look for ads and spammy content. Consider if the source of the content has any authority on
the subject.

Understanding the Questions:

1. Does this page contain harmful or offensive content such as malware, scams or
content promoting hate, discrimination, terrorism, disinformation, or violence?
For articles discussing such difficult topics in an unbiased, informative way should be
answered No.

a. Harmful pages would include pages which provide information that if acted
upon, could be dangerous to the user as well as content like malware that
could directly harm the user.
2. How would you rate the usability and style of the result page?
Consider the colors and layout of the page as well as ad placement and overall readability.

a. Grammar, spelling, and the layout of the page should be examined as well as
the amount and placement of ads. Images and videos should be viewable.

3. How authoritative is the source in providing the content of the page?

Content farms and aggregator sites should Not be labeled Best Source or High Authority.

a. The Best Source for information would be where the information originated
from, the person who first said it. Not all content has a best source.
b. High Authority sites would include news sites, editorials, etc.
c. Poor Authority sites would be aggregator sites or content farms, SPAM sites,
clickbait, or sites hosting pirated materials.
d. Unknown Authority would be Reddit or other social media posts where the
source’s knowledge of the subject is questionable, or any site that you
cannot clearly label as High or Poor.

Understanding the Labels:

 1. Does this page contain harmful or offensive content such as malware, scams or
content promoting hate, discrimination, terrorism, disinformation, or violence?
For articles discussing such difficult topics in an unbiased, informative way should be
answered No.

a. Yes – The purpose of the page is to promote harmful content.

b. No – The page does not promote harmful content.

2. How would you rate the usability and style of the result page?
Consider the colors and layout of the page as well as ad placement and overall readability.

a. High – High rated pages would be easy to read, fully functional, and with few
ads that do not interfere with reading the main content.
b. Poor – Poor pages may contain things like spelling and grammar errors,
obstructive or annoying ads, and content of little actual use or minimal
content on a subject.
c. Neutral – Neutral should be used only if the result page cannot clearly be
labeled High or Poor.

3. How authoritative is the source in providing the content of the page?

Content farms and aggregator sites should Not be labeled Best Source or High Authority.

a. Best Source – should only be used when there is no other source with equal
or better authority.
b. High Authority – should be used when the source is highly trusted and
respected.
c. Poor Authority – should be used when the source has low to no authority on
the subject or the page is SPAM, Clickbait or contains pirated information.
d. Unknown Authority – should be used when the authority cannot be
determined, and the result page is clearly not the Best Source, High Authority
or Poor Authority.
Labeling the Hitapp – select the label that best answer the hitapp questions.

Questions to consider:

Is the content present in a clear, factual way, or is it biased toward a particular opinion?

Can you easily read the main content of the page?

Do ads bock any of the content?(see chart below)

Who is the source of the information? The website? A paid contributor? A user? Does the
website allow anyone to add content without any form of moderation?
 Does the source of the information have authority on the subject? Are they educated on the
subject? Have real world experience with it? Are they a journalist or writer who has done
research on the subject?

Does the information come directly from a reliable source, or is it being provided by a someone
else?

Is the source well know with a good reputation?

Ad Placement on Webpage

Main
Main Main Content
Content Content Main
Content

Left of Content Right of Content Before Content After Content

Ads in any of these positions, or any combinations of these positions are acceptable as
they do not block the user from seeing the main content of the page.

Main Main Main Main

Content Content Content Content

Covering Content Multiple Ads Floating over Auto-playing

Within Content Content video ads

Ads in any of these positions, or any combinations of these positions are obstructive and
not acceptable as they interfere with the user’s ability to see the main content of the page.

Main Ad
Content Content
 Examples

Question 1 Does this page contain harmful or offensive content such as malware, scams or content
promoting hate, discrimination, terrorism, disinformation, or violence?
Articles discussing such difficult topics in an unbiased, informative way should be answered No.

Result Page Label Explanation

https://www.disney.com/ No This site is for the Disney company and presents child friendly
material.
https://qanon.pub/ Yes A far-right forum site promoting belief in conspiracy theories
and fringe beliefs.
Article from Humansarefree.com Yes The information is presented as factual; however no actual
studies are cited and the website is known to promote
conspiracy theories.
Article on White Supremacy in Politics No The article discusses how White supremacy organization have
recently become more involved in politics but does not
promote white supremacy.

Question 2 How would you rate the usability and style of the result page?
Consider the colors and the layout of the page as well as ad placement and overall readability.

Result Page Label Explanation

https://www.youtube.com/ High Content is easy to read and organized to make it easy
to find information. There are no ads blocking content.

https://www.cnn.com/2021/08/30/media/jon- Poor Website has ads that popup covering the top of the
stewart-apple-show-date/index.html screen and along the right side. Video ads auto play.
Ads break up the content of the page.

http://www.greatdreams.com/ Poor Information is difficult to read due to the layout and

colors. Page is poorly designed.

https://www.cnet.com/ Neutral Page has auto playing video ads and ads above and to
the right of content. Ads are also inserted between
articles. Ads don’t obstruct content, but the
abundance of ads makes this questionable.
Question 3 How authoritative is the source in providing the content of the page?
Content farms and aggregator sites should Not be labeled Best Source or High Authority.

Result Page Label Explanation

Article from NASA.gov Best Source The article about how astronauts write in space
is written by NASA based on work done there.
They are have first hand knowledge of the
information and are the Best Source for this
information.
Nike.com Best Source Nike’s official website is the best source for
their own products.

Article from Huffington Post High The Huffington Post is a well-respected news
Authority source with a good reputation. The writer is a
staff reporter for the news outlet. This should
be considered High Authority as other news
outlets would be considered an equal source
for the information.

https://www.kmweiland.com/ Best Source This is the author’s official website, and thus is
the best source for information on her and her
books.

Unknown
https://www.youtube.com/watch?v=tZnHysAmYaY Authority The user uploading this video is not officially
affiliated with the company. This is a product
review.

https://youthduo.com/home Poor This website makes questionable claims about

Authority anti-aging products and should be considered
to be a poor authority.
https://cheesecurdinparadise.com/ Unknown With no real information on the writer of the
Authority blog it would be difficult to say High Authority.
As she has written a book about food, we
cannot say Poor Authority. Unknown is the best
option.

Post on Reddit.com Unknown While Reddit is a well-known forum, the

Authority responses to this medical question are from
users that cannot be verified as having any real
medical or first aid knowledge. As there is no
way to clearly say this is High or Poor, Unknown
is the best option.

Can’t judge the hit? – some hits can’t be judged for reasons that are outside of your control. If that
occurs, please select the appropriate reason and submit.

• In Wrong Language – if the query or result page is in a different language from the one
marked in the title (Search Intent [EN]). English queries with English Result Pages should
always be judged normally.
• Does Not Load – If the page doesn’t load.
• Adult Content – If the page has adult content.
• Requires Login – If you cannot view the content of the page without logging in to the
site.
• Harmful Content – If you suspect the page might download something harmful to your
computer (spyware or malware).