In this project, you will be looking at keywords and queries, making judgments about how their 

intents are related, and deciding how close in meaning the keywords are to the queries. 
What are keywords and queries? 
 
You already know what queries are: a user q ​ uery​ is the set of words that a user enters into the 
Google search engine. U ​ ser intent​ is what the user hopes to accomplish by using the Google 
search engine. 
 
Keywords​ are words and phrases that advertisers choose in order to have their advertisements 
to show up alongside particular Google search results. Despite the name, keywords don't have 
to be single words. Most of the keywords you encounter in this project will contain multiple 
words. ​Advertiser intent​ is what the advertiser hopes to accomplish by placing their ad 
alongside a query that matches the keyword they’ve chosen. Advertisers often formulate 
keywords very similarly to how users enter search queries, and expect their advertisements to 
show on queries that have a very similar meaning to their specified keyword. 
 
You should research the keyword and query to form an opinion about what the advertiser and 
user meant by them, and what each may have been looking for. 

Task Description 
In each task, you will be presented with a keyword and a query. You will be asked to decide what 
these terms mean, and how their user and advertiser intents are related to each other. We want 
to know whether advertisements that are relevant to the keyword will also be relevant to the 
query. 
 
Before you begin, research each term by clicking on it to look at the Google search results page 
for that term. If a keyword or query has several plausible meanings, focus on the one where the 
intent makes the most sense within the context of the keyword-query pair as you answer the 
questions in the task. 
 
When you are ready, you will be asked to define h ​ ow the intent of the keyword and the intent of 
the query are related. 
 
If you are not able to answer this question because the keyword or query is unclear, 
ungrammatical, missing or in a foreign language, select the 'unrateable' checkbox next to the 
keyword and query. You will then be asked to identify whether the problem lies with the keyword, 
the query, or both. 

How are these two terms related? 

Your task is to determine how the keyword and query are related. There are five options: 
 Rating  Rating Description 

Same  The keyword and query have the same intent. They refer to the same product, 
intent  service or concept possibly using synonyms, paraphrases, or implied terms. 

Query  The query is a more specific case of the keyword. The query adds a more 
more  specific, non-implied, aspect, product sub-type, location, or brand, but preserves 
specific  the full intent of the keyword. Nothing is more specific in the keyword. 

Keyword  The keyword is a more specific case of the query. Some aspect of the keyword is 
more  not reflected in the query: the query may be missing a non-implied brand, location, 
specific  or product specification that is present in the keyword. Nothing is more specific 
in the query. 

Related  The keyword and the query have related but different intents. For example, they 
but  are about related but different products. The query is at the same time more and 
different  less specific than the keyword. The query refers to an accessory or service for the 
intents  keyword (or the other way around). 

Unrelated  The keyword and the query have completely unrelated intents. For example, they 
intents  represent items from completely different non-overlapping product areas. 
Below you will find additional guidance and specific examples that will help you correctly 
categorize tricky and unclear cases. 

Option 1: Same intent 

Select this option if the keyword and the query have the same intent. For this task, this does not 
mean that the terms need to contain all the same words or be phrased in the same way: ​as long 
at the keyword and query refer to the same product, service, or concept they should be rated 
as having the same intent. 
 
You should pick this option if you think the keyword and query pick out the same things as 
depicted in the picture below. The query, ​fruit​, and the keyword, f​ ruit​, pick out the same set of 
items. 
 

 
Not every pair you rate will be as easy as the pair above. Below we review several common 
types of Query-Keyword pairs that have the same intent. Sometimes you will encounter a mix of 
the cases covered below. In this case, you should still select this option. 

1) Synonyms: the keyword and query express the same intent using different words, for 
example, synonyms or paraphrases. 

Keyword  Query  Explanation 

safety shoes  hazard  The query may use synonymous words to refer to the same. 
shoes 

pediatrician  doctors for  The query is a paraphrase of the keyword. 

children 

diaper bag  men diaper  The keyword paraphrases the query with additional words. 
for a man  bag 

climate  global  The keyword and query are synonyms; they use different 
change  warming  words but target the same concept. 

2) Identical Search Intent: the keyword and query target the same information and express 
essentially the same search intent. 
 
In the table below are some examples of keywords and queries that have very similar search 
intents. These keywords and queries target the same information even though they are not 
exact paraphrases of each other. 
 
Keyword  Query 

male infertility low sperm count  how to get pregnant with low sperm 
count 

romantic music online  love songs online playlist 

global map turkey  turkey's location on the world map 

 
Note: You can usually tell that the keyword and query express the same search intent by 
checking the actual search results for each. However, you should be careful since sometimes 
queries and keywords that have different search intents show similar search results. For 
example, 'flights la ny' shows similar results to 'flights ny la' even though the search intents are 
different. 

3) Implied Terms: the keyword contains an additional word or words that are always or almost 
always implied by other terms in the query (or the other way around). 
 
In the table below you can see keywords or queries that should be rated as having the same 
intent even though the keyword or query has additional words that are not included in the other. 
 
Keyword  Query  Explanation 

toyota  camry  The Toyota Camry is a specific car model. It is fine to drop the 
camry  brand "toyota" here since it is implied by the model name 
"camry". 

houses for  houses for  Since Colville is in WA (Washington State), "wa" is implied by 
rent in  rent in Colville  Colville. 
Colville  wa 

daydream  daydream v
​ r  Since all daydream headsets are vr (virtual reality) headsets, 
headset  headset  the term "vr" is implied in the keyword. 

4) Implied Intent: the keyword has the same implied intent as specified by the query (or the 
other way around). 
 
A keyword or query has an implied intent if a user searching for a concept does not need to 
specify that intent to achieve his or her goal.   
 
Keyword  Query  Explanation 

hydrographic hydrographic Since "hydrographics" is a complex topic, the query, which 

s  s​ how it  asks for an explanation of hydrographics, should be treated 
works  as having same intent as the keyword. 

what is​ health  health  A user searching for "health insurance marketplace" probably 
insurance  insurance  wants to find out details about it, so the keyword which offers 
marketplace  marketplace  a definition should be treated as expressing the same intent. 

vacation  vacation  Since vacation homes are almost always rented, the keyword 
home r​ entals  homes  and query have the same intent. 

play​ games  games online  A user looking for "games online" likely wants to ​play​ games 
online  online, so the keyword and query have the same intent. 

christmas  christmas gift  When a user searches for gifts (in this case 'christmas gifts'), 
gifts  ideas  they are likely to want gift ​ideas​, so this query has the same 
intent as the keyword. 
 5) Commercial Intent: the query expresses the intent to purchase the product or service 
referred to by the keyword (or the other way around). 
 
Keyword  Query 

pro tools 11 download  buy​ pro tools 11 download 

where to purchase​ rx bars  rx bars 

hyundai accent  hyundai accent p

​ rice 

adidas ​shop  adidas 

 
Note: if the keyword or query expresses a specific price point, for example "cheap" or 
"discount", that is not present in the other you should not select this option. F
​ or example the 
keyword or query "handbags" includes expensive handbags like Louis Vuitton handbags, but the 
keyword or query "discount handbags" would exclude expensive handbags. Therefore, you 
should treat "discount handbags" as more specific than "handbags". 

Examples that don't fall into this option (same intent). 

Below you will find some additional tricky examples that seem like they might belong in this 
category, but don't. 
 
Keyword  Query  Correct Rating  Explanation 

acer laptop  used  The query is  Although the addition of the word 'used' implies 
acer  more specific  commercial intent, it also implies that the user does 
laptop  than the  not seek new laptops, which is included in the 
keyword.  keyword. 

train f​ rom  train  The keyword  Although both the keyword and query both refer to 
tahoe to sj  sj to  and query are  trains between Tahoe and sj (San Jose), the keyword 
tahoe  related.  and query have different directions of travel. 

iphone  iphon The keyword  The keyword and query are about different products: 
case  ew​ ith  and query are  the keyword is about iphone cases while the query is 
case  related.  primarily about iphones (that come with cases). 

Option 2: Query more specific 

This option applies if the query adds a more specific aspect (for example, a product sub-type, 
location, or brand) to the keyword, but preserves the main intent of the keyword. 
You should pick this option if you think the query picks out a narrower set of things than the 
keyword as depicted in the picture below. The query, ​apples​, picks out a narrow set of items 
than the keyword, f​ ruit​. 

 
Again, the case depicted above is simpler than many cases you will see. Below we review 
several common types of queries and keyword that should fall in this category. 

1) Product Subtype: the query is a specific product subtype of the keyword. 

Keyword  Query  Explanation 

office  office  Since an office desk is a type of office furniture, the query is more 
furniture  desk  specific than the keyword. 

shoes  women's  "women's boots" belongs to the category "shoes". 

boots 

2) Additional Aspect: the query has an additional non-implied aspect to its intent, but still 
preserves the main intent of the keyword. 

Keyword  Query 

aftermarket auto  cheap​ after market auto 

parts  parts 

wineries in long island  best​ long island vineyards 

lunch recipes  easy​ lunch recipes 

buy printer  color​ printer 

micro sd  64gb​ micro sd card 

 coffee beans  buying coffee beans ​online 

play games  play games f​ or free 

3) Specific Product Intent: the query adds specific intent directly related to the product or 
service referred to by the keyword.  
 
Common additional intents include users looking for reviews, comparisons, coupons, or 
discounts on a specific product or service. 
 
Keyword  Query 

hp  hp notebook r​ eviews 

notebook 

credit cards  credit card 

comparisons 

oil change  oil change c

​ oupon 
 
Note: if the keyword describes a product and the query intent shifts from the product to a 
service or accessory for that product, you should treat them as having related but different 
intents (option 4).​ For example ,"iphone" and "iphone case" (accessory), "tesla" and "tesla 
charging station" (service), and "laptop" and "laptop scratched screen" (problem) are related but 
have different intents.  

4) Specific Product Information: the query requests specific information about the properties 
of the product, service or topic that the keyword refers to. 

Keyword  Query  Explanation 

learn  how long does it  The user query asks for specific information about the 
piano  take​ to learn the  aspect of the service offered by the keyword. 
piano 

hp  hp notebook s
​ creen  The keyword is about a specific product, and the query 
notebook   size  requests information about a specific aspect of that 
product. 
 
Note: if the keyword describes a product or service and the information requested by the query 
shifts the intent from the product or service to a problem with that product or service, you 
should treat them as having related but different intents (option 4). ​For example, an advertiser 
keyword "hp notebook" intends to offer "hp notebooks" while a user query "hp notebook won't 
turn on" requests help with a specific problem with an "hp notebook". The problem the user has 
is related to product, but they are distinct enough that they should be treated as related. 
 
Similarly, if the query shifts the intent from a product to an accessory or service related to the 
product, you should treat them as having related but different intents (option 4). F ​ or example, 
if the keyword is "hp notebook" and the query is "hp notebook power cord" (an accessory) or "hp 
notebook repair" (a related service) , you should mark the keyword and query as having related 
but different intents. 

5) Request Additional Information: the query formulates a specific request or question about 
the problem, topic, or question that the keyword refers to. 
 
If the keyword offers a general problem, topic, or question and the query asks a specific 
question or makes a specific request about the keyword, you should select this option. 
Keyword  Query 

child  do i have enough to gain full child custody 

custody 

puppy fleas  what do you do for fleas on a 2 week old puppy 

 
Note: this is only applicable if the keyword describes a problem, topic, or question. If the 
keyword describes a product or service, this does not apply. 

6) Specific Location: the query adds a location to the keyword or specifies a more specific 
location than the keyword. 

Keyword  Query  Explanation 

doctors  doctors  The query specifies doctors in a location while the keyword is for 
near me  doctors generally.  

texas  dallas  Since Dallas is a city in Texas, the query is asking for a more 
apartments  apartments  specific location than the keyword. 

7) Specific Brand: the query adds a specific brand, business, or merchant to the keyword or 
specifies a more specific brand, business, or merchant than the keyword. 
 
Note: You should treat a query for a brand, business, or merchant to be more specific than the a 
keyword for the main product or service the brand, business, or merchant is primarily associated 
with. However, if a brand or merchant is associated with many types of products (for example, 
Amazon is associated with a wide range of products), you ​should not​ consider it as more 
specific than any of its particular products (for example, keyword 'books'). 
 
 Keyword  Query  Explanation 

printer repair  epson​ printer  The query specifies the brand of printer to fix. 
fix 

general motors  chevrolet  Chevrolet is a specific division of General Motors. 

cars  cars 

auto parts  rockauto  The query is about the merchant 'rockauto' that mainly 
sells autoparts. 

Examples that don't fall into this option (query more specific). 
Below you will find some additional tricky examples that seem like they might belong in this 
category, but don't. 
 
Keyword  Query  Correct  Explanation 
Rating 

set up  set up  The keyword  Although one could complete the keyword with 
chromecas is unrateable.  "chromecast" to get the query, there is no way to 
t  know that the advertiser intended this. We can't 
know what the advertiser meant because the 
keyword is incomplete. 

cast iron  cast iron  The keyword  Although, the query only adds words to the keyword, 
repair  and query are  the query shifts the intent from a product (cast iron) 
related.  to a service (repair) related to the product. 

2015  2015 truck  The keyword  Although the query adds words to the keyword, the 
truck  for sale  and query  additional words only specify a commercial intent, 
have the  which can be assumed in the keyword. 
same intent. 

laundry  target  The keyword  The query "target" picks out a merchant that sells 
basket  and query  many products, including laundry baskets. However, 
have related  since Target does not primarily sell laundry baskets, 
but different  you should chose Related. 
intents. 

Option 3: Keyword more specific 

This option applies if an aspect of the keyword is not reflected in the query. The missing aspect 
may be a brand, a location, a product specification or other additional information. 
You should select this option if the query picks out a broader set of things than the keyword as 
depicted in the picture below. The keyword, a ​ pples​, picks out fewer items than the query, f​ ruit​. 

 
When rating, you should consider the same aspects of the keyword and query that you would 
take into account when deciding whether the query is more specific than the keyword (option 2) 
above. 
 
If the keyword does not preserve the main intent of the query, do not choose this option: instead 
mark the pair as having related but different intents (Option 4) or unrelated intents (Option 5). 

Option 4: Related but different intents 

This is the case for example if the keyword and query are about related but different products, 
or if the query is at the same time more and less specific than the keyword. 
 
One way for a keyword and query to be related is if they pick out overlapping sets of items. For 
example, the query, c ​ ats​, and keyword, h
​ appy animals​, pick out d
​ ifferent but overlapping items 
in the picture below. 

 
Another way a keyword and query can be related is if they pick out entirely different sets of 
items that still have something to do with one another. For instance the query, ​orange juice​, and 
the keyword, o​ range​, should be treated as related because orange juice is m
​ ade from​ oranges.  
 

 
 
Below we review several types of queries and keywords that should fall into this category. 

1) Different Products: the keyword and query specify related but different products, services, 
problems, or topics. 

Keyword  Query  Explanation 

macbook pro  macbook air  Keyword and query are different models of the same brand. 

large format  used wide  The keyword asks for the service "printing" while the query 
printing  format printers  asks for the product "printer". 

sofa recliner  super size  One product is a sofa, the other a chair. 
recliner 

2) Accessory/Service Shift: the keyword shifts the intent expressed in the query, for example, 
the keyword refers to an accessory or a service for the query's product (or the other way 
around). 

Keyword  Query  Explanation 

macbook  macbook  The query is an accessory for the product referred to by the 
pro  pro  keyword, so these have related but different intents. 
keyboard 

toyota car  toyota  The query "toyota" implies that the user is looking for toyota cars. 
parts  The keyword "toyota car parts" picks out accessories for Toyota 
cars. 

desktop pc  desktop pc   The keyword refers to a service related to the product referred to 
repair  by the query, so these have related but have different intents. 
 hairdresser training in  The keyword offers the a hairdressing service in NYC (New York 
s in nyc  hairdressin City). The query is for a related service, namely training to become 
g nyc  a hairdresser in NYC. 

granite  granite  The keyword refers to granite countertops while the query 
countertop countertop specifies that the user is looking for images of granite 
s  s images  countertops. The intents are related but different. 
 
Note: If the query shows that the user's main intent is focused on the product in the keyword 
itself, then the query is more specific than the keyword (option 2). For example, if the user is 
looking for reviews of a product or service, wants to compare the product or service to other 
alternatives, or is looking for coupons, discounts, or delivery of the product, the query is more 
specific than the keyword. 

3) Incompatible Aspects: the keyword and query refer to the same product, service, problem, 
or topic but include incompatible locations, brands, merchants, or other properties. 

Keyword  Query 

italian restaurants sf  italian restaurants LA 

nike shoes  reebok shoes 

refugee crisis  refugee crisis causes 

solutions 

4) More and Less Specific: the keyword is at the same time more and less specific than the 
query. 

Keyword  Query  Comment 

arthritis of  arthritis of  "cervical" is not reflected in the query. "surgery" is not reflected 
the cervical  the spine  in the keyword. 
spine  surgery 

first aid  first aid  "children" is not reflected in the keyword. "certification" is not 
certification  children  reflected in the query. 

used  campers for  The keyword offers used campers that may or may not be in 
campers for  sale in  Virginia, while the query is for campers in Virginia that may or 
sale  virginia  may not be used.  

5) Direction Switching: the keyword and query refer to the same action, but switch directions. 
 
The instructions for Option 1 (same intent) discussed cases in which the keyword and query can 
use different words or different orders of words to express the same intent, but sometimes you 
will encounter keywords and queries that use the same or similar words, but express different 
intents. This is often the case when the order of the words in the keyword or query imply a 
direction. 
Keyword  Query  Explanation 

turkey to  toronto  Both the keyword and query are looking for flights between Toronto 
toronto  turkey  and Turkey, the keyword picks out flights from Turkey to Toronto, but 
flight  flight  the query picks out flights from Toronto to Turkey, so the intents are 
different​. 

rupee to  today's  The keyword and query specify different directions of the exchange 
dollar  dollar rate  rate between Rupees and Dollars. 
rate  in rupees 
today 

translate  translatio The keyword and query specify different directions for the translation 
french to  n german  between French and German. 
german  french 

Option 5: Unrelated intents 

 
Select this option if the keyword and query have completely unrelated intents. 
 
Select this option if the keyword and query represent non-overlapping items that come from 
entirely different non-overlapping domains or product areas. For example, the query, ​orange​, and 
the keyword, o ​ range cat​, should be considered unrelated since they have no items in common, 
and the items they pick out don't really have anything to do with one another.  

 
 
Below are some examples of keywords and queries that are unrelated. 
Keyword  Query 

lord of the rings  wedding rings 

las vegas entertainment  stock option 

 options  quotes 

international year of the potato  the international 

 
Note: If the keyword and query have partially overlapping intent, you should choose the option 
related but different​ instead. For example, the query "1984 symbolism essay" is related to the 
keyword "1984 book", because they are both related to the novel 1984.  

Advice for ambiguous queries and keywords 

1. Start by clicking on the keyword or query and looking at the search results for each. These 
may reveal the correct interpretation of a keyword or query. 
 
2. If a keyword or query is unclear, has many interpretations, expresses only partial concepts, or 
includes non-sensical or ungrammatical text (usually keywords), that you cannot fully interpret 
you should treat the keyword or query as unrateable. Some examples include keywords or 
queries like "blue", "cakes in", and "inthe word". 
 
3. If a keyword or query has several ​reasonable and clear​ meanings that are all equally likely you 
should consider interpretation of the keyword or query that is mostly likely and best matches the 
other. Below are some examples. However, try not to stretch the meaning of the keyword or 
query or look for one that isn't clearly present. 
 
Keyword  Query  Rating 

luxury  jaguar  Since Jaguar is a brand of luxury cars, t​ he query is more specific than 
cars  the keyword. 

big cats  jaguar  Since jaguars are big cats,​ the query is more specific than the keyword.  

contact  contact Since contact lenses are often called 'contacts', t​ he keyword and query 
lenses  s  have the same intent. 

address  contact Since contacts can also refer to a list of people who's contact 
book  s  information you have, t​ he keyword and the query have the same intent​. 
 
4. If a keyword or query has several reasonable interpretations and one is clearly more plausible 
than the others, you should consider only that interpretation. For example, the keyword or query 
[paris] could refer to plaster of paris or Paris, France. Because the interpretation Paris, France is 
much more plausible, you should provide a rating as if Paris, France were the only interpretation.  
 
The Unrateable Flag 
If either the keyword, the query, or both are unclear, nonsensical, missing, or in a foreign 
language, select the unrateable flag. If you select the unrateable flag, you will be asked to 
specify whether the problem is with the keyword, the query, or both. Choose the appropriate 
option and submit the task. 
 
Note: If the keyword or query is in a foreign language, you should select the unrateable flag 
only if the keyword or query is not something you would expect a typical speaker in the locale 
to understand. For example, a query that was a mix of German and English should not be 
marked unrateable in a German language task, since many people who primarily speak 
German also know at least some English.  
 
Note: Porn-related terms are expected for this task, and should be rated normally. 
If either the keyword or the query is so ambiguous or so general that it is difficult to understand 
its intent, it may not be rateable. If it’s not clear what the advertiser wanted when they chose a 
keyword, or what the user was looking for when they entered a query, it may be impossible to 
provide a rating. Below are some examples of keywords or queries that you should rate as 
unclear or nonsensical. 
 
Keyword  Explanation 
or Query 

take  The keyword or query "take" could refer to anything from 'taking pills' to 'take a 
job'; it's too hard to rate this keyword/query because we cannot say what the 
advertiser or user really wanted. 

blue  The keyword/query "blue" is too general for have a clear intent. The user or 
advertiser could be interested in anything from blue dresses to the blue man 
group. 

cakes in  The keyword or query "cakes in" is nonsensical.  

banda  The keyword or query is in Spanish while English ratings were requested.  
larga 
 
Keep in mind that not all keywords or queries that seem nonsensical actually are. It is important 
to research the keyword and query to make sure that you are not misunderstanding its intent. 
 
If you encounter a technical problem with this rating task, use the "Report A Problem" link in the 
lower-right hand corner of the rating page.