ESCI Search Query:

Introduction & Training

2021

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared 1
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Table of Contents

• General Information
• Judgements
• Defects
• Next Steps

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Important Terminology

• Query = input in Amazon Search (e.g., iphone 11 accessories)

• ASIN = Amazon Standard Identification Number (i.e., product identifier)

Product ID B08L5M9BTJ

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
3
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: General Information

• The Marketplace Category needs to be

considered when evaluating a query.

• If a marketplace category is included in the

search, it means the customer selected that
category before entering the query. This
information can help clarify a broad query.

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
4
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: General Information

As a Rater, you will be presented with a

customer’s query and a product to be judged for
relevance to that query. Before making The judgements as highlighted by the
judgments, you must understand the query in ESCI Guidelines are as follows:
order to anticipate what kind of product the
customer is looking for. In instances you are • Exact
unsure, you may use Google to clarify a query.
• Substitute
• Complement
• Irrelevant
Evaluate products according to the ESCI • Unjudgeable
Guidelines

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
5
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Judgements

EXACT
This product fully satisfies the query. A broad or ambiguous query may be satisfied by a diverse set of products, so long as it
is reasonable to expect that a customer who entered that query might buy that product. A very specific query will only be
satisfied if the product matches all characteristics included in the query.

Query: adidas t-shirt green short sleeve

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
6
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Judgements

SUBSTITUTE
This product differs from the one intended, but it is functionally equivalent. This will be the same type of product, but may
differ by brand, model, color, title, or other detail. If the requested product is included in a bundle with other associated
products, it is judged as Substitute, so long as it is reasonable that the customer may want to buy both together.

Query: google home

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
7
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Judgements

COMPLEMENT
This product is not the one intended, but it could be useful to the customer if they did buy the intended product. This could
be an accessory, a component, replacement parts, or even another item designed to be used with the product.

Query: adidas shoes

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
8
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Judgements

COMPLEMENT
This product is not the one intended, but it could be useful to the customer if they did buy the intended product. This could
be an accessory, a component, replacement parts, or even another item designed to be used with the product.

Query: 24 inc tv

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
9
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Judgements

IRRELEVANT
This product does not satisfy a reasonable understanding of the customer’s intent. It could be entirely unrelated, or it could
differ by one crucial feature that would make it unsuitable for the customer.

Query: samsung s8 case

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
10
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Judgements

IRRELEVANT
This product does not satisfy a reasonable understanding of the customer’s intent. It could be entirely unrelated, or it could
differ by one crucial feature that would make it unsuitable for the customer.

Query: cat litter scoop

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
11
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Judgements

UNJUDGEABLE
This judgment is only to be used if the product cannot be judged, either because the query cannot be understood, or
because the product is lacking enough information to determine if it matches the query. An Unjudgeable query is one that is
complete gibberish, is seeking help, is attempting to navigate to a website, is in a non-market language, or is a string of
numbers that does not correspond to an ISBN or model number in Google searches.

Query: samsung s8 case

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
12
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Defect Table

• A defect is an aspect or feature of a product that detracts from its helpfulness or renders it irrelevant in relation to
the query and/or marketplace category.
Defect Judgement

Bundled product Substitute

Accessory/related product Complementary

Wrong color Substitute

Foreign Language Unjudgeable

Wrong size (one size up/down) Substitute

Wrong size (more than one size up/down) Irrelevant

Correct product, wrong brand (brand specified query) Substitute

Wrong product, correct brand (brand specified query) Irrelevant

Wrong gender/age Irrelevant

• Please be aware that this information may not fit perfectly to every scenario, so please base your overall judgement
on the query, marketplace category and the product itself.
This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
13
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Tool

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
14
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Tool

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
15
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Search Query Annotation: Next Steps

• Familiarize yourself with the Guidelines. Queries and products should be considered
carefully while always keeping the Guidelines and ESCI approach in mind. Also, remember
to consider context and user intent!

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.
 Thank You!

This material contains information that is proprietary and confidential to TELUS International. It cannot be shared
17
with third parties without TELUS International written consent. DO NOT COPY. DO NOT DISTRIBUTE.