Search Engine Side by Side – Guidelines

Who is this task for?

• Have you ever used a search engine like Google in your life?
• Do you consider yourself smart, attentive, and pay good attention to detail?
• Are you curious about different topics and would like to learn more about things while you earn money?

If the answer to all the questions above is YES, then this task is for You! Read these guidelines & watch our
short video which can be accessed inside the HITAPP to become a master of SEARCH ENGINE SIDE BY SIDE!

How to Perform the Task

1. Understand the query, the intent and expected results.

• You will be shown a Query (the search terms someone types into Google or another search engine)
• The location from where the query was issued. You will be able to click on a map in the tool to see more.
• A helpful Query intent pop-up which tells you what the possible intent of the search is (read this to save time!).
• A helpful Expected Results description which tells you what type of results are best for this search.
• Use your intuition in addition to Query intent to understand what the user might be searching for.
• Use your common sense in addition to Expected results to visualize what best answers for the query would be.
• Spend a few seconds reading and understanding what these mean to provide a judgement successfully.

Picture of the HITAPP showing where to find all this information.

Pro Tips

• If the query intent or expected results look wrong to you (based on your knowledge), mark them as bad!
• By marking the bad intents and expected results, you are helping us improve our quality!
• Please continue to Judge the HIT and provide comments as to what is the best intent and results.
• Please also provide a comment in case you thought the intent/expected results are bad (for a bonus)
2. Evaluate both Search Engines on both sides and explore the features…
• Check for good and bad features from the top & scroll to the end if needed for both sides
• Place more emphasis on the side that answers the search towards the top clearly and concisely.
• Determine the features on either side that best answer the query and those that fail to help answer them.
• Moving your mouse over each feature gives you hints of how to evaluate it.
• Remember, a more relevant (correct) not rich-feature is better than a rich non-relevant feature

Pro Tips:

• Click on the – and + buttons to adjust your zoom level so you can see both screens side by side fully.
• If you want to see one side more than the other, you can drag the center resize bar
• Please note that Relevance and correctness is more important than beautiful and/or more content
• A more relevant & less rich search engine is better than a less relevant richer one!

3. Give stars, likes & dislikes on features based on how useful or bad they are (on both sides)
• Select the star button on that feature if it perfectly answers the query. Remember, it is only for the best feature.
• Select the like button on other features which satisfy the intent well.
• Select the dislike button on features that are completely wrong, useless in answering the query, or are defective.
• Also dislike features that add no additional value to the results but actively distract you from getting the answer.
• As you move the mouse over a block on the page, you can see hints on how to rate each block. Use them!

Provide Your Final Rating

Once you have evaluated both sides and found the better Search Engine and identified the features that helped it win, it’s
time to give your final rating! To do this, move the slider at the bottom to whichever side you feel is stronger. The more
you move it towards left or right, the stronger your preference is for that side. You have three range of choices to choose.
Winning side Slightly Better

While both sides do a good job on answering the query, the winning side has a better page layout (or) answered the query
more thoroughly towards the top of the page. In addition, the winning side does not distract you with features that add no
additional value.

Winning side Better

The winning side answers the query with content or features that would completely satisfy the user intent at the top of the
page. The losing side shows much weaker or irrelevant features that maybe incomplete, defective, or less trustworthy
compared to the winning side.

Winning side Big Win

The winning side is full of useful, relevant, and trustworthy content while the losing side has much less useful, relevant,
and trustworthy content. The losing side fails to answer the query or gets the intent completely wrong.

Pro Tips:

• Each of these ratings have a range and it provides you flexibility to show more / less weight to your final rating.
• E.g. if you think a side is slightly better but it is close to being better, you can slide it as close as you want!
• REMEMBER: Relevance is more important than richness. A side which answers things correctly is better than a
side which does not answer the query regardless of how much richness is present there.
Comment and Submit
Once you have provided your rating, take a few moments to check if everything is correct and provide a comment that’s
well thought out and explains why you chose your rating. You can provide a comment by clicking on the blue link that
comes up saying “justify your answer”. Depending on the side you choose, the link will appear on that side.

On clicking this blue link, A pop-up window will appear and let you write comments on why you chose the
rating. Writing well thought out comments can earn you a bonus on this task. This is optional for you.

After completing this (optional) step, go ahead and hit submit your judgment by clicking on the submit button!

Congratulations! You have completed your first Search Engine Side by Side HIT! Please provide us your
feedback and we hope to have you judge our task and earn money while learning about new things that you
never knew about!

Thank you for working on our HITAPP! Happy Judging!

NOTE: SPAMMERS will be banned forever from UHRS!