Instructions

In this task you are asked to assess the quality of chatbot responses in conversations between
an AI chatbot with its users. The chatbot can speak with a user about any subject, in order to
help them practice their English speaking skills.
In order for the conversations to be useful, we want to make sure that the bot's responses are
high quality: natural, logical, interesting and safe.

💬 Conversation - 🔍 Response
Read the:

Label the 🔍 Response: 👉👉👉 Bot: Hi how are you? 👈👈👈

Remember: You are only labeling the last bot response. Not the conversation that precedes it.

💬 Conversation: Is the conversation history between the Bot and the current user. Scroll up
and down to read the entire Conversation.

🔍 Response: Is the Bots' final response in the conversation. The Response is what you are
'Labeling'. The Response should be: High-Quality, Logical, and Interesting.

Labels

Is the response interesting?

Interesting is if the response would likely catch someone’s attention or arouse curiosity;
For example: a response to “How do I throw a ball?” could be “You can throw a ball by first
picking it up and then throwing it”, which makes sense and is specific to the question, but is not
very interesting. An alternative deeper and more satisfying answer could be “One way to toss a
ball is to hold it firmly in both hands and then swing your arm down and up again, extending
your elbow and then releasing the ball upwards.” Such an elaborate response should be marked
as interesting.
Also use that rating for anything insightful, unexpected, empathetic, or witty.

Yes: If it is interesting.
Maybe: If in doubt.
No: If the response is monotonous and predictable
Interesting Examples

The response dives deeper about a subject and is knowledgeable.

The response dives deeper about a subject and is oh a slightly controversial topic
The response is unexpected

The response dives deeper about the topic and provides some personal perspective

The response is on an unexpected subject

The response is very empathic and nice, the bot treats the user as a friend.

The response shares knowledge

The response dives deeper about a subject and asks an interesting followup question.
Not Interesting Examples:
The responses above are kind of general and flat: they are not surprising, just something you
would expect as a general response. They don’t have anything unique about them and are a bit
boring.

IMPORTANT TO REMEMBER: You are only labeling the last bot response.
If there are problems in the conversation that lead to the response, it doesn’t mean that the
response has problems as well.

Thank you and good luck!!