28/12/2016 Guide 

Rules: Confusing Scenarios · rainforestapp/tester­chrome­extension Wiki · GitHub

Personal Open source Business Explore Pricing Blog Support This repository Search Sign in Sign up

rainforestapp / tester-chrome-extension Watch 46 Star 32 Fork 56

Code Issues 3 Pull requests 0 Projects 0 Wiki Pulse Graphs

Guide Rules: Confusing Scenarios

kristen edited this page 14 days ago · 28 revisions

Our test guide has some pretty strict rules! Here are a few scenarios you might find yourself in, Pages 10
along with what you should do. Of course this doesn't cover everything you may encounter, but
hopefully these examples provide enough information that you can apply these concepts next time Home

you're confused about what to do. Blocking Policy

Frequently Asked Questions

Examples of when your work will be rejected
Guide Rules & Tester Role

You answered a question with "yes" that should have been answered with "No" Guide Rules: Confusing
You answered a question with "No" that should have been answered with "Yes" Scenarios

You don't follow test instructions. ﴾e.g. you were asked to try to click on a disabled icon, but Idle Policy
you only hovered over the disabled icon﴿
Payment Details

Rejections
Examples of when to answer "Yes"
Training Tasks
Be tolerant of intentionally‐vague tests, as long as it makes sense.These tests are intentionally Troubleshooting & Common
vague and SHOULD BE PASSED because the tester author DOES NOT include quotations Errors Explained
around specific words, so what you see on the test doesn't have to match exactly.
If the instructions ask  Click on the login button  and there is a “Sign In” button, answer
Clone this wiki locally
Yes.
https://github.com/rainforestapp/tester‐chrom
If the instructions ask  Click the button to send a message  and there is a “Submit”
button that looks like it will send a message, answer Yes.
Clone in Desktop
If the instructions ask  Click on the login button  and there is a “Log In!” button, answer
Yes.
If the instructions ask  Click on the login button  and there is a “Click here to log in”
button, answer Yes.
You must match quotes exactly, except in the case of capitalization differences.
If the instructions ask  Click on the "Log In" button , but there is only a “log in” button,
answer Yes.
If the instructions ask  Click on the "Log In" button , but there is only a “LOG IN” button,
answer Yes.
There is one exception to the text‐in‐quote rule. If you see _ in the instructions, it is meant to
be a placeholder for what you will see on the test. Do not report this.
If the instructions ask  Do you see a message that says "_ Users Online"  and you see  37
Users Online  , answer Yes

If the question asks  Are you brought to a new test page with the title "Unnamed Test"?  but

the page says #47211 Unnamed Test, answer Yes.  Unnamed Test  is clearly the title and #47211
is separate.
When asked to  Click on the edit button  and there is a button with a pencil icon, that is OK.
Answer Yes.
If the page looks wrong ﴾e.g. styling is weird﴿, but you can follow the instructions exactly and
the question does not ask about the overall look of the page, answer Yes. If the test allows

https://github.com/rainforestapp/tester­chrome­extension/wiki/Guide­Rules:­Confusing­Scenarios 1/5
28/12/2016 Guide Rules: Confusing Scenarios · rainforestapp/tester­chrome­extension Wiki · GitHub

leaving subjective comments, write a note about the possible issue there.
If there are multiple questions asked and ALL of them are true, answer Yes.

Examples of when to report “No”

The answer to the question is No.
You could not follow the instructions, or the instructions were not clear enough to follow.
There was an unexpected error ﴾one that is not mentioned in the instructions﴿, answer No
If the page does not load, answer No.
If you you experience browser errors ﴾e.g. browser crash﴿, website errors ﴾e.g. red boxes
showing up﴿ or any other unexpected errors ﴾e.g. alerts﴿, answer No.
If your actions do not have expected consequences, answer No.
E.g. if the instructions say  Click the "Continue" button to go the confirmation page , but
clicking the “Continue” button does nothing, answer No.
Report any mismatched text in quotes, including differences in spelling, spaces and
punctuation ﴾disregard capitalization﴿.
If the question asks  Do you see a "Log In" button? , but there is only a “Sign In” button,
answer No.
If the question asks  Do you see a "Send" button? , but there is only a “Submit” button,
answer No.
If the question asks  Do you see a "Log In" button? , but there is only a “login” button, it
answer No.
There is one exception to the text‐in‐quotes rule: if you see  _  inside quotes, that means that
any text is allowed. For example:
If the instructions ask  Do you see a "Results (_)" button?  and there is a “Results ﴾15﴿”
button, answer Yes.
If the instructions are intentionally vague and general, follow them as closely as you can. You
should report any errors that show up and anything that prevents you from completing the
task.
If the page looks wrong and the question asks whether the page looks OK, answer No.
If the question asks  Do you see a blue "Continue" button? , but there is only a green
“Continue” button, answer No.
Do not perform any actions that are not specified.
If the question asks  Do you see the title "hats"? , but to see that you first have to
expand the “clothes” menu, which is not mentioned, answer No.
If the question asks  Did you receive a notification? , but there was no notification shown,
answer No.
If there are multiple questions asked and ANY of them is false, answer No.
If the question asks  Is the home icon is in the right hand corner?  but it's in the left hand
corner, answer No
If the question asks  Do you see a page with 4 images?  but there are only 3 images, answer
No
If the question asks  Do you see the number 20 in the last row of the table?  but you see the
number 20.5 in the last row, answer No.
If the instructions say  Click on the button  but there are 3 different buttons, answer No since
you have no way of knowing which button to click.

Miscellaneous Questions

Are these strict rules useful to the test authors?

Yes! Following these rules will 1﴿ Give them the feedback they need to improve their
product 2﴿ Help them write more accurate test instructions 3﴿ Give them the confidence to
rely on testers to catch subtle mistakes

September Additions!
https://github.com/rainforestapp/tester­chrome­extension/wiki/Guide­Rules:­Confusing­Scenarios 2/5
28/12/2016 Guide Rules: Confusing Scenarios · rainforestapp/tester­chrome­extension Wiki · GitHub

September Additions!

When to say "No"

If the instructions say  Go to the page "www.test.com/help.rl/". Are you redirected to
"/help/"?  answer No. If the instructions asked it in a more specific way it would be acceptable
﴾e.g. Are you redirected to a URL that ends with '/help/'﴿, but it's not specific enough to be
clear.
You should report obvious typos in specific things you are being asked to test, and always
report a typo if it involves quoted instructions. If you see a typo in something you aren’t asked
to test, you should mention it in the test notes if they are enabled for that specific job.
If the instructions ask  Click on the login button  , but there is only a “loginn” button,
answer No.
If the instructions ask  Do you see a message that says something to the effect of "Looks
like there aren't any strength numbers in the system for" followed by the name of the
athlete?  ‐ where the author did not put the whole sentence inside the quote, instead, he
mentioned after the quote that the message is followed by a name. You should report this as
No. If a test author uses quotes, it’s okay to quote only part of the message, but they can’t use
terms like “something like, something to the effect of, etc.” They need to specifically say what
you are looking for if they are using quotes.
If the instruction says Click on the button which says something like "Run tests"  and the
website says "Run test", answer No. The test author is using quotes inappropriately. If they
didn't want you to worry about looking for the exact phrase, they should not have used quotes
around "Run tests". Them saying something like beforehand doesn't change the rule.
If a test author includes an instruction like this, ignore it  Note: Please complete all steps ‐‐
even if a step fails.  You must always report a step as No if it fails.

If the instructions say  Fill out the "name" field with "Jazmin"  but the form has two  name 
fields, answer No. It’s impossible for you to know which  name  field they want you to use.

When to say "Yes"

If the instructions say  Click on the first property. Do the headers say "Example Header 1"
and "Example Header 2"?  and the headers do say that, but a comma separates them, answer
Yes. The headers are both displayed exactly how you see them in the quoted instructions. The
comma is not part of either header, it is just separating the two.
If the instructions say  Select the following state from the dropdown menu: Alabama  but the
dropdown shows  AL  ﴾the abbreviation for Alabama﴿, answer Yes. There were no quotes
around  Alabama  in the instructions, so the abbreviation  AL  is acceptable.
If the instructions ask  Does the page say “Search Engine”?  and you can see  Search Engine  as
a title and other words below or after it, answer Yes. The page does say Search Engine , even
though it also contains other things.
If the instructions ask  Click the "Create" button  but you see a button that has Create with a
symbol next to it ﴾e.g.  +Create ﴿, answer Yes. Most symbols can't be "typed out" so it is
acceptable for the author not to include the symbol in the quotes.
If the instructions asks  Do you see a message that starts with "This is the beginning of
a"?  and you see a message that says  This is the beginning of a very long message , answer
Yes. The instruction asked you to look for specific words within the message, and the
beginning of the message quoted in the instructions is exactly how you see the beginning of
the message displayed on the page. ﴾Note: see "No" section on when to report these types of
partial‐quote questions﴿
Instructions say  Enter the email address "nobody@there.com" and password "123456". Are you
able to log in?  The instructions don’t specifically tell me to log in through. Two subtle things:

https://github.com/rainforestapp/tester­chrome­extension/wiki/Guide­Rules:­Confusing­Scenarios 3/5
28/12/2016 Guide Rules: Confusing Scenarios · rainforestapp/tester­chrome­extension Wiki · GitHub

i. If it asked  are you able to log in?  ﴾like above﴿, this implies you must do something

after typing in the username and password. You have to try logging in order to answer
the question, so you should do whatever is reasonable to log in. In this case you should
answer Yes, unless you try and you’re actually unable to log in.
ii. However, if the question ended with  are you logged in?  ﴾not  are you able to login?  ﴿
you should answer "No" without trying to log in. In this case the question doesn’t ask if
you can try to sign, just if you are signed in, and you shouldn't do something you were
not asked to.

Links/Buttons/Icons
A link is something that takes you to another location whether it be a new page, new tab, new
window, etc. The only difference between a button and a link is that a button is a GUI element ﴾e.g.
is enclosed in a border, unlike a hyperlink.﴿ For testing purposes, treat buttons and links the same
— if you click on it and it takes you to a new location, it is a link. If someone tells you to click on
the button, but it looks like an icon to you, click it. If it takes you somewhere it's a button, and
therefore a link.

So just to emphasize it further, for testing purposes button=link=icon as long as it's clickable and
takes you to a new location. If an instruction tells you to click on the "Go" link and you only see a
"Go" button, do NOT report it unless there is an actual problem that needs to be reported ﴾e.g. the
link/button/icon doesn't take you anywhere, you get an error message, etc.﴿ Hopefully this clears
up further confusion when testing!

Pop-ups
Only report a pop‐up if it is a pop‐up produced by the customer’s website that the test author
didn’t warn you about.
Do NOT report pop‐ups that are related to browsers, networks, hardware etc.

Here are a couple examples of the distinction we’re drawing:

If you log into a customer’s website and you get a pop up asking if you want Chrome to save
your password, do NOT report this.
If you are testing for Best Company Ever and encounter a pop‐up asking, “Do you want to
subscribe to weekly email updates from Best Company Ever?”, but the test author never
mentioned encountering this pop‐up in the instructions, you need to report it.

Other pop up notes:

If a test author instructs you at the beginning of the test to close any pop‐ups you might
encounter and continue testing, this counts as them mentioning any pop‐up you might
encounter. Do NOT report pop‐ups you encounter after a test author says this.
You still need to report error messages you receive. So if you encounter an error message in
pop‐up format, you need to report this.

Bug Count
You must always report a bug, even if you don’t always see your bug count go up. There are two
reasons why you might not see your bug count go up — either people experienced the bug and
didn’t report it ﴾which is bad testing behavior﴿, or people didn’t experience the same bug as you.

Example question: The instructions ask  Do you see a dropdown from 0 ‐ 20?  but the

dropdown is from 1 ‐ 20. I reported this several times, but I never see my bug count go up.
Should I continue to report this?
Answer: Yes, you are correct to report this. Please continue to do so.

https://github.com/rainforestapp/tester­chrome­extension/wiki/Guide­Rules:­Confusing­Scenarios 4/5
28/12/2016 Guide Rules: Confusing Scenarios · rainforestapp/tester­chrome­extension Wiki · GitHub

© 2016 GitHub, Inc. Terms Privacy Security Status Help Contact GitHub API Training Shop Blog About

https://github.com/rainforestapp/tester­chrome­extension/wiki/Guide­Rules:­Confusing­Scenarios 5/5