Io Blue Floor Hazards Reviewer

23/12/2020 https://docs.google.
com/document/d/e/2PACX-1vQL4yZWPBHH9pmbCDdSGyW_wl35Ldrf6ksptOL4_09UOplMWo4KJVkPJ-2QHv55A…
Io BLUE Floor Hazards Spec Document

Bbox, polygon | IN PROGRESS
Contents
● Labels & Attributes Table
● Annotation Rules
● Critical vs. Non-Critical Errors
● Edge Cases
● Labels & Attributes Description/Examples
Zoom around each task carefully to nd all the objects listed in the label/attribute table! Select between box
and polygon in the top right corner.
Key to Project
Label/Annotate
1) Objects on the floor that would interfere with the Robot
2) Objects not on the floor that, if they were on the floor, would interfere with the Robot.
Context Images
During Tasking, you will see a black box on the bottom of your screen. These images are Context Images and are meant
to assist you in understanding the task.
Clicking on these images will give you additional views of the task.
**To return to tasking, hit the "esc" key!**
Labels & Attributes Table
Task Label Nested Label Attributes

type/Geometry
Bbox Floor_Hazard messy_floor_hazards Messy hazard type
[animal_poop,
animal_vomit,
crumbs_or_cereal]
Floor hazard location

[on_the_floor,
NOT_on_the_floor]
Polygon Floor_Hazard stringy_floor_hazards Stringy hazard type
[backpack_or_handbag,
belt_or_strap,
cable,
blanket_or_comforter,
carpet_tassels,
clothes,
towel,
headphones_or_earbuds,
https://docs.google.com/document/d/e/2PACX-1vQL4yZWPBHH9pmbCDdSGyW_wl35Ldrf6ksptOL4_09UOplMWo4KJVkPJ-2QHv55APLSuc76cnBT… 1/19
23/12/2020 https://docs.google.com/document/d/e/2PACX-1vQL4yZWPBHH9pmbCDdSGyW_wl35Ldrf6ksptOL4_09UOplMWo4KJVkPJ-2QHv55A…
shoelace,
yarn_or_string,
jewelry]

[on_the_floor,
NOT_on_the_floor]
*note: all polygons are limited to 10 vertices.

See Annotation Rules below
Polygon Floor_Hazard threshold Just doors
Bbox Floor_Hazard miscellaneous_floor_hazards Miscellaneous hazard type

[hair_tie_or_clip,
hanger,
paper_or_magazine,
pens_markers_pencils_crayons,
plastic_or_paper_bag,
rubber_band,
shoes,
shredded_paper_or_confetti,
small_toys_or_legos,
socks,
toilet_brush_or_plunger]

[on_the_floor,
NOT_on_the_floor]
Bbox Floor_Hazard
other_floor_hazard
Bbox Floor_Hazard liquid_spill_or_puddle Floor hazard location
[on_the_floor,
NOT_on_the_floor]
Annotation Rules - Bbox
Rule Description Example
Annotation Ordering Draw polygon rst, then bounding

box
Grouped Objects Group small objects if their pixels Individual small objects and pixels are separated, box
are touching. (small: shoe or separately:
smaller. Example: books, legos,
clothing in a pile, shoes in a pile.)
For larger objects, or objects that

aren’t touching, use separate
annotations.
Small objects and pixels are touching, box together:
Visibility Visibility is how much an object is

visible.
If an object is occluded, only box

what is visible.
*objects in this image are random examples - only label

the objects in the label/attribute table
Minimum Pixel Size Only label objects that are 5x5

pixels or bigger.
5 x 5 = annotate
5 x 17 = annotate
3 x 17 = do not annotate
30 x 2 = do not annotate
Label Con dence Annotate and label if you are

75%+ sure what the object is. If
you have a good guess of what
the object is, label it that way!

Glass/Reflections Annotate objects that you see

clearly through glass or plastic
(you need to see the item clearly
and have 75% con dence that
you know what it is).
Annotate objects that you see

perfectly reflected in a mirror.
Do NOT annotate bad reflections,

like a window reflecting on a
black TV screen.



Blurry Objects Clip the blur to what you think the
real size of the object is.

Bounding Box Label all visible objects. Make

Tightness sure the boxes t very tightly.
There should be no more than 3px

of space between box edge and
object.
Truncation If an object is truncated (cut off at

the edge of the image), keep the
box tight around the visible part
of the object.
The box should end at exactly the

edge of the image for truncated
objects.
Annotation Rules - Polygon
Rule Description Example
Annotation Ordering Draw polygon rst, then bounding

box
Holes Do NOT cut out holes from objects.

Simply cut around the outside edge
of the object.
10 Vertex Limit Any polygon created will be hard-

capped to 10 vertices.
Once 10 vertices are selected, the

polygon will automatically close.
Users will then have to adjust

annotations to best approximate the
object’s shape.
Divided Objects If an object is divided by something

else in front of it, use one polygon
and go on top of the occluded part.
*objects in this image are random

examples - only label the objects in
the label/attribute table
All other rules same as bbox
Edge Cases
Edge Case How to Handle It Example
When to annotate All floor hazards are small objects

floor hazards that might tangle/catch a small
robotic device. Hazards do NOT
include large objects like furniture,
carpets, pets/people - never
annotate these as hazards.
For each object that has a

speci c floor hazard label (for
example, shoelaces, toys,
animal_poop) annotate the object
anywhere you see it. It can be on
the floor or elsewhere, like on a
table, bed, shelf.
Exception: when you use the

other_floor_hazard label, only
annotate objects on the floor. Do
NOT use the other_floor_hazard
label on objects that are on tables,
beds, walls, or anywhere else.
Apply the other_floor_hazard label

to
1. Objects that are SMALL
(at most two inches high,
or about as high as a
shoe).
1. Small hazard objects on
the floor that don’t t into
other classes
2. Small hazard objects on
the floor where you can’t
tell what the object is
3. Includes small
cardboard boxes, cereal
boxes, keys, and folded
cardboard boxes
How to annotate a Annotate the cable part as cable.

power strip
Annotate the outlet part as other
floor hazard.
When to mark floor If any part of the object is touching

hazard as “on the the ground, mark the object as
ground” being on the ground.
Labels & Attributes Description/Examples
Labels/Attributes Description Example
messy_floor_hazards Animal_poop
Animal_vomit
Crumbs
liquid_spill_or_puddle Liquid on the floor. NOT animal

poop or vomit (those have their
own labels)
stringy_floor_hazards Backpack_or_handbag (include

the straps)
Belt (do NOT include clothing

that is being worn by a person)
Headphones_or_earbuds
(include the earpiece and the
cord)
Cables (any type of cable, such

as electronic, power, speaker
cable, ethernet) NOT
headphones - they have their
own
“Headphones_or_earbuds”
label.
Clothes (do NOT include

clothing that is being worn by a
person) Group together if in
piles/hanging together closely.
Jewelry (do not include if worn

by a person)
https://docs.google.com/document/d/e/2PACX-1vQL4yZWPBHH9pmbCDdSGyW_wl35Ldrf6ksptOL4_09UOplMWo4KJVkPJ-2QHv55APLSuc76cnB… 10/19
Carpet_tassels (decorative
stringy ends on some carpets)
Blanket_or_comforter
Shoelace (only if coming away

from the shoe or without a
shoe)
Yarn_or_string (any yarn or

string, including cords from
blinds)
Threshold Change in floor height, usually
between 2 rooms
miscellaneous_floor_hazard Plastic_bag (be careful if you

see this next to a carpet - check
that the object isn’t actually a
carpet tassel)
Paper or magazines: includes
magazines, letters, napkins,
tissues (NOT tissue box),
crumpled paper. Do NOT
include books, small scraps of
paper (they have their own
label), or toilet paper.
Shredded_paper_or_confetti
(small pieces of paper)
Toilet_brush_or_plunger (often
near a toilet)
Small_toys
Pens_markers_pencils_crayons
Rubberband
Shoes (do NOT include clothing

that is being worn by a
person). Do NOT include
shoelaces that are coming off
the shoe - there is a special
shoelace label for that. Group
piles of shoes, but individually
box individual shoes (for
example shoes that are
organized on a rack).
Hair tie and hair clips
Socks (not socks on a person)
Towel (kitchen or bath towel)
other_floor_hazards Keys on the ground
Cardboard on the ground
Common Errors
Common Error Image
1. You should ONLY use the

label “Other_floor_hazard” IF
the object is on the floor, and
also only if the object is small
(at most two inches high, or
the height of a shoe).
2. These are carpet tassels.

NOT plastic bags.
Change log
Instruction Deadlines
● Labels & Attributes Table -- needs to be locked by end of Quali cation Run
● Annotation Rules -- needs to be locked by end of Quali cation Run
● Critical vs. Non-Critical Errors - needs to be locked by end of Quali cation Run
● Edge Cases - iterative

Io Blue Floor Hazards Reviewer

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Io Blue Floor Hazards Reviewer

Uploaded by

Copyright:

Available Formats

23/12/2020 https://docs.google.

Io BLUE Floor Hazards Spec Document

**To return to tasking, hit the "esc" key!**

Labels & Attributes Table

Task Label Nested Label Attributes

Floor hazard location

Floor hazard location

*note: all polygons are limited to 10 vertices.

Bbox Floor_Hazard miscellaneous_floor_hazards Miscellaneous hazard type

Floor hazard location

Annotation Rules - Bbox

Rule Description Example

Annotation Ordering Draw polygon rst, then bounding

For larger objects, or objects that

Small objects and pixels are touching, box together:

Visibility Visibility is how much an object is

If an object is occluded, only box

*objects in this image are random examples - only label

Minimum Pixel Size Only label objects that are 5x5

Label Con dence Annotate and label if you are

*objects in this image are random examples - only label

Glass/Reflections Annotate objects that you see

Annotate objects that you see

Do NOT annotate bad reflections,

*objects in this image are random examples - only label

*objects in this image are random examples - only label

*objects in this image are random examples - only label

*objects in this image are random examples - only label

Bounding Box Label all visible objects. Make

There should be no more than 3px

Truncation If an object is truncated (cut off at

The box should end at exactly the

Annotation Rules - Polygon

Rule Description Example

Annotation Ordering Draw polygon rst, then bounding

Holes Do NOT cut out holes from objects.

10 Vertex Limit Any polygon created will be hard-

Once 10 vertices are selected, the

Users will then have to adjust

Divided Objects If an object is divided by something

*objects in this image are random

Edge Case How to Handle It Example

When to annotate All floor hazards are small objects

For each object that has a

Exception: when you use the

Apply the other_floor_hazard label

How to annotate a Annotate the cable part as cable.

When to mark floor If any part of the object is touching

Labels & Attributes Description/Examples

Labels/Attributes Description Example

liquid_spill_or_puddle Liquid on the floor. NOT animal

stringy_floor_hazards Backpack_or_handbag (include

Belt (do NOT include clothing

Cables (any type of cable, such

Clothes (do NOT include

Jewelry (do not include if worn

Shoelace (only if coming away

Yarn_or_string (any yarn or

Threshold Change in floor height, usually

miscellaneous_floor_hazard Plastic_bag (be careful if you

Shoes (do NOT include clothing

Hair tie and hair clips

To return to tasking, hit the "esc" key!