Io Pet Person Sit Sleep Table Spec Document

Bbox, polygon | IN PROGRESS

Contents

Labels & Attributes Table -- needs to be locked by end of Qualification Run

Annotation Rules -- needs to be locked by end of Qualification Run
Critical vs. Non-Critical Errors - needs to be locked by end of Qualification Run
Edge Cases - iterative
Labels & Attributes Description/Examples - needs to be locked by end of
Qualification Run

Zoom around each task carefully to find all the objects listed in the
label/attribute table!

!!!Watch the instructional video here!!!

https://drive.google.com/file/d/12X2USgVcyjsIB2Epo59Uq1xabnec-03r/view

Labels & Attributes Table

Task type/Geometry

Label

Nested Label

Attributes

Bbox

Person_Pet

cat,

dog,

person,

other_pet_or_animal

Bbox

Pet_Accessories

aquarium,

cage_or_kennel,

litter_box,
pet_bed,

pet_food_or_water_bowl,

pet_furniture,

pet_toy,

other_pet_related

Pet accessory location

[on_the_floor,

NOT_on_the_floor]

Bbox

Sit_Sleep_Table

table

Table type

[coffee_table,

dining_table,

sidetable,

tv_table,

desk,

other_table]

Bbox

Sit_Sleep_Table

chair

Chair type

[baby_highchair,

bar_stool,

dining_chair,

livingroom_chair,

office_chair,

other_chair]
Bbox

Sit_Sleep_Table

sleep_and_other

Sleep/other type

[bed,

crib,

sofa]

Annotation Rules - Bbox

Rule

Description

Example

Grouped Objects

Group small objects if their pixels are touching. (small: shoe or smaller. Example:
books, legos, clothing in a pile, shoes in a pile.)

For larger objects, or objects that aren’t touching, use separate annotations.

Individual small objects and pixels are separated, box separately:

*objects in this image are random examples - only label the objects in the
label/attribute table

Small objects and pixels are touching, box together:

*objects in this image are random examples - only label the objects in the
label/attribute table

*objects in this image are random examples - only label the objects in the
label/attribute table

Visibility

Visibility is how much an object is visible.

If an object is occluded, only box what is visible.

Minimum Pixel Size

Only label objects that are 5x5 pixels or bigger.

5 x 5 = annotate

5 x 17 = annotate

3 x 17 = do not annotate

30 x 2 = do not annotate

Label Confidence

Annotate and label if you are 75%+ sure what the object is. If you have a good
guess of what the object is, label it that way!

*objects in this image are random examples - only label the objects in the
label/attribute table

Glass/Reflections

Annotate objects that you see clearly through glass or plastic (you need to see the
item clearly and have 75% confidence that you know what it is).

Annotate objects that you see perfectly reflected in a mirror.

Do NOT annotate bad reflections, like a window reflecting on a black TV screen.

*objects in this image are random examples - only label the objects in the
label/attribute table

*objects in this image are random examples - only label the objects in the
label/attribute table

Extremities

If a person or animal is pushing/pulling/carrying something, do NOT include that

object in the box.
Annotate it separately if it is in the label list. Otherwise, ignore it.

Blurry Objects

Clip the blur to what you think the real size of the object is.

*objects in this image are random examples - only label the objects in the
label/attribute table

Bounding Box Tightness

Label all visible objects. Make sure the boxes fit very tightly.

There should be no more than 3px of space between box edge and object.

Truncation

If an object is truncated (cut off at the edge of the image), keep the box tight
around the visible part of the object.

The box should end at exactly the edge of the image for truncated objects.

Edge Cases

Edge Case

How to Handle It

Example

How to pick the correct chair type

Match chair type to image examples. Do NOT consider location. A dining chair might
be at a desk, or an office chair might be at a dining table.

Critical vs. Non-Critical Errors

Critical Errors
Non-Critical Errors

Labels & Attributes Description/Examples

Labels/Attributes

Description/Example

Person_pet

Cat

Dog

Person

Other_pet_or_animal: birds, snakes, hamsters, turtles, ferrets, etc. Animals bigger

than a fish.

Pet_Accessories

Includes pet related objects like pet toys, bowls, pet beds.

Aquarium

cage_or_kennel

Litter_box (box for a cat to pee or poop)

Pet_bed
Pet_food_or_water_bowl

Pet_furniture

pet_toy

Other_pet_related (anything else pet related EXCEPT poop or animal vomit)

Sit_Sleep_Table

Bed (for human adults)

Crib (bed for a baby)

Chair (Note: do NOT include shoe racks that have a sitting area).

baby_highchair

Bar_stool Tall chair often at a bar

Dining_chair Regular chair often at a dining table

Livingroom_chair Softer chair for lounging

office_chair
Desk

Sofa: include the pillows that are part of the sofa. Do NOT include throw pillows.

Coffee_table (often in front of sofa)

Dining_table (taller larger table often with chairs around it)

Sidetable (or nightstand, often next to sofa or bed)

Tv_table (must be able to see through it. It must NOT have any cabinets or
compartments that you can’t see inside)