Assignment 2 - Object Detection and Semantic

Segmentation

Given your task (Semantic Segmentation or Object Detection), you have to test three (3)
pretrained models on the test set of a given dataset, comparing the results between the
models. For the comparison, you have to use at least three (3) metrics commonly used in
your task.
For example, if student A is given Object Detection task and the Ballons v2 dataset, then
his/her task is to choose 3 pretrained object detection models [R-CNN, YOLO, ...] and
evaluate them on the Ballons v2 test set, comparing the results using 3 metrics [mAP,
Recall, ...]. If a dataset does not have a testing set, then the student should split at least
10% of the training set to create the test set.
Both the task and dataset are given
here: https://docs.google.com/spreadsheets/d/1KbIIl2YfRXMFM1eQLkiggZepr2oirqe8
9jge62arzSY/edit?usp=sharing (Links to an external site.)
Each student is attributed 1 task and 1 dataset, but is free to choose any 3 models and
any 3 metrics.
Use this notebook as template for the assignment (copy it to your local
drive): https://colab.research.google.com/drive/10x2AhPSJklfyz2uNhjiqug8NqU9U_KG
y?usp=sharing (Links to an external site.) (Links to an external site.)
The list of datasets per task is below:
Object Detection:

 CelebA https://mmlab.ie.cuhk.edu.hk/projects/CelebA.html (Links to an

external site.)
 Coco Detection https://cocodataset.org/#detection-2020 (Links to an
external site.)
 KITTI Object
2D http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=2d (
Links to an external site.)
 Pascal VOC
Detection http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html#data
(Links to an external site.)

Semantic Segmentation:

 CityScapes https://www.cityscapes-dataset.com/ (Links to an external site.)

  Pascal VOC
Segmentation http://host.robots.ox.ac.uk/pascal/VOC/voc2012/index.html#
data (Links to an external site.)
 KITTI Semantics Pixel
level http://www.cvlibs.net/datasets/kitti/eval_semseg.php?benchmark=sema
ntics2015 (Links to an external site.)
 ADE20K http://groups.csail.mit.edu/vision/datasets/ADE20K/ (Links to an
external site.)

http://groups.csail.mit.edu/vision/datasets/ADE20K/

IMPORTANT: If there are any difficulties downloading the dataset, use pytorch
datasets: https://pytorch.org/vision/stable/datasets.html (Links to an external site.)