INTRODUCTION

Tangible programming to manipulate the robot's movement.

Current system: Open-source library, recognition through markers.

Problems: Issues with Lighting, Camera Angles and Background Contrast

Research Question:

What object detection algorithm best addresses the challenges posed by

poor lighting conditions, robustness, and computational efficiency in
tangible-based programming environments designed for children?
PRELIMINARY RESEARCH

Traditional algorithms are challenged by computational complexity.

One-Stage Object Detector Algorithm

YOLO

SSD

Two-Stage Object Detector Algorithm

Faster R-CNN
METHODOLOGY
Custom Dataset Preparation

Object Detection Algorithm Comparison

Tangible Features Influence on Computer Vision Performance

DATA PREPARATION

Image Capture:
Varied lighting conditions for realistic, tangible
programming scenarios

Roboflow Tool
Labelling
Preprocessing
Augmentation
ALGORITHM COMPARSION

Evaluation Metrics:

mAP (Mean Average Precision)

Inference time

Low lighting conditions

Model Size

Training Time
FEATURE INFLUENCE

Algorithm Finalization: Selection based on

performance and adaptability

Evaluating Block Features: Assessing how block

characteristics impact detection accuracy

Design Implications: Informing the creation of new

tangible blocks for enhanced detection

RESULTS
Object Detection Algorithm Comparison

Custom Dataset and COCO Dataset Experiment

Using smaller model sizes (YOLOv5s, SSD Mobilenet, Faster R-CNN MVGG Backbone)

Tangible Features Influence on Computer Vision Performance

CUSTOM DATASET RESULTS
METRICS YOLO SSD FASTER-RCNN

mAP 91.9 92.5 95.7

Inference Time 57 98 354

LLC Moderate Good Moderate

Model Size 14.8MB 29MB 230MB

Training Time 15 min 20 min 35 min

COCO DATASET RESULTS
FASTER-
METRICS YOLO SSD
RCNN

mAP 37.4 40.8 55.7

Inference Time 61 85 172

Ease of
Easy Hard Hard
Implementation
ALGORITHM COMPARISON RESULTS

YOLOv5s Optimal Choice

A great blend of speed vs accuracy

Amazing performance on low-power computers

Very easy to train and implement

FEATURE INFLUENCE RESULTS

Symbol + Text 91.8

Symbol 91.9

Text 87.6

Greyscale 91.2
0 25 50 75 100

mAP
FEATURE INFLUENCE RESULTS

Symbols are crucial for detection

Text is not important - overpowered by symbols

Colours do not improve object detection accuracy

High contrast between symbol and background

FINDINGS AND FUTURE WORKS

YOLOv5s is the optimal choice for our use case

Unique symbols and high symbol-background contrast

Utilise these findings to train the model on remaining untrained tangibles

Use findings as guidelines for designing new blocks

Integrate object detection model into the “KareNao” application

THANK YOU FOR
LISTENING