GESTURE RECOGNITION USING WEARABLE VISION SENSORS

TO ENHANCE VISITORS MUSEUM EXPERIENCES

ABSTRACT
We introduce a novel approach to cultural heritage experience: by means of ego-vision
embedded devices we develop a system, which offers a more natural and entertaining way of
accessing museum knowledge. Our method is based on distributed self-gesture and artwork
recognition, and does not need fixed cameras nor radio-frequency identifications sensors. We
propose the use of dense trajectories sampled around the hand region to perform self-gesture
recognition, understanding the way a user naturally interacts with an artwork, and demonstrate
that our approach can benefit from distributed training. We test our algorithms on publicly
available data sets and we extend our experiments to both virtual and real museum scenarios,
where our method shows robustness when challenged with real-world data. Furthermore, we run
an extensive performance analysis on our ARM-based wearable device.