Hyperspectral Single Pixel Computer Vision

Andreas Coclet

December 22, 2023

Introduction

▶ Hyperspectral single-pixel imaging combines strengths of

hyperspectral and single-pixel imaging.
▶ Aim: Capture scene spectral data with a single-pixel detector.
▶ Applications: Remote sensing, environmental monitoring,
medical imaging.
Advantages

▶ Remarkable spectral resolution at the expense of spatial

detail, ideal for applications prioritizing spectral precision.
▶ Reduce the amount of data that needs to be transmitted or
stored.
▶ Single Pixel Computer Vision hyperspectral goes beyond limits
of silicon sensors, sees what our eyes cannot see.
Images Reconstitution Methods

Traditional methods:
▶ Compressing sensing with l1 minimization.
▶ FHT + IFT.
Limitations: Time-consuming, sparse representation.
CNN-based methods (state of the art):
▶ Limitations: Learning ability of CNN is limited, only considers
spatial characteristics, ignores spectral characteristics,
convolution not effective for long-range dependency modeling.
Methods

Swin Transformer Layer for Spatial Correlation Priors:

Capture global interactions between images or their patterns.
3D CNN to Learn Spectral Priors of the HSI.
Motivation

▶ The self-attention mechanism in the Transformer requires

each pixel of the image to pay attention to every other pixel,
enabling the Transformer to have long-range modeling
capabilities. Well-suited for extracting the spatial prior of
HSIs.
▶ For the spectral prior, 3D-CNN is a natural and effective
choice, paying attention to the correlation of spatio-spectral
while learning spectral priors.
Related Work

▶ J. Liang, Image restoration using Swin Transformer,

Proceedings of the IEEE International Conference on
Computer Vision, 2021.