LUCC Classification with Big Earth

Observation Data: Challenges and

Current Status

Gilberto Camara
https://gilbertocamara.org
image: Alamy
Big EO data for sustainable development

Big satellite data Data cubes Machine learning

In what ways is big spatial data special?

images: NASA, Swiss Data Cube, ECMWF
Big EO data: access to image time series

image: F Petitjean, Monash Univ

 Time series measure change

How to classify land use and land cover using time series?
4
graphics: Maus, Victor
 Space first, time later or time first, space later?

Space first: classify images;

compare results in time

Time first: classify time series;

join results to get maps
graphics: V Maus, author
 objects exist, events occur
image: Sicilian Convention Bureau

Mount Etna is an object. Etna’s 2018 eruption was an event.

 Objects and events in EO data

Objects Events
Deforestation, degradation,
Forest
regeneration
Crop area Sowing, Senescence, Bare Soil

Savanna Burned area

Ice shelf Break-up, collapse

Pasture Renewal, degradation

Pristine forest Degradation by logging

Deforestation as a series of events

Degradation by fire Clear-cut

images: Alamy, INPE

 Deforestation event detection

2010 2011
Distinguishing forests by temporal evolution

source: Chazdon et al (2016)

 How to analyze image time series?

CCDC, BFAST,
Parametric TimeSat

Atemporal: RF, MLP,

XGBoost
Data-driven
Time-sensitive:
TempCNN, LightTAE
CCDC (Bullock et al., RSE 2020)
 Yu et al. (2021)

Machine learning models for image time series

Random forests (hierarchical model) Transformers (relational model)

Corn

Soybeans

Transformers: 2-5% gain over RF (improved UA/PA over less frequent classes)
 Deforestation in Amazonia: PRODES

Transparent and authoritative data: REDD+ funds (US$ 1,3 billion),

decision-makers (Brazil’s NDC), researchers (1,000+ papers)
 Event-based samples (model estimates)
Burned area Cleared area

Forest Forest cut with regeneration

 Tropical deforestation monitoring

PRODES (the “gold standard”) 400 samples, LTAE (97% agreement)

SITS workflow

18
 End-user tool for cloud services (TRL 8)

Digital Earth Africa Brazil Data Cube

Earth on AWS FAO - SEPAL

Planetary Computer 19
What is an EO data cube?

Data cube = regular partitions of space and time

20
 Good training data is key to success

Santos et al. (2021)

SOM – self-organized maps for dimensionality reduction

 Using SOM to assess sample quality

Alves et al., “Quality control and class noise reduction of satellite image time series”, ISPRS
Journal of Photogrammetry and Remote Sensing, vol. 177, pp 75-88, 2021
 Easy-to-learn API

“We looked at Google Earth Engine and copied the good parts”
Open Source Availability

https://github.com/e-sensing/sits

https://e-sensing.github.io/sitsbook/
 References (author)
1. Rolf Simoes, Gilberto Camara, et al. “Satellite Image Time Series Analysis for Big Earth Observation Data”.
Remote Sensing, 13, p. 2428, 2021.
2. Lorena Santos, Karine Ferreira, et al., “Quality control and class noise reduction of satellite image time
series”. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 177, pp 75-88, 2021.
3. Gilberto Câmara, “On the semantics of big Earth observation data for land classification”, Journal of Spatial
Information Science, 20, p. 21–34, 2020.
4. Michelle Picoli, Ana Rorato, et al., “Impacts of Public and Private Sector Policies on Soybean and Pasture
Expansion in Mato Grosso—Brazil from 2001 to 2017”. Land, 9(1), 2020.
5. Michelle Picoli, Gilberto Camara, et al., “Big Earth Observation Time Series Analysis for Monitoring
Brazilian Agriculture”. ISPRS Journal of Photogrammetry and Remote Sensing, 2018.
6. Gilberto Camara, Max Egenhofer, et al., “Fields as a generic data type for big spatial data”. GIScience 2014.
Vienna, Austria, 2014.
7. Gilberto Câmara, Max Egenhofer, et al., “What’s in an image?” In: D. Montello (ed), COSIT´01, Conference
on Spatial Information Theory, Morro Bay, EUA, 2001.
 References (others)
1. Xu et al., “Towards Interpreting Multi-Temporal Deep Learning Models in Crop Mapping.” Remote Sensing
of Environment, 2021.

2. Crawford, Tuia, and Yang, “Active Learning: Any Value for Classification of Remotely Sensed Data?
Proceedings of the IEEE, 101, 2013.

3. Monarch, R., “Human-in-the-Loop Machine Learning: Active Learning and Annotation for Human-centered
AI”. Manning Publications, 2021.

4. Garnot, Landrieu, et al., “Satellite Image Time Series Classification With Pixel-Set Encoders and Temporal
Self-Attention”. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

5. Garnot, Landrieu, et al., “Lightweight Temporal Self-attention for Classifying Satellite Images Time Series”.
Arxiv:2007.00586., 2021.

6. Ofori-Ampofo, Pelletier, and Lang, “Crop Type Mapping from Optical and Radar Time Series Using
Attention-Based Deep Learning”. Remote Sensing, 2021.
Backup slides
Our concepts have limits

“The gradation of properties in

the world means that our
smallish number of categories
will never map perfectly onto all
objects” (Murphy, 2004)

image: Amazon
 What’s in an image?

“A remote sensing image is a measurement that captures snapshots of

change trajectories. The focus of the ontological characterization of images should
be on searching for changes instead of searching for content.” (Camara et al,
COSIT 2001)
images: USGS
 Natural or man-made landscapes?

Natural savanna (Brazilian Cerrado) Cattle in pasture (in the Cerrado)

images: Nature Conserv, Alamy
Distinguishing natural and man-made: benefits
Locally relevant?
ESA World Cover 2020

Simões et al (2021)
@geosec2025
www.earthobservations.org
 Words refer to objects in the world images: Alamy

urban area agriculture

forest savanna
Cerrado: Brazilian Savanna

covers 2 million km2, latitude ranges from 50S to 250S

Open cerrado Cerrado strictu sensu Cerradão (dense woodland)

images: Del-Claro and Silingardi (Annals Brazilian Academy Sci, 2019 ), Schwieder et al. (IJAEOG, 2016)
 SITS land classification workflow

LUCC samples Quality Analysis LUCC maps

EO data cubes ML models Classification and smoothing

When is a forest not a forest?

images: Alamy
 Tropical forest Temperate forest
The ambiguity of “forest”

Dry forest Planted forest

images: Alamy