Few-shot Detection of Animal Sounds

at the Planetary Scale

Vincent Lostanlen
LS2N, Centrale Nantes, CNRS

Paris, FR, January 2023. My research is funded by the French government.

2

Need for automated biodiversity surveys

10k GBIF points

One year of eBird

GBIF
equivalent (Kitzes
area et al. 2019)
3

Machine learning meets sensor networks

(Tuia et al. 2021)
4

Recognition in terra incognita

(Beery
et al. 2018)
5

Recognition in terra incognita

(Beery
et al. 2018)
6

Advantages of computational bioacoustics

• (Quasi) omnidirectional • Many species are more easily heard than seen

• Operates at night • Potentially long-distance

• (Quasi) independent of weather • Potentially polyphonic

• Able to localize sources • Gives insight on animal behavior


 

 Andrew Farnsworth www.allaboutbirds.org
Rosetta Stone to the Warblers

Ovenbird Worm-eating Warbler Louisiana Waterthrush Northern Waterthrush Golden-winged Warbler Blue-winged Warbler Black-and-white Warbler Prothonotary Warbler Swainson’s Warbler Tennessee Warbler
(Seiurus aurocapilla) (Helmitheros vermivorum) (Parkesia motacilla) (Parkesia noveboracensis) (Vermivora chrysoptera) (Vermivora cyanoptera) (Mniotilta varia) (Protonotaria citrea) (Limnothlypis swainsonii) (Oreothlypis peregrina)

Orange-crowned Warbler Colima Warbler Lucy’s Warbler Nashville Warbler Virginia’s Warbler Connecticut Warbler MacGillivray’s Warbler Mourning Warbler Kentucky Warbler Common Yellowthroat
(Oreothlypis celata) (Oreothlypis crissalis) (Oreothlypis luciae) (Oreothlypis ruficapilla) (Oreothlypis virginiae) (Oporornis agilis) (Geothlypis tolmiei ) (Geothlypis philadelphia) (Geothlypis formosa) (Geothlypis trichas)

Hooded Warbler American Redstart Kirtland’s Warbler Cape May Warbler Cerulean Warbler Northern Parula Magnolia Warbler Bay-breasted Warbler Blackburnian Warbler Yellow Warbler
(Setophaga citrina) (Setophaga ruticilla) (Setophaga kirtlandii) (Setophaga tigrina) (Setophaga cerulea) (Setophaga americana) (Setophaga magnolia) (Setophaga castanea) (Setophaga fusca) (Setophaga petechia)

Chestnut-sided Warbler Blackpoll Warbler Black-throated Palm Warbler Pine Warbler Yellow-rumped Warbler Yellow-throated Warbler Prairie Warbler Grace’s Warbler Black-throated
(Setophaga pensylvanica) (Setophaga striata) Blue Warbler (Setophaga palmarum) (Setophaga pinus) (Setophaga coronata) (Setophaga dominica) (Setophaga discolor) (Setophaga graciae) Gray Warbler
(Setophaga caerulescens) (Setophaga nigrescens)
Bill Evans. www.oldbird.org
 Bill Evans. www.oldbird.org

autonomous recording units

8 pm 10 pm midnight 2 am 4 am
 8 pm 10 pm midnight 2 am 4 am

train

insects

reduced

car traffic
13

How it began: bird migration monitoring

with
B. Van Doren
A. Dokter
A. Cramer
J. Salamon
A. Farnsworth
J. P. Bello
14

Problem: generalizing to new species?

?
?

? ?

?
?
15

Idea: a new benchmark and open challenge

https://dcase.community/challenge2023
16

Multidisciplinary consortium
https://dcase.community/challenge2023
17

Task formulation

Onset-offset
prompt

“Like
tab completion
for sound!”
18
Samples
19

Baseline
20

2022 competitors
21

Leaderboard
22

Conclusion
• Machine listening has untapped potential in the life sciences.

• Spectrogram annotation is tedious and demands expertise.

• We have organized the first challenge for few-shot bioacoustics.

• SOTA has went from ~25% to ~75% F-score since 2021.

• We plan to add new animal taxa in future years.

• The leaderboard remains open. See you in 2023 ?

https://dcase.community/challenge2023