Expert Systems With Applications 238 (2024) 121902

Contents lists available at ScienceDirect

Expert Systems With Applications

journal homepage: www.elsevier.com/locate/eswa

Acoustic scene classification: A comprehensive survey

Biyun Ding a, 1, Tao Zhang a, *, 2, Chao Wang a, 3, Ganjun Liu a, 4, Jinhua Liang b, 5, Ruimin Hu c, 6,
Yulin Wu c, 7, Difei Guo d, 8
a
School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
b
Centre for Digital Music, Queen Mary University of London, London, UK
c
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, China
d
Tianjin Communication and Broadcasting Group Company Limited, Tianjin, China

A R T I C L E I N F O A B S T R A C T

Keywords: Acoustic scene classification (ASC) has gained significant interest recently due to its diverse applications. Various
Acoustic scene classification audio signal processing and machine learning methods have been proposed for ASC. The volume and scope of
Data augmentation ASC publications covering theories, algorithms, and applications have also been expanded. However, no recent
Datasets
comprehensive surveys exist to collect and organize the knowledge, impeding the ability of researchers and its
DCASE
Deep learning
applications. To fill this gap, we present an up-to-date overview of ASC methods, covering earlier works and
Environmental sound recent advances. In this work, we first define a general framework for ASC, starting with a historical review of
Feature extraction previous research in the ASC field. Then, we review core techniques for ASC that have achieved good perfor­
Modeling mance. Focus on machine learning based ASC systems, this work summarizes and groups the existing techniques
in terms of data processing, feature acquisition, and modeling. Furthermore, we provide a summary of the
available resources for ASC research and analyze ASC tasks in Detection and Classification of Acoustic Scenes and
Events (DCASE) challenges. Finally, we discuss limitations of the current ASC algorithms and open challenges to
possible future developments toward practical applications of ASC systems.

1. Introduction
Acoustic scene classification (ASC) is to assign a predefined semantic
label to an audio stream recorded in a certain environment by analyzing
the audio signal (Waldekar & Saha, 2020b). For instance, an ASC system
may categorize an audio clip under predefined labels such as bus, office,
or home (Virtanen, Plumbley, & Ellis, 2018). Generally, semantic labels
stem from environmental sound categorizations that describe the
ambiance of audio streams. An overview of an ASC system is shown in
Fig. 1. ASC is usually treated as a single-label classification task, and its
corresponding multi-label classification task is audio tagging. An
acoustic scene refers to the entirety of sounds from various sources,
typically merging real-world scenarios into a composite (Virtanen et al.,
2018). The different terms for acoustic scenes include environments of
sounds and acoustic/sonic/sound/audio environments.
Various potential applications promoted the ASC study, such as
multimedia information retrieval (Virtanen et al., 2018), context
awareness in smart devices (Vivek, Vidhya, & Madhanmohan, 2020;
Chu et al., 2006; Hüwel, Adiloğlu, & Bach, 2020), smart homes/cities,
bioacoustic scene analysis, and acoustic abnormal monitoring (Dong
et al., 2018). ASC could enable several prospective technologies, such as
automated sound archives assigning metadata to audio; smart devices

* Corresponding author.
E-mail addresses: beatrice@tju.edu.cn (B. Ding), zhangtao@tju.edu.cn (T. Zhang), wwchao@tju.edu.cn (C. Wang), ganjun_liu@tju.edu.cn (G. Liu), jinhua.liang@
qmul.ac.uk (J. Liang), hrm@whu.edu.cn (R. Hu), wuyulin@whu.edu.cn (Y. Wu).
1
https://orcid.org/0000-0001-6387-5857
2
https://orcid.org/0000-0003-2317-644X
3
https://orcid.org/0000-0002-1144-8041
4
https://orcid.org/0000-0003-3919-5370
5
https://orcid.org/0000-0002-4570-0735
6
https://orcid.org/0000-0002-0290-5757
7
https://orcid.org/0000-0003-1940-2141
8
No orcid

https://doi.org/10.1016/j.eswa.2023.121902
Received 6 December 2022; Received in revised form 20 August 2023; Accepted 27 September 2023
Available online 2 October 2023
0957-4174/© 2023 Published by Elsevier Ltd.
B. Ding et al.
Fig. 1. Overview of an ASC system.
leveraging ASC to adapt functions to their surroundings (e.g., intelligent
wearable devices, hear aids or robotic wheelchairs); information capture
for intelligent decision-making (e.g., noise monitoring, classification of
domestic activities (Ebbers, Keyser, & Haeb-Umbach, 2021), autono­
mous surveillance (Chandrakala & Jayalakshmi, 2019), and workplace
scene analysis (Jati et al., 2021)); or acoustic abnormal monitoring for
early warning (e.g., building health monitoring (Kawamoto & Hama­
moto, 2020) and fiber optic security system (He & Zhu, 2021)).
Similar to ASC, Sound Event Detection (SED) is also used to capture
environment information. It recognizes what is happening in an audio
signal and when it is happening (2017; Xia et al., 2019; Mesaros et al.,
2021). Its annotation is event classes with temporal information (i.e.,
onset and offset times of a sound event). In practice, SED aims to
recognize in what temporal instances different sounds are active within
an audio signal. The difference between ASC and SED lies in the anno­
tation, where ASC is a single-label without temporal information task
and SED is multi-label with temporal information task. In some contexts,
like multimedia retrieval systems (Vivek, Vidhya, & Madhanmohan,
2020), the lines between SED and ASC might blur. Also, ASC can be used
to boost the SED performance by providing prior information about the
probability of certain events (Barchiesi et al., 2015). Therefore, there
may be overlapping applications for both.
ASC has been an active research field for decades. The perception of
environmental sounds has garnered significant attention. Several
important challenges have made significant progress for ASC, such as
the Detection and Classification of Acoustic Scenes and Events (DCASE)
(Bai et al., 2020).
In comparison to speech and music signals, audio signals often
exhibit greater diversity and complexity. Therefore, ASC lags behind
other audio domains like speaker identification and music classification
in development. Various signal processing and machine learning
methods have been proposed to address the low-performance problem,
such as signal representation, data augmentation, feature learning,
efficient convolutional neural networks (CNN) based modeling (Lee,
Fig. 2. General framework of the ASC system.
2
Expert Systems With Applications 238 (2024) 121902
Lim, & Kwak, 2021), and post-processing methods. The general frame­
work of the ASC system is shown in Fig. 2, including data processing,
feature acquisition, and modeling steps. These three stages have been
the focal points for investigating various ASC techniques.
In the training stage, the input features are obtained from the
training set by data processing and feature acquisition, then along with
labels fed to a model that is initialized randomly or initialized by transfer
learning for model training. In the testing stage, the same input features
extracted from the testing set are fed into the well-trained ASC model to
get outputs.
Data processing aims to enhance data quality or increase quantity for
boosting ASC performance, including channel selection and data
augmentation. Channel selection refers to selecting appropriate signal
channels beneficial to performance improvement for ASC. Commonly-
used signal channels in ASC include mono, binary (left and right),
mid/side channels (Lee et al., 2021), and signal variants generated by
Harmonic Percussive Source Separation (HPSS) or Nearest Neighbor
Filtering (NNF). Data augmentation aims to increase training data
through techniques such as mixing, pitch shifting, time stretching,
adding random noise, mixup, channel confusion, dynamic range
compression, spectrum augment, spectral rolling, random erasing,
Between-Class learning, spectrum correction, Generative Adversarial
Networks (GAN), etc. Sometimes the combination of several data
augmentation methods is also employed (Hu et al., 2020; Hu et al.,
2021).
Feature acquisition is used to obtain distinguished features for effi­
cient classification, including feature learning, handcraft feature
extraction, and the techniques of feature selection and dimensionality
reduction. Common handcraft features for ASC include Log Mel Band
Energies (LMBE), their delta and delta-delta values, Mel Frequency
Cepstral Coefficients (MFCC), Constant-Q-Transform (CQT) features,
etc. Feature learning is usually implemented by deep learning methods,
such as Deep Neural Networks (DNN), CNN, Visual Geometry Group
(VGG), Nonnegative Matrix Factorization (NMF), Restricted Boltzmann
Machine (RBM), etc. In addition, some networks are designed for feature
learning in audio classification, such as L3-Net and SoundNet. Feature
selection and dimensionality reduction aim to obtain a significant and
compact feature set.
Modeling is generally implemented using machine learning methods,
such as supervised, unsupervised, semi-supervised, and self-supervised
methods. Supervised methods are the most popular methods for ASC,
including conventional machine learning and deep learning methods.
Most of the earlier ASC works employed conventional machine learning
methods for modeling, such as K-Nearest Neighborhood (KNN),
Gaussian Mixture Modelling (GMM), Support Vector Machines (SVM),
Hidden Markov Model (HMM), random forest, and decision tree.
Recently, deep learning methods have been successfully applied in ASC
and achieved excellent results, such as Time Delay Neural Network
(TDNN), Bidirectional Long Short Term Memory (Bi-LSTM), Feedfor­
ward Neural Networks (FNN), CNN (e.g., traditional CNN, VGG, Re­
sidual Network (ResNet), Inception, Xception, and MobileNetV2, etc),
Fully Convolutional Neural Network (FCNN), Convolutional Recurrent
Neural Network (CRNN), etc. Furthermore, attention mechanisms,
transfer learning, and multitask learning methods are also introduced
into ASC. Typical examples include SoundNet, SubSpectralNet, DCA­
SENet, CAA-Net, and PANNs. Unsupervised, semi-supervised, and self-
supervised methods are studied as new directions for classification
tasks to overcome the shortcoming of requiring a large amount of
labeled data in deep learning. They have been successfully applied in
image classification and natural language processing, then are gradually
introduced into ASC. Also, late fusion methods are used in ASC to
improve performance (Ding et al., 2022; Paseddula & Gangashetty,
2021).
Beyond performance concerns, real-world implementations often
run on devices with a limited computational capacity (Martín-Morató
et al., 2021). Most systems with high model complexity pose a big
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

challenge in wide use. To address this problem, various model
compression and simplification methods are applied in ASC, such as
parameter pruning and sharing, low-rank factorization, transferred/
compact convolutional filters, knowledge distillation (Gou et al., 2021;
Kim, Yang, Kim, & Chang, 2021; Kim, Hyun, Chung, & Kwak, 2021), and
model simplification (Li et al., 2020).
Recently, ASC trends from the classification using multiple devices
(which entails domain adaptation), the use of external data (which in­
volves transfer learning), and open set to more recently low-complexity
ASC. Numerous methods aim to enhance performance while reducing
model complexity. The challenge lies in optimizing performance within
the bounds of limited computational complexity. At present, there are no
clear instructions on optimal ASC algorithms and methods to reduce
complexity. Although there are some literature reviews for ASC, most of
them summarize ASC from one or two perspectives, such as review of
deep learning for ASC (Abeßer, 2020), survey on ASC and SED for
autonomous surveillance (Chandrakala & Jayalakshmi, 2019), and
recent survey on preprocessing and classification techniques for acoustic
scene (Singh, Sharma, & Sur, 2023). Besides, some of them were pub­
lished in early period (Barchiesi et al., 2015; Dandashi & AlJaam, 2017),
which means they cannot provide a summary and analysis of the latest
progress. In this work, we present a comprehensive survey of the state-
of-the-art methods for ASC from multiple perspectives.
The main contributions of this paper are: i) A systematic review of
ASC, covering the unified description of ASC, environmental sound
categorization for defining acoustic scenes, the development of ASC, and
the current research, ii) A categorization grouping ASC methods ac­
cording to their processing, allowing us to link ASC independently of
their particular application, and iii) Suggestions for future research.
The following section introduces the systematic literature review
methodology used in this work. Section 3 introduces the ASC back­
ground, including environmental sound categorization, the history of
ASC, and problems in ASC. The core methods of ASC include data pro­
cessing, feature acquisition, and modeling methods, which are discussed
in Section 4, Section 5, and Section 6, respectively. Then, we review
open resources for ASC and ASC tasks of DCASE challenges in Section 7
and Section 8. Finally, Section 9 discusses open challenges and future
works before Section 10 concludes this article. All related acronyms are
listed in Table 1.
2. Methodology
This paper provide a comprehensive survey on ASC research from the
perspectives of background, core technologies, open source resources,
challenges, and ultimately future development trends. This content
borrows extensively from many ASC-related publications (before August
2023). In this section, we detail the systematic literature review meth­
odology used in this work to illustrate the survey scope, retrieval data­
bases, and criteria for inclusion and exclusion.
To intuitively show the scope of the survey, we provide the Preferred
Reporting Items for Systematic Reviews and Mete-Analysis (PRISMA)
flow diagram in Fig. 3. This diagram shows how many papers were
retrieved from the given query, how each screening phase removed
undesired ones, and eventually how many papers were included in this
survey. In Fig. 3, there are 683 ASC-related studies identified from the
retrieval databases, 165 documents screened from the DCASE commu­
nity, and 161 included studies obtained by other methods. A total of 330
papers are cited in this work.
From Fig. 3, we firstly identify scientific publications from
commonly-used digital database, such as Scopus, Web of Science
(abbreviated as Wos), IEEE Xplore (abbreviated as IEEE), and Springer
Link (abbreviated as Springer). Due to several different terms for the
acoustic scene, we set the query strings during searching according to
the common terms. The query strings include acoustic scene classifica­
tion, environmental sound classification, sound scene classification, and
audio scene classification. Also, we adopt slightly different search
methods according to different database search settings. For example,
we search papers based on multiple query strings simultaneously
through OR (satisfying one of them) in the Scopus and Wos databases.
We search papers based on a single query string in the IEEE and springer
databases. In addition, we also use the automatic filtering tools provided
by the databases to conduct preliminary screening of search results.
Then, we integrate the search results of Scopus & Wos, IEEE, and
Springer through removing duplicates. Next, we exclude records that are
irrelevant to ASC based on human evaluation. Finally, 683 papers are
obtained after integrate all above search results and removing dupli­
cates. After meta-analysis, 128 of them are included in this work.
Furthermore, we also collected 602 documents (i.e., papers in DCASE
workshops and technical reports of ASC tasks in DCASE challenges) from
DCASE community. Then, 165 documents can be screened by excluding
papers irrelevance to ASC and low-ranking reports (i.e., low-
performance ASC methods). Next, 41 of them are selected by meta-
analysis as cited references. In addition, we identify 161 studies via
other methods, such as reference tracking, citation searching, and
searching with extra keywords (e.g., acoustic features, channel, data
augmentation, deep learning, model compression, multitask, transfer
learning, attention mechanism, and multiple devices). Among these
studies, 104 are related literature on ASC, and 57 are related to specific
techniques.
In summary, this work cites about 18.7 % (128/683) of the searching
studies from databases, which account for about 38.8 % (128/330) of
the cited references. Moreover, cited papers come from DCASE

Table 1
List of important abbreviations in alphabetical order.
Abbr. Definition Abbr. Definition Abbr. Definition

AMFB Amplitude modulation filterbank GAN Generative Adversarial Networks NMF Nonnegative Matrix Factorization
ASA Auditory Scene Analysis GFB Gabor Filterbank NNF Nearest Neighbor Filtering
ASC Acoustic Scene Classification GFCC Gammatone Feature Cepstral Coefficients PCA Principal Component Analysis
ASM Acoustic Segment Model GMM Gaussian Mixture Modelling PCEN Per-Channel Energy Normalization
Bi-LSTM Bidirectional Long Short Term Memory HMM Hidden Markov Model RBM Restricted Boltzmann Machine
BOF Bag-Of-Frames HOG Histogram Of Gradient ResNet Residual Network
CASA Computational Auditory Scene Analysis HPSS Harmonic-Percussive Source Separation RNN Recurrent Neural Network
CASR Computational Auditory Scene Recognition ICA Independent Component Analysis RQA Recurrence Quantification Analysis
CELP Code Excited Linear Prediction KD Knowledge Distillation S2SAE Sequence To Sequence Auto-Encoder
CNN Convolutional Neural Network KNN K-Nearest Neighborhood SED Sound Event Detection
CQCC Constant-q Cepstral Coefficients LBP Local Binary Pattern SPD Subband Power Distribution
CQT Constant Q Transform LDA Latent Dirichlet Allocation STFT Short-Time Fourier Transform
CRNN Convolutional Recurrent Neural Network LMBE Log Mel Band Energies SVM Support Vector Machines
DNN Deep Neural Networks LPCC Linear Prediction Cepstral Coefficients TDNN Time Delay Neural Network
DRC Dynamic Range Compression MFCC Mel Frequency Cepstral Coefficients VGG Visual Geometry Group
FCNN Fully Convolutional Neural Network MLP Multilayer Perceptron
FNN Feedforward Neural Networks MP Matching Pursuit

3
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

Fig. 3. The PRISMA flow diagram of this work. Due to the limited image space, we abbreviate classification as C in the figure. For example, acoustic scene clas­
sification is abbreviated as acoustic scene C. Studies identified via other methods: reference tracking, citation searching, and keywords searching.

community account for about 12.4 % (41/330). While cited references
obtained via other methods account for 48.8 % (161/330), where cited
references related to ASC account for 31.5 % (104/330), and cited ref­
erences related to specific techniques account for 17.3 % (57/330).
To provide a global view of the ASC-related literature, we also
display the distribution of ASC-related literature over different years in
Fig. 4. From this figure, it can been seen that the number of ASC-related
publications has increased with the years, especially since 2016. In
particular, the number of publications from 1997 to 2005 was relatively
small. Although there was a slight increase in the number of publications
from 2006 to 2015, it is still small. The number of ASC-related publi­
cations from 2016 to 2023 has rapidly increased. Therefore, the number
of ASC publications in this period is much higher than in early periods. It
means that the ASC study has received widespread attention.
3. Background
Although ASC has attracted extensive attention, its research is still in
its developing stages. In this section, we first introduce environmental
sound categorization for defining acoustic scenes. Then, we summarize
the development history of ASC in terms of three phases. Finally, we
summarize and analyze the problems faced in the ASC field.
3.1. Environmental sound categorization
In the 1990s and 2000s, there has been a growing body of literature
about the categorization of everyday sounds. For instance, Gaver
(Gaver, 1993) grouped the sound event into three categories: solids,
gasses, and liquids based on sound-producing materials. Moreover, two
categorization principles (i.e., sound source similarity and the similarity
between events, or actions, causing the sound) are proposed (Van­
Derveer, 1979). Situational factors based principles are also highlighted
(Marcell et al., 2000; Gygi, Kidd, & Watson, 2007), such as the sound’s
location or context and hedonic judgments associated with emotional
responses. Houix et al. (Houix et al., 2012) found a close interaction
between action and sound source, which is opposite to the independent
relationship suggested by Gaver.
Guastavino identified two urban soundscapes in a free sorting task

Fig. 4. Distribution of ASC related literature with years. n_searching presents the papers identified by searching key words from databases (including Scopus, Wos,
IEEE, and springer). n_included points the references cited in this work. n_included_searching means the references cited in this work, which identified from da­
tabases. It is the intersection of n_searching and n_included. n_included_others presents the references cited in this work, which identified via other methods.
Therefore, n_included is the total of n_included_searching and n_included_others.

4
B. Ding et al.
according to the absence or presence of human activity in the sound
recording related to hedonic judgments (Guastavino, 2007; Virtanen
et al., 2018). Their findings provided support for another form of
interaction between categorization principles based on sound sources,
activities, and hedonic judgments. Guastavino also indicated that
sounds can be cross-classified according to different categorization
principles depending on the sound’s context and the participants’ goals
and theories. The extent to which participants rely on the acoustic
properties of the sound vs. the semantic properties of the sound source
also varies for different types of sounds (Giordano, McDonnell, &
McAdams, 2010).
Cross-classification into different categorization schemes allows lis­
teners to get a full understanding of their environments from a goal-
oriented perspective (Virtanen et al., 2018). For instance, Brown et al.
(Brown, Kang, & Gjestland, 2011) proposed a taxonomy of acoustic
environments based on principles of environment types, the presence or
absence of human activity, and sound sources. Salamon et al. (Salamon,
Jacoby, & Bello, 2014) proposed a taxonomy of urban sounds using
principles of sound sources, actions, and their combinations. A rich
description of sound events across different descriptor layers (i.e., sound
sources, actions, and contexts) was proposed (Virtanen et al., 2018). In
addition, influence factors of everyday sound categorization are also
studied, consisting of similarity (Lemaitre et al., 2010), person-related
factors (expertise of the listeners, age (Berland et al., 2015), and pref­
erences), and situational factors (mood, attention, activity) (Steffens,
Steele, & Guastavino, 2017).
Environmental sound categorization is critical for designing and
evaluating sound classification systems. It defines acoustic scenes and
thus provides predefined labels for ASC tasks. However, there is a dif­
ference between theoretical sound categorization and the definition of
sound classes for practical classification tasks. In specific sound classi­
fication tasks, real data do not always cover all sound categories. For
example, ASC tasks in the urban environment focus more on traffic and
human activities but less on natural sounds, which is very different from
a rural or wilderness environment.
To overcome the problem above, we derive a taxonomy of acoustic
environments shown in Fig. 5. This taxonomy is constructed in terms of
types of environments—urban, rural, wilderness, and underwater
(Brown, Kang, & Gjestland, 2011). Acoustic environments generally
present mixtures of specific sound sources. Therefore, we further cate­
gorize them in terms of different principles. We first divide urban en­
vironments into three abstract levels: indoor, outdoor, and
transportation, and then further group them in terms of types of places,
e.g., home, office, and public spaces (Virtanen et al., 2018; Chandrakala
& Jayalakshmi, 2019; Bai et al., 2020). We divided rural, wilderness,
and underwater environments based on sound sources and actions
Fig. 5. Taxonomy of acoustic environments inspired by previous studies and ASC tasks of DCASE challenges.
5
Expert Systems With Applications 238 (2024) 121902
(Salamon, Jacoby, & Bello, 2014). We only list some typical classes at
the low level in Fig. 5 because the acoustic environment taxonomy relies
on application scenarios. There are overlaps among urban, rural, wil­
derness, and underwater environments. Hence, the acoustic environ­
ment divide will not always be clear-cut. Despite all this, the proposed
taxonomy of acoustic environments is applicable and encourages the
definition of acoustic scenes using coincident nomenclatures.
3.2. The history of ASC
We introduce key developments of ASC with aspects of three phases:
problem raise, primary exploration, and early in-depth research phases.
The first phase begins with the soundscape (Schafer, 1977), pioneering
work in which researchers pay attention to acoustic environments.
Later, multiple ASC related concepts were proposed, and the automatic
classification of environmental sounds received some attention. In the
second phase, the publication of “computational auditory scene anal­
ysis” (Wang & Brown, 2006) and the study of comprehensively evalu­
ating computers’ and humans’ perception of audio context (Eronen
et al., 2006) started the early works of ASC. In this phase, ASC algo­
rithms applied features from speech recognition areas and simple ma­
chine learning methods, as well as several ASC datasets were released.
The third phase started from the DCASE2016 challenge (Eghbal-Zadeh
et al., 2016), where deep learning methods were introduced into ASC
and achieved high performance. In this phase, ASC received widespread
attention, and the ASC publications rapidly increased. For a more
concise description, we abbreviate the DCASE task1 (ASC task) to Dxx.
For example, DCASE 2013 task1 is abbreviated to D13. Similarly, DCASE
2019 task 1A (resp. 1B) is abbreviated to D19A (resp. D19B).
3.2.1. Problem raise phase (1977–2005)
In this phase, many ASC-related concepts were proposed and then
ASC studies were started. We review the development of this phase
(shown in Fig. 6). Table 2 provides early ASC approaches evaluated on
self-create databases in this phase. Here, ASC relates to perceptual
studies and computational research (Barchiesi et al., 2015). The
perceptual studies refer to soundscape cognition (Dubois, Guastavino, &
Raimbault, 2006) by defining soundscapes as the auditory equivalent of
landscapes (Schafer, 1977), aiming to understand the human cognitive
processes that enable our understanding of acoustic scenes. The
computational research is called Computational Auditory Scene
Recognition (CASR) (Peltonen et al., 2002), attempting to automatically
perform ASC (Barchiesi et al., 2015). CASR relates to the area of
Computational Auditory Scene Analysis (CASA) (Brown & Cooke, 1994;
Wang & Brown, 2006) and is particularly applied to the study of envi­
ronmental sounds (Ellis, 1996).
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

Fig. 6. Summary of key developments in problem raise phase of ASC (Schafer,1977; Bregman, 1990; Gaver, 1993; Brown & Cooke, 1994; Ellis, 1996; Sawhney &
Maes, 1997; Clarkson, Sawhney, & Pentland, 1998; Peltonen et al., 2001; Peltonen et al., 2002; Eronen et al., 2003). We marked important concepts and data with red
color. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

different listening levels. It is practical for designing functional and

Table 2 aesthetically stimulating acoustic environments (Droumeva, 2005).
ASC approaches evaluated on self-create databases in problem raise phase.
After that, Peltonen et al. (Peltonen et al., 2001) conducted a
Ref. Acc. Approaches Classes Samples Dataset listening test to study the human ability to recognize everyday sounds,
Duration
achieving an accuracy of 70.0 % for 25 scenes. It also mentioned the
(audio
duration) importance of prominently identified sound events for ASC. One year
later, they used KNN and GMM for ASC (17 scenes), achieving 68.4 %
(Sawhney 68.0 % Filter-Banks 5 100 3 h (15 s)
& Maes, based features +
and 63.4 %, respectively (Peltonen et al., 2002). They proposed the
1997) nearest-neighbor concept of CASR, focusing on recognizing contexts, or environments,
(Peltonen 70.0 % Human 25 34 34 min (1 instead of discrete sound events. For example, Couvreur et al. (Couvreur
et al., evaluation (19 min) et al., 1998) used Linear Prediction Cepstral Coefficients (LPCC) features
2001) subjects)
and discrete HMMs to recognize five types of environmental noise
(Peltonen 68.4 % Band-energy, 17 209 104.5 min
et al., flux, roll-off, (30 s) events. Clarkson classified seven contexts using spectral energies and
2002) centroid, and HMM (Clarkson & Pentland, 1999). El-Maleh et al. (El-Maleh,
ZCR + KNN Samouelian, & Kabal, 1999) classified five environmental noise classes
63.4 % MFCC + GMM using line spectral features and a Gaussian classifier. Acoustic features
used in different audio information retrieval tasks (Foote, 1999) were
In the field of perceptual studies, Schafer (Schafer, 1977) first pro­ also introduced into ASC. Casey used a front-end system to transform
posed soundscape in 1977 in the context of acoustic ecology and has log-spectral energies into a low-dimensional representation with
grown to connect with other related fields, such as community noise, singular-value decomposition and independent component analysis
acoustics, and psychoacoustics. Then he published the famous book on (ICA) (Casey, 2001). There are also many CASR-related studies (Pelto­
soundscape (Schafer, 1993) in 1993, where he defined the soundscape as nen et al., 2002), such as noise classification (Couvreur et al., 1998),
the totality of all sounds in the listener’s dynamic environment. These speech/music discrimination (Carey, Parris, & Lloyd-Thomas, 1999),
studies have made a significant contribution to the study of environ­ sound source classification (Martin, 1999), content-based audio classi­
mental sound in the framework of acoustic ecology. In the same year, fication (Zhang & Kuo, 2001b), and classification of general audio (Li
Gaver (Gaver, 1993) attempted to construct a new framework for et al., 2001).
describing sound using audible sound source properties according to the Subsequently, Eronen et al. (Eronen et al., 2003) proposed an ASC
content of everyday sounds. algorithm using MFCC and GMM/HMM. Meanwhile, Gerhard (Gerhard,
On the other hand, Bergman (Bregman, 1990) proposed Auditory 2003) reviewed the history and current techniques of audio signal
Scene Analysis (ASA) in 1990 in his academic classic monograph classification in 2003. The author presented the background for under­
“auditory scene analysis”. It is a milestone piece of research on an standing the general research domain of audio signal classification (i.e.,
auditory cognitive mechanism and plays a critical guiding role in sound signal processing, spectral analysis, psychoacoustics, and auditory scene
processing. Based on ASA, CASA was first proposed by Brown in 1994 analysis), the basic elements of classification systems, and the state of
(Brown & Cooke, 1994), which referred to the computational analysis of the ASC research literature at that time.
an acoustic environment and the recognition of distinct sound events in
it. Then, Ellis (Ellis, 1996) proposed a prediction-driven CASA to alter­ 3.2.2. Primary exploration phase (2006–2015)
nate the data-driven approach (i.e., original CASA). The data-driven In this phase, many major ASC-related research topics and compe­
CASA uniquely converts the concrete input data to the more abstract titions emerged one after another following the publication of
output data using a set of operations. It has the shortcomings of “computational auditory scene analysis”. This phase is summarized in
neglecting sound context and cannot detect the presence of a sound Fig. 7. This phase started with the first study of comprehensively eval­
hidden by other components. In contrast, the prediction-driven CASA uating computers’ (accuracy for contexts/high-level classes: 58 % vs. 82
can analyze a complex sound mixture into a set of discrete components %) and humans’ (69 % vs. 88 %) perception of audio context (Eronen
(i.e., individual sound-producing events). et al., 2006). This study provides a baseline for evaluating an ASC sys­
Then, Sawhney et al. (Sawhney & Maes, 1997) first proposed ASC in tem. In the same year, the book “Computational Auditory Scene Anal­
1997, achieving an accuracy of 68.0 %. In the second year, they ysis: Principles, Algorithms, and Applications” (Wang & Brown, 2006)
(Clarkson, Sawhney, & Pentland, 1998) reported an ASA system that was published. Meanwhile, Dubois et al. (Dubois, Guastavino, &
classified sound objects with HMMs and detected scene changes by Raimbault, 2006) studied how individuals define their semantic cate­
clustering the space formed by the likelihoods of these sound object gory classification when they do not have prior knowledge.
HMMs. Next, Truax (Truax, 2001) treated listening as an active process In 2008, Tardieu et al. (Tardieu et al., 2008) demonstrated that
of interacting with the environment and distinguished it among several sound sources, human activities, and room effects (e.g., reverberation)

6
B. Ding et al.
Fig. 7. Summary of key developments in the primary exploration phase of ASC (Wang & Brown, 2006; Eronen et al., 2006; Stiefelhagen, Bowers, & Fiscus, 2007;
Temko et al., 2007; Stiefelhagen et al., 2007; Stiefelhagen et al., 2008; Tardieu et al., 2008; Waibel & Stiefelhagen, 2009; Giannoulis et al., 2013a; Salamon, Jacoby,
& Bello, 2014; Piczak, 2015b; Rakotomamonjy & Gasso, 2015; Barchiesi et al., 2015). We marked important concepts and data with red color. (For interpretation of
the references to color in this figure legend, the reader is referred to the web version of this article.)
are not only the factors driving the formation of acoustic scene cate­
gories but also the clues to scene classification. Next, Chu et al. (Chu,
Narayanan, & Kuo, 2009) mentioned that the classification accuracy fell
rapidly with the increasing number of scene classes because of
randomness, high variance, and other difficulties in processing envi­
ronmental sounds. It is supported by examples of accuracy limited to
around 92.0 % for 5 classes (Chu et al., 2006), 77.0 % for 11 classes
(Malkin & Waibel, 2005), and 60.0 % for 13 or more classes (Peltonen
et al., 2002; Eronen et al., 2006).
Various countries and regions have released many major research
plans about the auditory cognition abilities of machines related to the
ASC field. For example, Video Analysis and Content Extraction (VACE)
and Cognitive Assistant that Learns and Organizes (CALO) (Stiefelhagen,
Bowers, & Fiscus, 2007) were released in America. Computers in the
Human Interaction Loop (CHIL) (Waibel & Stiefelhagen, 2009) and
Augmented Multi-party Interaction (AMI) were proposed in Europe. The
major research plan on Cognitive Computing of Visual and Auditory
Information was released by the National Science Foundation of China
(NSFC, 2019).
Furthermore, some related competitions have also been held, such as
CLEAR (Temko et al., 2007; Stiefelhagen et al., 2007; Stiefelhagen et al.,
2008), DCASE (Giannoulis et al., 2013a), and TRECVID (audio-visual
multimodal event detection) (Lee et al., 2021) challenges. They have
attracted contributions to the audio domain. CLEAR evaluation is an
international effort in CHIL to evaluate systems for the multimodal
perception of people, their activities, and interactions (Stiefelhagen
et al., 2007; Stiefelhagen et al., 2008). The DCASE challenge (Giannoulis
et al., 2013a) was released by IEEE AASP in 2013 for the first time to
evaluate the existing environmental sound detection methods, as
comprehensively summarized in (Barchiesi et al., 2015).
Additionally, various methods and datasets have been proposed to
promote the development and application of ASC. For example, a noise-
robust classification method combined with ICA and Matching Pursuit
(MP) was proposed, achieving an accuracy of 76.0 % (Mogi & Kasai,
2013). Meanwhile, the EAR-IT project was proposed to realize street
traffic flow monitoring and indoor energy use (i.e., smart city and smart
home) based on household monitoring through smart audio technology
(European Commision, 2014). Also, open ASC datasets, such as Urban­
Sound8k (baseline accuracy: 68.0 %) (Piczak, 2015a; Salamon, Jacoby,
& Bello, 2014), ESC-50 (baseline/human accuracy: 44.3 % vs. 81.3 %),
ESC-10 (baseline/human accuracy: 72.7 % vs. 95.7 %) (Piczak, 2015b),
and LITIS (Rakotomamonjy & Gasso, 2015) datasets were released to
fairly evaluate ASC algorithms.
Following the previous studies, ASC has been developed to a certain
extent. ASC datasets and ASC-related theories have been gradually
presented. Nevertheless, audio scene analysis is still a new and under­
explored research topic, and there are still many problems to further
7
Expert Systems With Applications 238 (2024) 121902
study.
3.2.3. Early in-depth research phase (2016 to present)
In this phase, ASC research has made revolutionary progress, and its
accuracy has been significantly improved with the development of
signal processing and deep learning methods. The development of this
phase is shown in Fig. 8.
Since 2016, the DCASE challenge and workshop have been held
annually, and the results and publications have provided explicit com­
parisons among ASC methods. Systematical analyses for the results of
the D13 (Barchiesi et al., 2015; Stowell et al., 2015), D16 (Mesaros et al.,
2018), D17 (Mesaros, Heittola, & Virtanen, 2018a), and D21A (Martín-
Morató et al., 2021) competitions are also provided. They demonstrated
that the emergence of deep learning brought a breakthrough for ASC,
replacing traditional approaches (e.g., SVM and GMM) and achieving
higher classification accuracy.
Also, some overviews of ASC were proposed. For instance, Dandashi
and AlJaam (Dandashi & AlJaam, 2017) presented a survey of the
advanced methods and datasets for audio-based classification. Virtanen
et al. (Virtanen et al., 2018) published the book “Computational analysis
of sound scenes and events” in 2018. This book is often considered a
beginner-friendly textbook of ASC and SED, which provides founda­
tions, history, methods, applications, and further perspectives on ASC
and SED. Then, Abeßer systematically summarized ASC algorithms
based on deep learning (Abeßer, 2020). Lee et al. (Lee et al., 2021)
summarized the commonly-used CNN for ASC in literature. They also
used deep separable convolution to reduce model complexity. Martín-
Morató et al. (Martín-Morató et al., 2021) concluded that advanced
methods combine the best-performing methods with model compression
methods for low-complexity ASC tasks. Recently, Vij et al. (Vij et al.
2023) provided a short overview of ASC and SED studies. Singh, Sharma,
& Sur (Singh, Sharma, & Sur, 2023) provided a systematic review of pre-
processing and classification techniques for acoustic scenes.
Various ASC algorithms have been proposed, achieving higher ac­
curacy. For instance, the accuracy of ASC methods on the LITIS dataset is
improved from 97.1 % (Bisot et al., 2017a), 98.1 % (Zhang, Zhang, &
Wu, 2018), and 98.7 % (Phan et al., 2019), and finally to 98.9 % (Pham
et al., 2021) by observing Fig. 8. To further identify the state of pub­
lished algorithms, we list the state-of-the-art approaches on common
ASC datasets in Table 3, where the trend of accuracy changes over time
is clarified. By observing Table 3, the accuracy of an ASC approach is
affected by its techniques, such as data augmentation, features, classi­
fiers, and late fusion. Early methods were mainly based on MFCC fea­
tures and simple machine learning methods, such as random forest,
MLP, and SVM. Recently, features like LMBE, spectrogram, CQT, and log
Mel spectrogram (or their variants) were adopted with CNN-based
classifiers, such as ResNet, FCNN, and Xception. The ASC algorithms
B. Ding et al.
Fig. 8. Summary of key developments in the early in-depth research phase of ASC (Aytar, Vondrick, & Torralba, 2016; Mesaros, Heittola, & Virtanen, 2016b;
Dandashi & AlJaam, 2017; Mesaros et al., 2018; Mesaros, Heittola, & Virtanen, 2018b; Virtanen et al., 2018; Kroos et al., 2019; Chen et al., 2019; Suh et al., 2020;
Abeßer, 2020; Lee et al., 2021; Kim et al., 2021; Schmid et al., 2022; Schmid et al., 2023). We marked important concepts and data with red color. (For interpretation
of the references to color in this figure legend, the reader is referred to the web version of this article.)
combining feature learning with a simple classifier (e.g., MLP) also
perform well in some datasets (Arniriparian et al., 2018). Furthermore,
most high-performance approaches employ data augmentation and late
fusion methods, especially for processing complex datasets (e.g., D17 to
D23).
In addition, we also list reported performance on common ASC
datasets to intuitively show accuracy changes (see Fig. 9). Notably, we
do not report data on the D18C and D19C datasets (Wilkinghoff, 2021)
because of the limited space and their few published papers. From Fig. 9,
the ASC accuracy is significantly improved. Although the ASC accuracy
on early datasets has been significantly improved to over 95 %, the
accuracy on recent datasets is still low (less than 90 %), such as D20A,
D21A, D22, and D23 datasets recorded by multiple devices. This is also
due to the increase of task complexity (e.g., larger-scale data, multiple
devices, limited model complexity, and shorter segments). Compared
with the D20A task, D21A only added a low-complexity constraint,
slightly decreasing accuracy.
In summary, ASC research made great progress with significantly
improved accuracy. However, performance is still low for the increasing
task complexity of ASC, as shown in Fig. 9. It means that ASC technol­
ogies are still not ideal, which leaves a large margin to develop.
3.2.4. Summary
To intuitively show the conceptual relationships of ASC-related
fields, we have summarized nine relationships in Fig. 10. Relationship
1 has been mentioned in (Barchiesi et al., 2015). ASC includes percep­
tual studies (i.e., soundscape cognition) and computational research (i.
e., CASR). The authors further explained that CASR is a task associated
with the area of CASA and is particularly applied to the study of envi­
ronmental sounds (Barchiesi et al., 2015), which can infer relationship
2. Recently, ASC is considered CASR by default, which can be divided
into four groups: audio event-based, BOF-based, raw waveform-based,
and spectrogram-based methods (Ding et al., 2022), driving relation­
ship 3.
CASA is the computational study of ASA, which was motivated partly
by the demand for practical sound separation systems (Wang & Brown,
2006). It is obvious to obtain relationship 4. Moreover, Wang defined
CASA as the procedure of a machine turning the ambient sound signal
into a meaningful representation and listed its related areas as ASC, SED,
and source separation (Wang, 2018). Therefore, we concluded rela­
tionship 6. In addition, CASA is grouped into prediction-driven and data-
8
Expert Systems With Applications 238 (2024) 121902
driven methods in terms of implementation strategy (Ellis, 1996), cor­
responding to relationship 7. Naturally, Relationship 5 appears.
Different from the data-driven CASA, the prediction-driven CASA is
based on a set of discrete sound events in audio data. Therefore, the
data-driven CASA for acoustic environment analysis corresponds to
BOF-based, raw waveform-based, and spectrogram-based ASC methods
(relationships 8). The prediction-driven CASA for acoustic environment
analysis corresponds to the audio event-based ASC method (relation­
ships 9).
3.3. Problems
Although the ASC field has made great progress, there are also some
problems that need to be addressed. We analyze them from the perspectives of
acoustic scenes, datasets, classification performance, and deployment in the
real world.
1) Unclear definition of acoustic scenes. ASC requires a clear definition
of acoustic scenes, as it needs predefined labels. However, it is hard
to define a generalized boundary for acoustic scenes. There are two
main reasons: (i) The acoustic signal is complex, multi-source,
overlapping, and unstructured, (ii) The acoustic scene definition
and its category granularity depend on application scenarios.
Because the distribution of sound source components varies from
application-to-application scenarios. For example, human activities
and traffic are very intensive in urban environments, distinct from
rural environments.
2) Lack of large-scale comprehensive datasets. As a complex multi-
classification task, a high-performance ASC system requires a
large-scale dataset for model training. However, the size and di­
versity of the available data are currently limited. Large-scale
comprehensive datasets are lacking for developing a specific ASC
system, despite many existing ASC datasets.
3) Low classification performance. Compared with other audio fields,
the ASC’s performance is still deficient. There are mainly three rea­
sons: (i) The high diversity among acoustic features of the same
scenes and the similarity among acoustic features of different scenes
make it difficult to extract efficient features for ASC, (ii) Audio sig­
nals lack structural information, making modeling difficult
compared with speech signals, and (iii) Mismatched data distribution
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

Table 3
The state-of-the-art approaches on common ASC datasets. ‘spec.’ presents spectrogram. ‘top system’ is the top ranking system in the top system submitted to the DCASE
challenge. ‘DA’ (resp. ‘LF’) point whether data augmentation (resp. late fusion) is used for the ASC algorithm.
Datasets Acc. Years DA Algorithm LF Ref.

D13 71.0 % 2013 MFCCs, classified with a bag-of-frames approach (top system) (Stowell et al., 2015)
88.0 % 2016 SoundNet embedding + SVM (Aytar, Vondrick, & Torralba, 2016)
97.0 % 2019 L3-Net M256 + MLP (Cramer et al., 2019)
D16 89.7 % 2016 (spec. + CNN) ⊕ (MFCC + Binaural I-vectors) (top system) (Eghbal-Zadeh et al., 2016)
93.0 % 2018 layer-wise SoundNet embedding √ (Singh et al., 2018)
97.4 % 2019 1D-LTP, MFCC + SVM (Aziz et al.,2019)
D17 87.1 % 2017 √ LMBE, spec. + MLP,RNN,CNN,SVM (top system) √ (Mun et al., 2017a)
91.7 % 2017 √ Log Mel spectrogram + CNN √ (Han et al., 2017)
91.1 % 2018 Feature learning by DCGAN ⊕ S2SAE + MLP √ (Arniriparian et al., 2018)
95.0 % 2022 Wavelet Scattering Transform (WST) features + RSC √ (Hajihashemi et al., 2022)
D18A 76.9 % 2018 √ LMBE + CNN (top system) √ (Sakashita & Aono, 2018)
79.8 % 2018 LMBE + Xception √ (Yanget al., 2018)
90.0 % 2020 Raw waveform + multi-temporal CNN (e2e) (Kumar et al., 2020)
93.0 % 2023 Mel spec. and TCCs + Fisher layers + SVM (Fisher network) (Venkatesh, Mulimani, & Koolagudi, 2023)
D18B 63.6 % 2018 (LMBE, LMBE + NNF) + Multi-input CNN (top system) √ (Nguyen & Pernkopf, (2018)
77.3 % 2020 √ multi-scale semantic features + Xception (Yang et al., 2020)
91.0 % 2023 Mel spec. and TCCs + Fisher layers + SVM (Fisher network) (Venkatesh, Mulimani, & Koolagudi, 2023)
D19A 85.1 % 2019 √ LMBE, CQT + CNN √ (Chen et al., 2019).
91.0 % 2021 Log Mel spec. + CRNN-ELU (Mulimani & Koolagudi, 2021)
92.0 % 2023 Mel spec. and TCCs + Fisher layers + SVM (Fisher network) (Venkatesh, Mulimani, & Koolagudi, 2023)
D19B 70.0 % 2019 √ LMBE + CNN (top system) √ (Kosmider, 2019)
75.0 % 2020 √ Log Mel spec. + CNN (Kosmider, 2020)
91.0 % 2023 Mel spec. and TCCs + Fisher layers + SVM (Fisher network) (Venkatesh, Mulimani, & Koolagudi, 2023)
D20A 74.2 % 2020 √ LMBE, Δ, ΔΔ + ResNet, Snapshot (top system) (Suh et al., 2020)
81.9 % 2020 √ LMBE, Δ, ΔΔ + ResNet; FCNN; fsFCNN (2-stage Ensemble) √ (Hu et al., 2021).
88.0 % 2023 √ multi-scale Mel spec. (ms2) + RQNet (Madhu & Suresh, 2023)
D20B 97.3 % 2020 √ Perceptually-weighted LMBE + RF-regularized CNNs (top system) (Koutini et al., 2020)
D21A 75.9 % 2021 √ LMBE + CNNBC-ResNet (top system) √ (Kim et al., 2021)
79.4 % 2021 √ LMFB + TSL using a large two-stage ASC system pruned LTH √ (Yang et al., 2021a)
88.0 % 2023 √ multi-scale Mel spec. (ms2) + RQNet (Madhu & Suresh, 2023)
D21B 95.1 % 2021 √ LMBE, CQTbark + CNN, EfficientNet, Swin Transformer (top system) √ (M. Wang, Chen, Xie, Chen, Liu, & Zhang, 2021)
D22 58.0 % 2022 √ LMBE + RF-regularized CNNs, PaSST transformer (top system) √ (Schmid et al., 2022)
D23 58.4 % 2023 √ LMBE + RF-regularized CNNs, PaSST transformer (top system) √ (Schmid et al., 2023)
ESC-10 95.7 % 2015 Human baseline (Piczak, 2015b)
72.7 % 2015 MFCC & ZCR + random forest (Computer baseline) (Piczak, 2015b)
92.2 % 2016 SoundNet embedding + SVM (Aytar, Vondrick, & Torralba, 2016)
95.8 % 2020 √ Log Mel spec. + TS-CNN10 (Wang et al., 2020)
97.0 % 2021 Log spec. + ESResNet (Guzhov et al., 2021)
ESC-50 81.3 % 2015 Human baseline (Piczak, 2015b)
44.3 % 2015 MFCC & ZCR + random forest (Computer baseline) (Piczak, 2015b)
74.2 % 2016 SoundNet embedding + SVM (Aytar, Vondrick, & Torralba, 2016)
94.0 % 2017 spectrum + mixture model (Baelde, Biernacki, & Greff R, 2017)
94.7 % 2020 √ Log Mel spec. + fine-tune PANN (Kong et al., 2020)
95.7 % 2021 √ Log Mel spec. + AST-P √ (Gong, Chung, & Glass, 2021)
LITIS 97.1 % 2017 DNN embedding + LR (Bisot et al., 2017a)
98.1 % 2018 CNN with temporal transformer module (TT-CNN) (Zhang et al., 2018)
98.7 % 2019 √ Log Mel spec. + CRNN with spatio-temporal attention pooling √ (Phan et al., 2019)
98.9 % 2021 √ Multi-spec. + encoder-decoder (Pham et al., 2021)
UrbanSound8k 68.0 % 2014 MFCC + random forest (baseline) (Salamon, Jacoby, & Bello, 2014)
73.6 % 2015 Spherical k-means embedding from log Mel spec. + random forest (Salamon & Bello, 2015)
88.0 % 2017 TEO-GTCC ⊕ GTCC √ (Agrawal et al., 2017)
88.5 % 2020 √ Log Mel spec. + TS-CNN10 (Wang et al., 2020)
96.7 % 2021 pre-trained VGGlish (Tsalera, Papadakis, & Samarakou, 2021)
99.0 % 2022 √ Log Mel spec. + deep CNN (Madhu, 2022)
MSOS 93.2 % 2019 Multi-view SoundNet embedding + MLP (Singh et al., 2019)
96.0 % 2020 √ Log Mel spec. + fine-tune PANN (Kong et al., 2020)

between source and target data, such as multiple devices, might
cause low performance.
4) Computational efficiency and practicability. At present, ASC
models with high performance are relatively complex. There are
problems with computational efficiency optimization and algorithm
adaptability when deploying ASC algorithms in the real world. For
example, in the reality of embedded industrial applications, an ASC
system might be deployed in conditions of limited computational
power and memory capacity (e.g., IoT embedded devices), where the
high performance of the ASC system is also required. Therefore, the
computational efficiency of the ASC system needs to be considered to
judge whether it is practical for this application.
In addition, knowledge of scene type can be helpful to boost the
performance of basic perceptual algorithms, for instance, by providing
appropriate preprocessing or context-dependent grammar for speech
recognition modules (Rakotomamonjy, 2017). Therefore, ASC can be
used as a first step for a device to automatically understand its envi­
ronment, followed by a downstream task (e.g., speech enhancement),
instead of the end goal of a system.
4. Data processing
The goal of data processing is to process the input data to enhance its
quality or increasing its quantity, ultimately improving the performance
of the back-end classification. Frequently-used data processing methods
for ASC include channel selection and data augmentation. It is worth
noting that the data augmentation method can be combined with

9
B. Ding et al.
Fig. 9. Accuracies on commonly-used ASC datasets in state-of-the-art papers. The blue data dots present the accuracy of the baseline system. The green data dots
indicate the accuracy of the top system submitted to the corresponding DCASE challenge or the highest accuracies on other datasets. The highest accuracies on
DCACE ASC datasets are marked by the red font. Here, DCASE ASC datasets are the development datasets for DCASE ASC tasks. The oval with the same color presents
that the datasets contain the same audio data but for different DCASE challenge ASC task (e.g., D20A and D21A). (For interpretation of the references to color in this
figure legend, the reader is referred to the web version of this article.)
Fig. 10. Conceptual relationships of ASC-related fields.
channel selection. There exist many works combining data augmenta­
tion and channel selection according to the technical reports of DCASE
challenges ASC tasks. For instance, Chen et al. (Chen et al., 2019) uti­
lized average-difference channels (average and difference of binaural
channels) along with a GAN-based data augmentation method for ASC,
winning the DCASE2019 ASC competition. We will detail these two
techniques and discuss their application in ASC.
4.1. Channel selection
Generally, the audio input of ASC is monaural (Abeßer, 2020).
However, multichannel and multimodal data settings represent oppor­
tunities to address complex real-world scene and event classification
problems more effectively. Potentially complementary data streams can
lead to more robust analysis by effectively combining them using
appropriate techniques at the input representation, feature, or decision
level (Virtanen et al., 2018). Such techniques have been successfully
realized in various multichannel audio/audio-visual scene analysis
tasks.
The fusion of multichannel information focuses on merging the in­
formation from different channels to maximize the amount of informa­
tion and better model acoustic scenes. For example, Zieliński (Zieliński,
2018) proposed three information fusions of five-channel surround
sound (i.e., early fusion, late fusion variant 1/2) for ASC. In addition,
separation algorithms are employed to generate additional signal
channels/variants, such as HPSS (Ono et al., 2008), NMF-based HPSS
(Park, Shin, & Lee, 2017), and NNF (Nguyen & Pernkopf, 2018) algo­
rithms. Furthermore, multichannel audio was applied to extract spatial
information to supplement spectral features for ASC (Imoto & Ono,
10
Expert Systems With Applications 238 (2024) 121902
2017; Abeßer, 2020; Kawamura et al., 2023).
Undeniably, multichannel fusion is effective in boosting ASC per­
formance in some cases (Ding et al., 2022; Eghbal-zadeh et al., 2017;
Zhang, Han, & Shi, 2020; Imoto, 2021). However, Waldekar et al.
(Waldekar & Saha, 2020a) mentioned that it might be better to average
binaural audio into mono. Abeßer mentioned that current state-of-the-
art ASC algorithms process either mono or stereo signals without a
clear trend (Abeßer, 2020). We account that the effectiveness of multi­
channel information fusion is affected by task characteristics, audio
features, classification, and fusion strategy. Therefore, more research on
channel selection and fusion in ASC is necessary.
4.2. Data augmentation
Although deep learning has demonstrated remarkable performance
on many complex tasks, it suffers from the requirement for large
amounts of data (Summers & Dinneen, 2019). Considering ASC, the
relative scarcity of labeled data and the uncertainty of the acoustic scene
signal result in different statistical distributions of the training and
testing data (Salamon & Bello, 2017). Therefore, deep learning models
trained directly on training data may perform poorly on generalization
(Yang et al., 2020). This problem can be overcome by data augmentation
(Abeßer, 2020). Data augmentation usually generates new training data
through deformations and transformations on existing training re­
cordings (Cui, Goel, & Kingsbury, 2015; Cao et al., 2019). It does not
always directly process the original data (e.g., audio signals) but also
processes features extracted from the original data, such as the
spectrum.
Generally, data augmentation can be presented by (1), which con­
siders arbitrary functions f mapping one or two basic samples into a
single new training data. If f is a linear function, the data augmentation
method is considered linearity-based, otherwise, it is non-linearity-
based (Summers & Dinneen, 2019). Label-preserving data augmenta­
tion generates new training data based on basic samples while preser­
ving their original label.
(̃x, ̃y) = f ({(xi , yi )}Ni=1 ) 1⩽N < 3 (1)
where (xi , yi ) and (̃
x, ̃
y) are the sample-label vector of the ith basic
sample and new sample, respectively. We did not find improvement
beyond N = 2 from (Summers & Dinneen, 2019), so N is less than 3 here.
We summarize commonly-used data augmentation in ASC.
B. Ding et al.
1) Mixing audio (mixing) randomly mixes two audios in training
data from the same scene class (Hu et al., 2020).
2) Pitch shifting (pitch) randomly shifts the pitch of training data in
the uniform distribution (Hu et al., 2020; Salamon & Bello,
2017).
3) Time stretching (time) randomly changes the speed or the dura­
tion of an audio signal in uniform distribution without affecting
the audio pitch (Salamon & Bello, 2017). It is also called speed
change.
4) Adding random noise (noise) adds Gaussian noise for each
training data randomly (Hu et al., 2020).
5) Mixup mixes two samples in the training data from different
scene classes with a random weight (Hu et al., 2020). It is
implemented by ̃ x = λx1 +(1 − λ)x2 and ̃ y = λy1 + (1 − λ)y2 .
Here, (x1 , y2 ) and (x2 , y2 ) are two sample-label vectors drawn at
random from the training data. (̃ x, ̃
y) is the new sample-label
vectors after mixup, and λ ∈ [0, 1]. λ is obtained by the sam­
pling of the beta distribution with parameter α, and α ∈ [0, ∞]
(Zhang et al., 2018). Hu et al. (Hu et al., 2020) use mixup with α
equal to 0.4.
6) Dynamic Range Compression (DRC) compresses the dynamic
range of the sample (Salamon & Bello, 2017). It maps the natural
dynamic range of a signal to a smaller range. For instance, Hu
et al. (Hu et al., 2020) generated data of simulated devices by
adding reverberation followed by DRC to audio collected with a
real device.
7) Spectrum augment (SpecAugment) applies time and frequency
masking on spectrogram of the signal (Hu et al., 2020). It is
proposed for the automatic speech recognition (Park et al., 2019).
8) Spectral rolling (SpecRoll) randomly shifts spectrogram excerpts
of training data over time (Koutini, Eghbal-zadeh & Widmer,
2019).
9) Random erasing (RandErasing) replaces random boxes in feature
representations of training data with random numbers (Zhong
et al., 2020). It has been successfully applied in image classifi­
cation, object detection, and person re-identification.
10) Between-Class (BC) mixes two samples in the training data from
different scene classes with a mixing ratio obtained by training a
model using KL loss (Tokozume, Ushiku & Harada, 2018). It is
implemented by (2) and (3). The experiment on ESC-50 verified
its effectiveness.
px1 + (1 − p)x2 1
x = √̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅
̃ where p =
1− λ (2)
G1− G2
p2 + (1 − p)2 1 + 10 20 ⋅
λ
y = λy1 + (1 − λ)y2
̃ (3)
where (x1 , y2 ) and (x2 , y2 ) are two sample-label vectors drawn at
random from the training data. (̃ x, ̃
y) is the new sample-label vectors
after using BC method. G1 and G2 is the sound pressure level of x1 and x2
[dB], and λ ∼ U[0, 1].
11) Channel confusion (ChanConfusion) randomly swaps two
channels in input data (Hu et al., 2020).
12) Spectrum correction (SpecCorrect) computes the correction co­
efficients from the spectrum of n aligned pairs of recordings and
then transforms all the recordings using these coefficients. The
corrected spectra should look alike for all the devices (Kosmider,
2019). Here, n can be quite small (e.g., 30) (Nguyen & Pernkopf,
2019a; Kosmider, 2019).
13) Combination generates new training data using several data
augmentation techniques. For example, Hu et al. (Hu et al., 2020)
combined more than five data augmentation methods for ASC.
Lasseck et al. applied the combined local time stretching and
frequency stretching for acoustic bird detection (Lasseck, 2018).
11
Expert Systems With Applications 238 (2024) 121902
14) GAN, a special reinforcement learning, augments training data
using GAN (Goodfellow et al., 2014). Generally, GAN is per­
formed in the audio signal domain (Chen et al., 2019) or feature
domain (Mun et al., 2017a). Moreover, some GAN variants have
been applied for data augmentation, such as Conditional-GAN
(Yang et al., 2019), EnvGAN (Madhu, 2022), ACGAN (Chen
et al., 2021), CycleGAN (Kacprzak & Kowalczyk, 2021), and
DCGAN (Bahmei, Birmingham, & Arzanpour, 2022). However,
GAN has the drawback of being hard to train and tune.
15) Label smoothing spatial-mixup (ls-spatial-mixup) randomly se­
lects and concatenates parts of Log-Mel spectrograms of two
samples with mixup (Yang et al., 2020). Then, the label of the new
data is smoothed by (4) and (5). The smoothed label aims to
overcome the over-confident problem of network training and
generate meaningful virtual data using the spatial-mixup method.
β
y = (1 − β)(λy1 + (1 − λ)y2 ) + 1N−
̃ 1 (4)
N
{ βmin y1 = y2
β= (5)
− 4(βmax − βmin )(λ − 0.5)2 + βmax y1 ∕
= y2
Where β ∈ [0, 1] is a smoothing parameter, 1N− 1 in (6) is an N
dimensional all-one vector. Here, λ is generated from Beta distribution
(same as mixup). ‘+’ donates add operation.
16) LDA-based data augmentation (LDA-augment) generates new
data according to the probability distributions related to the key
audio events of each acoustic scene detected by a topic model
LDA (Latent Dirichlet Allocation). It could simulate acoustic
scenes in real environments more effectively (Leng et al., 2020),
which is more stable than pitch, time, and noise.
In addition, other methods that cannot generate new data are also
classified as data augmentation. In this paper, we only consider these
methods as data processing methods to enhance data quality and thus
improve performance. For instance, random cropping randomly crops
input data (i. e., feature map) into a fixed length along the time axis
during the training stage (Hu et al., 2020). It was combined with a mixup
in the temporal axis in (McDonnell & Gao, 2020).
To intuitively compare data augmentation methods in ASC, we list
their characteristics in Table 4 in terms of data type, basic sample,
linearity-based, label-preserving, learning-based, and complexity.
Apparently, the characteristics of the combination method are not
certain. GAN and LDA-augment methods augment data without
requiring basic samples. Only mixing, mixup, BC, and ls-spatial-mixup
methods are based on two basic samples. Others are based on only
one sample.
In conclusion, data augmentation methods provide an effective so­
lution to meet the requirements of deep learning methods for a large
amount of labeled data. Various data augmentation methods have been
verified in specific datasets. However, the practical implementation of
these methods to achieve good performance for various ASC tasks is still
an area of active research. For example, Hu et al. (Hu et al., 2021)
compared popular data augmentation methods to validate the effec­
tiveness of their combination. Han et al. (Han et al., 2023) also
compared a few data augmentation methods to validate their proposed
randmasking augment method on the D18A and D19A datasets. How­
ever, their effect might be uncertain when using them in other ASC al­
gorithms or datasets.
Furthermore, overfitting happens frequently when a large amount of
augmented data is used (Xie et al., 2021). Therefore, not all augmen­
tation methods are effective. Those samples augmented far from their
original ones will jeopardize the classification performance (Zheng, Mo,
& Zhao, 2022). Therefore, appropriate data augmentation methods or
sample filters (Zhang et al., 2018) are required for ASC to ensure
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

Table 4
The characteristics of data enhancement methods for ASC. The data type of audio/spectrum corresposnds augmentation occurring before/ after feature extraction.
Data Data type Basic Linearity- Label- Learning- Complexity Ref.
augmentation sample based preserving based

mixing Audio/ 2 √ √ × simple (Hu et al., 2020)

spectrum
pitch Audio 1 × √ × simple (Hu et al., 2020; Salamon & Bello, 2017)
time Audio 1 × √ × simple (Hu et al., 2020; Salamon & Bello, 2017)
noise Audio 1 √ √ × simple (Hu et al., 2020)
mixup Audio/ 2 √ × × simple (Hu et al., 2020; Salamon & Bello, 2017; Zhang et al.,
spectrum 2018)
DRC Audio 1 √ √ × simple (Hu et al., 2020; Salamon & Bello, 2017)
SpecAugment Spectrum 1 × √ × simple (Hu et al., 2020; Park et al., 2019)
SpecRoll Spectrum 1 × √ × simple (Koutini et al., 2019)
RandErasing Spectrum 1 × √ × simple (Zhong et al., 2020)
BC Audio 2 √ × √ complex (Tokozume et al., 2018)
ChanConfusion Audio 1 × √ × simple (Hu et al., 2020)
SpecCorrect Spectrum 1 × √ × complex (Nguyen, Pernkopf, & Kosmider, 2020; Kosmider, 2019;
Kosmider, 2020)
Combination Audio/ – – – – – (Hu et al., 2020; Lasseck, 2018; Wang et al., 2023)
spectrum
GAN Audio/ 0 × – √ complex (Goodfellow et al., 2014; Mun et al., 2017a)
spectrum
Ls-spatial-mixup Spectrum 2 × × × complex (Yang et al., 2020)
LDA-augment Audio 0 – – √ complex (Leng et al., 2020)
mixing Audio/ 2 √ √ × simple (Hu et al., 2020; McDonnell & Gao, 2020)
spectrum

augmentation quality. Also, data augmentation methods have the
drawback of consuming large storage spaces and long training times.
Moreover, their parameters require careful tuning to avoid excessive
deformations that take away features critical for acquiring
generalization.
Most of the augmentation methods mentioned above are offline
because their learning policies are isolated from their usage. Recently,
online data augmentation (OnlineAugment) was proposed, which
jointly optimizes data augmentation and target network training in an
online manner (Tang et al., 2020). Contrasted with traditional offline
methods, it is more efficient and adaptive as it requires less domain
knowledge and is easily applicable to new tasks. Also, it can be applied
together with offline methods.
5. Feature acquisition
Feature acquisition refers to the process of extracting features from
signals. It is crucial for the entire classification process since a set of
significant and compact features will make correct classification much
easier.
5.1. Pre-processing
Pre-processing is applied before extracting features to enhance
certain characteristics of incoming signals and maximize analysis per­
formance in the later phases of the analysis system. It is achieved by
reducing the effects of noise or emphasizing the target sounds in the
signal (Virtanen et al., 2018), mainly implemented through noise
reduction, framing, and windowing, and time–frequency representation.
1) Noise reduction. Both ASC and SED face the challenge that fore­
ground sound events in acoustic scenes are often overshadowed by
background noises (Abeßer, 2020). Noise reduction can be used to
reduce environmental noise interference in audio analysis (Schröder
et al., 2013). Several noise reduction methods for ASC have been
proposed to remove irrelevant noise and enhance target-related in­
formation. For instance, Lostanlen et al. (Lostanlen et al., 2018) used
per-channel energy normalization (PCEN) (Wang et al., 2017) to
reduce stationary noise and enhance transient sound events in
environmental audio. Han et al. (Han, Park, & Lee, 2017) used
background subtraction and median filtering over time to eliminate
irrelevant noise from the environment and the recording device.
2) Framing and windowing. Audio signals are generally non-stationary
since the signal statistics change rapidly over time. Therefore, the
short-time processing approach is utilized to capture the signal in a
quasistationary state. It analyzes audio signals periodically in short-
time segments referred to as analysis frames, which are transformed
into a spectrum for further feature extraction (Virtanen et al., 2018).
Analysis frames are obtained by framing, which slices the audio
signal into fixed-length frames by sliding a window, and windowing,
which smooths these frames by multiplying them with a window
function.
3) Time-frequency representation. The audio signal is generally first
transformed to a time–frequency representation to capture frequency
information, such as Short-Time Fourier Transform (STFT) spectro­
gram, Mel spectrogram, log Mel spectrogram, CQT spectrogram, and
wavelet spectrogram. The Mel spectrogram is obtained by processing
the STFT spectrogram with a Mel-scaled filter bank, which provides a
more compact spectral representation of sounds than the STFT
spectrogram (Abeßer, 2020). The log Mel spectrogram is obtained by
both Mel filtering and logarithmic scaling. The wavelet spectrogram
is the time–frequency representation of a signal by wavelet trans­
formation (Chen et al., 2021). Generally, STFT is mainly used to
analyze linear and stationary signals. Mel spectrograms and log Mel
spectrograms are commonly used for ASC because both use a non-
linear frequency scale motivated by human auditory perception.
CQT provides a higher frequency resolution for lower frequencies
and a higher temporal resolution for higher frequencies. Wavelet
analysis is suitable for analyzing non-stationary signals (Xie et al.,
2021), which can assist other time–frequency representations when
analyzing complex audio signals.
5.2. Hand-crafted feature extraction
Hand-crafted features are obtained by processing audio signals using
fixed pre-defined signal transformations. They incorporate a priori
knowledge of acoustics, sound perception, or specific properties into an
audio scene. Therefore, hand-crafted features are usually interpretable
and often bring an interesting insight into the content and behavior of
sound scenes (Virtanen et al., 2018). Moreover, it generally obtains a
compact data representation for efficient sound scene analysis

12
B. Ding et al.
approaches.
At present, the most frequently-used hand-crafted features for ASC
are mainly inspired by speech, music, or image processing, including
time domain, frequency domain, cepstral, and spectrogram image-based
features listed in Table 5.
1) Time domain features include envelope, Zero-Crossing Rate (ZCR),
short-time average energy, temporal waveform moments, and
autocorrelation coefficients (Virtanen et al., 2018; Peltonen et al.,
2002; Chu, Narayanan, & Kuo, 2009; Zhang & Kuo, 2001a). They are
easily computed and usually used with other refined features (Cha­
chada & Kuo, 2014).
2) Frequency domain features mainly include LMBE (Virtanen et al.,
2018), band energy, band energy ratio, bandwidth, spectral
centroid, spectral flatness, spectral roll-off point, spectral flux (Pel­
tonen et al., 2002; Eronen et al., 2006; Eronen et al., 2006), and
spectral kurtosis (Sigtia et al., 2016; Dwyer, 1983). LMBE is the most
popular feature and has obtained good results in DCASE ASC tasks. It
is powerful enough to be used on its own as input for classification or
feature learning (Virtanen et al., 2018). LMBE provided a compact
and smooth representation of the local spectrum but neglected
temporal changes in the spectrum over time. Therefore, the approach
of delta features and stacking feature vectors of neighboring frames
is employed to capture the temporal aspect of the features (Virtanen
et al., 2018). For example, their first and second derivatives (Δ and Δ
Δ) were used in ASC (Ding et al., 2022).
3) Cepstral features consist of MFCC (Chu, Narayanan, & Kuo, 2009;
Zhao et al., 2011; Tsau, Kim, & Kuo, 2011; Agrawal et al., 2017),
Code Excited Linear Prediction (CELP) (Tsau, Kim, & Kuo, 2011),
Constant-Q Cepstral Coefficients (CQCC) (Virtanen et al., 2018),
Gammatone Feature Cepstral Coefficients (GFCC) (Agrawal et al.,
2017; Huang et al., 2020), and Recurrence Quantification Analysis
(RQA) features (Roma, Nogueira, & Herrera, 2013; Battaglino et al.,
2015). They are used to estimate the rough shape of the spectrum of a
signal. MFCC features are the most popular features, and their first
and second derivatives are also applied. However, the performance
of MFCC for unstructured audio (e.g., environmental sounds (Tsau,
Kim, & Kuo, 2011)) is limited. Therefore, MFCC features are often
Table 5
Hand-crafted features applied for ASC.
Features Type Features
Time domain Envelope (Virtanen et al., 2018)
ZCR (Zhang & Kuo, 2001a; Peltonen et al., 2002; Chu,
Narayanan, & Kuo, 2009)
Short-time average energy (Zhang et al., 2023; Peltonen
et al., 2002; Chu, Narayanan, & Kuo, 2009)
Autocorrelation coefficients (Virtanen et al., 2018)
Frequency domain LMBE, ΔLMBE, and ΔΔLMBE (Virtanen et al., 2018; Ding
et al., 2022)
Band energy, band energy ratio, bandwidth, spectral
centroid, spectral flatness, spectral roll-off point, spectral
flux, spectral kurtosis (Peltonen et al., 2002; Eronen et al.,
2006; Chu, Narayanan, & Kuo, 2009)
Spectrogram image- HOG (Rakotomamonjy & Gasso, 2015; Virtanen et al., 2018)
based SPD / SPD + HOG (Dennis, Tran, & Chng, 2013; Bisot, Essid,
& Richard, 2015; Virtanen et al., 2018)
LBP (Battaglino et al., 2015; Abidin, Togneri, & Sohel, 2017;
Virtanen et al., 2018; Xie & Zhu, 2019)
AMFB (Schröder et al., 2017)
GFB (Schröder, Goetze, & Anemüller, 2015; Schröder et al.,
2017)
Cepstral features CELP (Tsau, Kim, & Kuo, 2011)
MFCC, ΔMFCC, and ΔΔMFCC (Chu, Narayanan, & Kuo,
2009; Zhao et al., 2011; Tsau, Kim, & Kuo, 2011; Agrawal
et al., 2017)
GFCC (Agrawal et al., 2017; Huang et al., 2020)
CQCC (Virtanen et al., 2018)
RQA (Roma, Nogueira, & Herrera, 2013)
13
Expert Systems With Applications 238 (2024) 121902
combined with other techniques. For example, time–frequency fea­
tures obtained by the MP algorithm (Chu, Narayanan, & Kuo, 2009)
were used to supplement MFCC features. MiMFCC features (Eghbal-
zadeh et al., 2017), RQA + MFCC (Giannoulis et al., 2013a), CELP +
MFCC (Tsau, Kim, & Kuo, 2011), and MFCC + GFCC (Huang et al.,
2020) are also applied.
4) Spectrogram image-based features contain HOG (Rakotomamonjy &
Gasso, 2015), Subband power distribution (SPD) (Dennis, Tran, &
Chng, 2013), Local binary pattern (LBP) (Virtanen et al., 2018;
Battaglino et al., 2015; Abidin, Togneri, & Sohel, 2017; Xie & Zhu,
2019), Amplitude modulation filterbank (AMFB) (Schröder et al.,
2017), and Gabor filterbank (GFB) (Schröder, Goetze, & Anemüller,
2015). They use computer vision technology to characterize the
shape, texture, and evolution of the time–frequency content of the
sound field. The spectrogram common image-based features in ASC
include HOG, SPD, and LBP. HOG aims to capture relevant time­
–frequency structures for characterizing sound scenes. The SPD
image can be used either as a feature (Dennis, Tran, & Chng, 2013) or
as an intermediate representation for extracting other image-based
features (Bisot, Essid, & Richard, 2015). LBP features can capture
the texture and geometrical properties of a scene’s spectrogram.
Usually, multiple features are used together in ASC, for instance,
MFCCs + LBP (Battaglino et al., 2015), LBP + HOG (Schröder,
Goetze, & Anemüller, 2015), and the combination (Xie & Zhu, 2019)
of MFCCs, LBP, HOG, moments, etc.
Several publications reviewed acoustic features for audio analysis.
For instance, Peltonen et al. (Peltonen et al., 2002) reviewed acoustic
features applied in ASC. Tsau et al. (Tsau, Kim, & Kuo, 2011) detailed a
summary of features for content-based audio retrieval. Geiger et al.
(Geiger, Schuller, & Rigoll, 2013) discussed the common features in the
early works of ASC. Chachada and Kuo (Chachada & Kuo, 2014) offered
a qualitative and elucidatory survey on early ASC development and
divided ASC methods into stationary and non-stationary. They consid­
ered features in (Mitrović, Zeppelzauer, & Breiteneder, 2010) as sta­
tionary features, including rough features, cepstral features, MPEG-7
features, and auto-regression features.
In addition, the MPEG-7 descriptor (Wang et al., 2006; Muhammad
et al., 2010) and perceptual features (Richard, Sundaram, & Narayanan,
2013; Virtanen et al., 2018) are also applied to ASC. Occasionally, novel
features are designed for ASC. For example, Song et al. (Song, Han, &
Deng, 2018) proposed auditory summary statistics as features for ASC
inspired by the neuroscience study about sound texture perception. Bai
et al. (Bai et al., 2019) used the Acoustic Segment Model (ASM) to give
finer segmentation and cover all acoustic scenes by excavating the
potentially similar units hidden in the auditory sense. Recently, Götz
et al. (Götz et al., 2023) presented an approach to extract low-
dimensional representations from reverberant speech signals in a
contrastive learning framework, and its effectiveness across the down­
stream tasks of acoustic parameter estimation and environment classi­
fication was demonstrated.
Also, combining a large variety of features is often used to improve
performance (Virtanen et al., 2018). However, hand-crafted feature
extraction requires expert knowledge. Moreover, it relies on predefined
transformations and neglects some particularities of the signals, such as
recording conditions and devices.
5.3. Feature learning
Feature learning obtains features by processing audio signals using
learnable signal transformations. It analyzes datasets to automatically
determine what transformations should be applied to convert input into
features (Virtanen et al., 2018). Different from hand-crafted feature
extraction, it requires a large amount of data from which to generalize
instead of domain knowledge. It is generally applied in the end-to-end
(e2e) system (Abrol & Sharma, 2020), which builds the scene model
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

directly from the raw signal or time–frequency representation by
learning both features and the classifier. Typical examples include Acl­
Net, AclSincNet (Huang et al., 2019), SoundNet (Aytar, Vondrick, &
Torralba, 2016), very deep CNN-based e2e system (Dai et al., 2017),
multi-temporal CNN-based e2e system (Kumar et al., 2020), as well as a
general-purpose e2e audio embedding generator (Lopez-Meyer et al.,
2021).
Moreover, feature learning is applied to interpret the signal trans­
formation as a learnable function, commonly considered the front-end
used for joint training with the classification back-end (Chen, Zhang,
& Yan, 2019). For instance, Kuncheva (Kuncheva, 2004) proposed su­
pervised extensions of NMF to adapt the decomposition to the task at
hand and learn better features. Li et al. (Y. Li, Liu, Wang, Zhang, & He,
2019) used deep embedding to capture the characteristics of various
acoustic scenes for acoustic scene clustering. Similarly, L3-Net (Cramer
et al., 2019; Varma, Padmanabhan, & Dileep, 2021), SoundNet (Aytar,
Vondrick, & Torralba, 2016; Singh et al., 2018; Singh, Rajan, & Bhavsar,
2019; Singh, Rajan, & Bhavsar, 2020; Singh, 2022), VGGish (Jansen
et al., 2017), CliqueNets (Nguyen & Pernkopf, 2019a), CNN (Heo, Jung,
Shim, & Yu, 2019; Olvera, Vincent, & Gasso, 2022; Zhang, Liang, &
Feng, 2022), VGG (Ren et al., 2017; Yao et al., 2019; Bahmei, Bir­
mingham, & Arzanpour, 2022), and other pre-trained embedding
models have also been used to learn features for ASC. The combination
of acoustic and visual embeddings (Chen, Wang, & Zhang, 2022), mul­
tiview embeddings (Devalraju & Rajan, 2022), and features learned by
different methods (Arniriparian et al., 2018; Zhang, Liang, & Ding,
2020) are also proposed for ASC. Recently, Jiang et al. (Jiang et al.,
2023) constructed an embedding space containing multi-level distance
based on the hierarchical relationship of similarity between acoustic
scene classes, aiming to learn advanced features that are less susceptible
to differences in device information from manual features that contain
device information.
In addition, unsupervised feature learning is applied to derive
semantically meaningful signal representations. Unsupervised feature
learning techniques based on K-means (Salamon & Bello, 2015) or
nonnegative matrix factorization (NMF) have been shown to be
competitive with the best hand-crafted features (Bisot et al., 2016; Bisot
et al., 2017a; Guo et al., 2017). Similarly, Lee et al. (Lee, Hyung, & Nam,
2013) also used a sparse feature learning algorithm to learn features for
ASC.
There are many successful cases of feature learning in ASC. We list
most of them in Table 6. From this table, feature learning methods work
well in ASC. However, the performance of the ASC system using feature
learning is still not at par with well-established ASC systems built using
hand-crafted features such as spectral features (Ding et al., 2022). This
conclusion can be drawn by comparing the accuracy of ASC systems
using feature learning in Table 6 with their corresponding highest
accuracy.

Table 6
Feature learning methods applied in ASC. Notably, the underlined data is better than the baseline accuracy and less (within 5%) than the highest accuracy.
Methods Acc. Dataset (Highest Methods Acc. Dataset
Acc.) (Highest Acc.)

(mixup) AemNet-DW (e2e) (Lopez-Meyer et al., 2021) 92.3 % ESC-50 Layer-wise SoundNet embedding + SVM (maximum 93.0 % D16 (97.4 %)
(95.7 %) likelihood) (Singh et al., 2018)
Pruned SoundNet embedding + 1D CNN (SVM ensemble) 92.1 % Pruned SoundNet embeddings + 1D CNN (SVM 89.9 %
(Singh, Rajan, & Bhavsar, 2020) ensemble) (Singh, Rajan, & Bhavsar, 2020)
(DCGAN, pitch, noise) VGG-19 embedding + CNN-RNN ( 91.5 % feature learning from CQT + Nonnegative TDL (Bisot 83.3 %
Bahmei, Birmingham, & Arzanpour, 2022) et al., 2017a)
VGG embedding (LS, AM-softmax) + VGG (Yao et al., 81.9 % (ConvNet + SNMF + HOG) + SVM (Rakotomamonjy, 80.9 %
2019) 2017)
L3-Net M256 + MLP (Cramer et al., 2019) 79.8 % Feature learning by DCGAN ⊕ S2SAE + MLP (optimal 91.1 % D17 (95.0 %)
weighted sum) (Arniriparian et al., 2018)
SoundNet embedding + SVM (Aytar, Vondrick, & 74.2 % (Feature learning; NMF + CQT) + (TNMF-AS + DNN-M) 90.1 %
Torralba, 2016) (average) (Bisot et al., 2017b)
SoundNet embedding + SVM (Aytar, Vondrick, & 92.2 % ESC-10 (97.0 %) Scalogram-DCNN with DCT-based temporal module 87.4 %
Torralba, 2016) (e2e) (Chen, Zhang, & Yan, 2019)
(DCGAN, pitch, noise) VGG-19 embedding + CNN-RNN ( 98.0 % UrbanSound8k L3-Net embeddings + SVM (Varma, Padmanabhan, & 84.7 %
Bahmei, Birmingham, & Arzanpour, 2022) (99.0 %) Dileep, 2021)
AemNet-DW (e2e) (Lopez-Meyer et al., 2021) 83.6 % VGG16 embedding (STFT + bump + morse) + GRNN 80.9 %
(Margin Sampling Value, MSV) (Ren et al., 2017)
L3-Net M256 + MLP (Cramer et al., 2019) 79.3 % Multi-view embeddings ((MvLDAN)) + SVM (Devalraju 77.1 %
& Rajan, 2022)
Spherical k-means embedding from logMel spec. + 73.6 % (SoundNet embeddings + PCA + ACDL) + non-linear 76.1 %
random forest (Salamon & Bello, 2015) SVM (late fusion) (Singh, 2022)
Raw waveform + CNN-18 (Dai et al., 2017) 71.8 % Raw waveform + multi-temporal CNN (e2e) (Kumar 90.0 % D18A(93.0)
et al., 2020)
Multi-view embeddings ((MvLDAN)) + SVM (Devalraju & 97.9 % LITIS (98.9 %) Deep CNN embedding + DNN (Pham et al., 2020) 78.0 %
Rajan, 2022)
DNN embedding + logistic regression (Bisot et al., 2017a) 97.1 % (CNN embedding ⊕ CNN-GRU embedding) + SVM 77.4 %
(average) (Heo et al., 2019)
NMF embedding from CQT spec. (Kernel PCA) + logistic 95.6 % 3 channels LMBE + end-to-end 3D CNN (SeNoT-Net-L2) 77.2 %
regression (Bisot et al., 2016) (Zhang, Han, & Shi, 2020)
Multi-view SoundNet embedding + MLP (Singh et al., 93.2 % MSOS (96.0 %) Multi-view embeddings (MvLDAN) + KNN (Devalraju & 75.6 %
2019) Rajan, 2022)
(mixup) AemNet-DW (e2e) (Lopez-Meyer et al., 2021) 91.6 % D13 (97.0 %) Deep CNN Embedding + DNN (Pham et al., 2020) 66.9 % D18B (91.0
SoundNet embedding + SVM (Aytar, Vondrick, & 88.0 % (mixup) CliqueNets embedding + CNN-MoE (Nguyen & 64.7 % %)
Torralba, 2016) Pernkopf, 2019a)
RBM embedding from Mel spec. + L2-SVM (Lee, Hyung, 72.0 % (domain adaptation) Feature learning by CNN-5 + 60.1 %
& Nam, 2013) softmax (Olvera, Vincent, & Gasso, 2022)
LMBE + end-to-end 3D CNN (SeNoT-Net-L2) (Zhang, 80.3 % D19A (92.0 %) (random crop, mixup) multi-level distance embedding 72.6 % D20A (88.0
Han, & Shi, 2020) learning (MEDL) + Trident ResNet (Jiang et al., 2023) %)
(mixup, SpecCorrect) Feature learning by CNN + CNN 70.1 % D19B (91.0 %) (image data augmentation) Acoustic embedding, visual 94.6 % D21B (95.1
(calibration by heated-up softmax) (Nguyen, Pernkopf, ImageNet embeddings + SoftMax (Chen, Wang, & %)
& Kosmider, 2020) Zhang, 2022)

14
B. Ding et al.
5.4. Features for recent deep learning models
According to recent publications, LMBE (Yang et al., 2020; Pased­
dula & Gangashetty, 2021; Hu et al., 2021), MFCC (Bai et al., 2020;
Huang et al., 2020), as well as their deltas and delta-deltas, are the most
powerful hand-crafted features for deep learning-based ASC systems. In
addition, STFT spectrograms, Mel spectrograms (Lee, Hyung, & Nam,
2013; Alamir, 2021; Shim et al., 2022), log Mel spectrograms (Kong
et al., 2020; Ren et al., 2019; Mulimani & Koolagudi, 2021; Mulimani &
Koolagudi, 2021), CQT spectrograms (Rakotomamonjy, 2017; Arnir­
iparian et al., 2018), and their combination (Ren et al., 2017) are often
employed directly for deep learning. Also, the wavelet spectrogram is
applied with other spectrograms together in some cases (Yang et al.,
2019; Chen, Zhang, & Yan, 2019; Wu & Lee, 2020). In addition, the raw
waveform is also used as the input for deep learning for building end-to-
end ASC systems (Dai et al., 2017; Kumar et al., 2020; Lopez-Meyer
et al., 2021). Occasionally, spectrogram image-based features like
SPD, LBP, and HOG are used as the input of deep learning models for
feature learning (Dennis, Tran, & Chng, 2013; Rakotomamonjy & Gasso,
2015; Bisot, Essid, & Richard, 2015).
5.5. Feature selection and dimensionality reduction
With the creation of large-scale databases and the consequent re­
quirements for good machine learning techniques, a large number of
features are extracted or learned to better represent the target concept.
However, not all features are significant since many could be either
redundant or irrelevant to the target concept. Thus, feature selection and
dimensionality reduction become inevitable to reduce feature dimen­
sionality and retain only the information that is relevant to discrimi­
nating the target classes (Virtanen et al., 2018).
Feature selection is the process of seeking for an appropriate feature
subset from a larger set of initial features that can describe an applica­
tion domain, and even improve classification performance. Therefore, it
can directly affect the complexity and accuracy of classifiers. Generally,
feature selection can be divided into filter and embedded approaches
according to whether the process takes place before or is integrated into
the classification process (Virtanen et al., 2018; Özseven and Arpacio­
glu, 2023).
In addition, dimensionality reduction techniques are applied to cope
with the potentially large dimensionality of the feature space. They are
generally implemented through transformation, such as Principal
Component Analysis (PCA), linear discriminant analysis (LDA), or the
recent bottleneck DNN (Virtanen et al., 2018). Moreover, dimension­
ality reduction by mapping features to a randomized low-dimensional
feature space was also employed.
6. Modeling
Modeling is performed on the features extracted from audio signals,
including supervised (i.e., conventional machine learning and deep
learning), unsupervised, semi-supervised, and self-supervised methods
(S. Li, Gu, Luo, Chambers, & Wang, 2019). In addition, result post-
processing is also applied to boost ASC performance. Therefore, we
summarize modeling methods with aspects of these six categories.
6.1. Conventional machine learning
Earlier ASC works relied on conventional machine learning methods,
such as KNN (Peltonen et al., 2002), GMM (Aucouturier, Defreville, &
Pachet, 2007; Chu, Narayanan, & Kuo, 2009), SVM (Geiger, Schuller, &
Rigoll, 2013), HMM (Eronen et al., 2006), random forest (Piczak,
2015a), and decision tree (Piczak, 2015a). They are usually combined
with hand-crafted features such as MFCCs to build the ASC model
(Geiger, Schuller, & Rigoll, 2013; Chachada & Kuo, 2014)]. The accu­
racy of these methods is vastly limited when they are used to process
15
Expert Systems With Applications 238 (2024) 121902
complex ASC tasks with a large-scale dataset, although they are gener­
ally low-complexity and train models quickly.
Recently, conventional machine learning methods have also been
combined with feature embedding and work well in ASC. For example,
SVM, Multilayer Perceptron (MLP), KNN, random forest, and logistic
regression are performed on high-level embedding extracted from audio
signals, as shown in Table 6.
6.2. Deep learning
In recent years, deep learning methods have been adopted to learn
high-level representations, achieving better performance than conven­
tional machine learning methods (Ren et al., 2020). However, CNN
consumes much longer training times and requires a large amount of
data. It can be solved by current excellent computing power, increasing
available datasets, and data processing techniques.
6.2.1. Deep learning methods
Deep learning methods, especially CNNs, have been continuously
introduced into ASC and have achieved excellent results. ASC systems
mostly use CNN-based network architectures since CNN usually pro­
vides a summary classification of longer acoustic scene excerpts
(Abeßer, 2020). Most state-of-the-art ASC algorithms employ CNNs,
according to the technical reports of DCASE.
Currently, deep learning methods applied in ASC include Recur­
rent Neural Network (RNN) (Li et al., 2017), TDNN (Schröder et al.,
2017), BLSTM (Y. Li, Li, Zhang, Wang, Liu, & Feng, 2018), FNN (Bisot
et al., 2017b; Takahashi et al., 2017), CNN (e.g., traditional CNN, VGG,
ResNet, Inception (Hershey et al., 2017), Xception, and MobileNetV2),
FCNN (Hu et al., 2020), CRNN (Mulimani & Koolagudi, 2021). Also,
some deep learning networks are designed for ASC-related tasks, such as
SoundNet (Aytar, Vondrick, & Torralba, 2016), SubSpectralNet (Phaye,
Benetos, & Wang, 2019), DCASENet (Jung et al., 2021), CAA-Net (Ren
et al., 2020), PANNs (Kong et al., 2020), Fisher network (Venkatesh,
Mulimani, & Koolagudi, 2023), and RQNet (Madhu & Suresh, 2023).
Recently, Hasan et al. (Hasan et al. 2023) proposed an automated ar­
chitecture design using a genetic algorithm for ASC, which can find
optimized CNN architectures for the ASC task. It employs frequency-
dimension splitting of the input spectrograms to explore the architec­
ture search space in sub-CNN models in addition to classical single-path
CNNs. The performance and specifications of these deep learning
methods for ASC are reported in Table 7. From this table, traditional
CNN and ResNet are the most popular architectures and work well in
ASC.
6.2.2. Attention mechanism
The attention mechanism highlights important information and en­
riches the representation. It can be used to estimate the contribution of
the feature vectors at each time step (Chorowski et al., 2015; Ren et al.,
2020), aiming to overcome the shortcoming that conventional global
pooling methods cannot estimate the contribution of each time­
–frequency bin in feature maps (Ren et al., 2018). The existing attention
methods can be divided into six categories: channel attention, spatial
attention, temporal attention, branch channel, channel & spatial atten­
tion, and spatial & temporal attention (Guo et al., 2022).
The recent literature about ASC has shown its effectiveness. For
example, Ren et al. (Ren et al., 2020; Zhang, Liang, & Feng, 2022) used a
time–frequency attention mechanism to analyze the contribution of
each time–frequency bin of the feature maps in the CNNs. Li et al. (Z. Li
et al., 2019) incorporated Gated Linear Units (GLU) in several steps of
the feature learning part of the network (multi-level attention). Wang
et al. (Wang, Santoso, & Wang, 2017) used self-determination CNNs
(SD-CNNs) to identify frames with higher uncertainty due to over­
lapping sound events. Wang et al. (Wang, Zou, Chong, & Wang, 2020)
proposed a novel parallel temporal-spectral attention mechanism for
CNN to enhance the temporal and spectral features by capturing the
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

Table 7
Comparison of different deep learning methods applied in ASC. Notably, Δ and ΔΔ represent deltas and delta-deltas features of a specific feature. For example, MFCC +
Δ + ΔΔ are the concatenated features of MFCC and their deltas and delta-deltas values. TL is transfer learning. Spc. is spectrogram. Acc. represents accuracy. AM-
softmax is the additive margin softmax loss. The bold data indicates the method achieves the highest accuracy, and the underlined data means that the difference
with the highest accuracy is no more than 5 %.
Classifier Specifications Acc. Datasets Classifier Specifications Acc. Datasets
(Highest Acc.) (Highest
Acc.)

DNN Smile6k features (Alamir, 2021; Eyben et al., 84.2 D16 (97.4 %) VGG Spec. features + VGG (Eghbal-Zadeh 89.9 % D16 (97.4
2010) % et al., 2016; Simonyan & Zisserman, %)
2015)
MFCC + Δ + ΔΔ (TL) (Mun et al., 2017b) 86.3 D16 (97.4 %) Feature learning using VGG, label 81.9 % ESC-50
% smoothing(LS), and AM-softmax (Yao (95.7 %)
et al., 2019)
Frame- and channel-concatenated MFCCs 88.2 D16 (97.4 %) ResNet (Mixup) transfer learning by pre- 74.7 % D18A
(DNN + GMM) (Takahashi et al., 2017) % trained ResNet mode (Ye et al., 2019; (93.0 %)
He et al., 2016)
Concatenate features from NMF features and 89.2 D17 (95.0 %) (Mixup) LMBE (late fusion) (Ding 87.6 % D19A
CQT representations (Bisot et al., 2017b) % et al., 2022) (92.0 %)
LPCC, SCMC, and LMBE features (Kong et al., 82.1 D17 (95.0 %) (Data augmentation combination) log 81.9 % D20A
2020) % Mel + Δ + ΔΔ(two stage fusion) (Hu (88.0 %)
et al., 2021)
72.3 D18A (93.0 %) Log Mel spec. (average) (Liu et al., 69.0 % D20A
% 2019) (88.0 %)
58.6 D18B (91.0 %) (mixup, device translator, etc.) log 75.3 % D21A
% Mel spec., BC-ResNet (Kim et al., (88.0 %)
2021)
RNN Smile6k features (Li et al., 2017) 80.2 D16 (97.4 %) Perceptually-weighted LMBE ( 97.3 % D20B
% Koutini et al., 2020) (97.3 %)
TDNN Log Mel spec., AMFB and GFB (Schröder 76.5 D16 (97.4 %) Inception LMBE + Δ + ΔΔ (Lee et al., 2021; 63.9 % D20A
et al., 2017) % Szegedy et al., 2016) (88.0 %)
BLSTM Deep Audio Feature (DAF) (Y. Li et al., 2018) 82.1 D17 (95.0 %) Xception RGB image features (TL) (Lu et al., 75.3 % ESC-50
% 2021; Chollet, 2017) (95.7 %)
Traditional Log Mel spec. (Li et al., 2017) 82.2 D16 (97.4 %) MobileNetV2 LMBE, Δ, ΔΔ (Hu et al., 2020; 96.7 % D20B
CNN % Sandler et al., 2018) (97.3 %)
1D, 2D, and 3D CNN fusion (Yin, Shah, & 91 % D16 (97.4 %) SoundNet Feature learning (Aytar, Vondrick, & 88.0 % D13 (97.0
Zimmermann, 2018) Torralba, 2016) %)
(Conditional-GAN) KL, CQT and Mel spec. ( 94.3 D16 (97.4 %) 74.2 % ESC-50
Yang et al., 2019) % (95.7 %)
89.8 D17 (95.0 %) 92.2 % ESC-10
% (97.0 %)
Log Mel spec. (average) (Schröder et al., 91.7 D17 (95.0 %) DCASENet Mel spec. (Jung et al., 2021) 69.5 % D20A
2013) % (88.0 %)
Deep scattering spectrum (DSS) features 78.3 D18A (93.0 %) CAA-Net Feature learning (Ren et al., 2020) 68.0 % D18B
(multi-level attention) (Z. Li et al., 2019) % (91.0 %)
(Mixup) log Mel spec.(average) (Seo, Park, & 80.4 D19A (92.0 %) 56.1 % D19B
Park, 2019) % (91.0 %)
(Mixup, SpecCorrect) feature learning 70.1 D19B (91.0 %) PANNs (Mixup and SpecAugment) log Mel 96.0 % MSOS
(calibration by heated-up softmax) (Nguyen, % spec. (TL) (Kong et al., 2020) (96.0 %)
Pernkopf, & Kosmider, 2020)
Transfer and adapt for feature learning from 83.5 ESC-50 (95.7 %) 94.7 % ESC-50
log Mel spec. (Kumar, Khadkevich, & Fügen, % (95.7 %)
2018)
TEO-GTCC, GTCC (weighted sum) (Agrawal 88.0 UrbanSond8K 76.4 % D19A
et al., 2017) % (99 %) (92.0 %)
Features extracted by temporal transformer 98.1 LITIS (98.9 %) SubSpectralNet Incorporating sub-spec. + CNN ( 74.1 % D18A
module (Zhang et al., 2018) % Phaye, Benetos, & Wang, 2019) (93.0 %)
FCNN LMBE, Δ, ΔΔ (Hu et al., 2020) 76.9 D20A (88.0 %) Fisher network Mel spec. and TCCs + Fisher layers + 93.0 % D18A
% SVM (Venkatesh, Mulimani, & (93.0 %)
Koolagudi, 2023) 91.0 % D18B
(91.0 %)
92.0 % D19A
(92.0 %)
91.0 % D19B
(91.0 %)
85.0 % D20A
(88.0 %)
CRNN Log Mel spec. (Mulimani & Koolagudi, 2021) 91.0 D19A (92.0 %) RQNet Multi-scale Mel spec. (ms2) (Madhu 88.0 % D20A
% & Suresh, 2023) (88.0 %)

importance of different time frames and frequency bands. Recently,
Shim et al. (Shim et al., 2022) proposed an Attentive Max Feature Map
(AMFM) to overcome the problem of excessively discarding information
of the attention mechanism.
6.2.3. Transfer learning
Transfer learning is used to improve a learner in one domain (the
target domain) by transferring information from a related domain (the
source domain) (Weiss, Khoshgoftaar, & Wang, 2016). It allows
retraining well-trained networks on different datasets (Tsalera, Papa­
dakis, & Samarakou, 2021). Therefore, it can not only accelerate the

16
B. Ding et al.
training process and thus reduce model development costs but also
overcome the shortcoming of the domain with limited labeled datasets
that cannot use some more complex knowledge discovery approaches.
Many ASC algorithms rely on well-proven neural network architec­
tures from the computer vision domain or related sound domains. They
use transfer learning to fine-tune pre-trained models developed for
related tasks. For instance, Huang et al. (Huang et al., 2019) fine-tuned
models pre-trained on AudioSet. Similarly, Kong et al. (Kong et al.,
2020) proposed PANNs pre-trained on AudioSet, which can be trans­
ferred to six audio classification tasks (e.g., ASC, SED, audio tagging,
etc.). Moreover, Kumar et al. (Kumar, Khadkevich, & Fügen, 2018)
compared three transfer learning strategies to adapt the pre-trained
deep CNN model to AED and ASC target tasks. There are many suc­
cessful application cases, such as pre-trained SoundNet (Singh et al.,
2018; Singh et al., 2019), ResNet (Ye et al., 2019), Xception (Lu et al.,
2021), VGG16 (Ren et al., 2017), GoogLeNet, SqueezeNet, ShuffleNet,
VGGish, YAMNet (Tsalera, Papadakis, & Samarakou, 2021), etc.
In the ASC tasks of the DCASE challenge, the use of publicly-available
external ASC datasets is explicitly allowed. However, most recently,
winning algorithms did not use transfer learning from these datasets but
instead focused on the provided training datasets combined with data
augmentation techniques (Abeßer, 2020).
6.2.4. Multitask learning
Multitask Learning involves learning to solve multiple related clas­
sification tasks jointly with one network (Ruder, 2017). It improves
generalization by leveraging the domain-specific information in the
training signals of related tasks (Caruana, 1998). Its terms also include
joint learning, learning to learn, and learning with auxiliary tasks.
Typical examples of multitask learning include SoundNet, DCASENet,
PANNs, and CAA-Net. They are proposed to address multiple audio
classification tasks (e.g., SED and ASC (Bear, Nolasco, & Benetos, 2019;
Komatsu, Imoto, & Togami, 2020)).
In particular, Imoto et al. (Imoto et al., 2020) jointed ASC and SED
tasks by using soft labels of acoustic scenes to relate sound events to
scenes. Heo et al. (Heo et al., 2019) used teacher-student learning to
extract soft labels to better represent similarities across different scenes.
In addition, a two-stage multitask model is also applied in ASC (Xu et al.,
2016; Abrol & Sharma, 2020; Shim et al., 2022), which develops a deep
learning model that predicts fine classes as the main task and coarse
classes (high-level classes) as the auxiliary task for boosting the per­
formance of the main task. Besides, Nwe et al. (Nwe, Dat, & Ma, 2017)
grouped acoustic scenes based on types of environments to formulate
ASC into a multitask learning framework for predicting the most likely
acoustic scene within each scene group. Summarily, multitask learning
has been successfully applied in ASC. However, it is not common in
state-of-the-art ASC algorithms (Abeßer, 2020).
6.3. Unsupervised learning
Unsupervised ASC algorithms (called acoustic scene clustering)
merge the audio recordings of the same scene class into a single cluster
without using prior information and training classifiers (Y. Li et al.,
2019). It won’t be limited by the unlabeled, weakly labeled, or incor­
rectly labeled problems in the ASC dataset because it does not require
reference labels. For example, Li et al. (Li & Wang, 2018) proposed an
improved spectral clustering algorithm using a randomly sketched
sparse subspace to classify unlabeled acoustic data or correct wrongly
labeled acoustic data. Moreover, a new-streaming based subspace clus­
tering algorithm was also proposed for the acoustic scene clustering of
overwhelmingly high-volume data (Li et al., 2017). It allows the data to
be clustered on the fly. Unlike these methods, Li et al. (Y. Li et al., 2019)
jointly optimized feature learning (CNN fed by a log Mel spectrum) and
clustering iteration (Agglomerative Hierarchical Clustering) by inte­
grating these two procedures into a single model with a unified loss
function.
17
Expert Systems With Applications 238 (2024) 121902
Although several studies have demonstrated the advantages and
potential value of unsupervised learning in ASC, it is still inferior to
supervised learning in terms of classification performance.
6.4. Semi-supervised learning
Semi-supervised learning can reduce the amount of labeled data
required during the development of deep learning models by using a
small amount of labeled data as well as a large amount of unlabeled data
(Van Engelen & Hoos, 2020). It is conceptually situated between su­
pervised and unsupervised learning. It can be categorized into trans­
ductive and inductive learning according to the primary objective, or
into wrapper methods, unsupervised preprocessing, and intrinsically
semi-supervised methods according to the way unlabeled data is used
(Huang et al., 2006). The semi-supervised learning method has been
introduced into audio classification recently. For instance, Sascha and
Estefanía proposed a FixMatch-based semi-supervised approach, which
was verified in the music, industrial sounds, and ASC tasks. Their
experimental results demonstrated the potential of semi-supervised
methods for audio data (Grollmisch & Cano, 2021).
6.5. Self-supervised learning
Self-supervised learning has gained popularity because it can avoid
the cost of annotating large-scale datasets (Jaiswal et al., 2021). Self-
supervised learning models are employed to learn unsupervised repre­
sentations by solving pretext tasks and using those for performing
downstream tasks such as classification or regression (Tripathi & Mis­
hra, 2021). Pretext tasks are pre-designed tasks for networks to solve,
and unsupervised representations are learned by learning the objective
functions of pretext tasks. The pretext tasks can be predictive, genera­
tive, contrasting, or a combination of them. The supervision signal for
pretext tasks is generated from the input data itself based on its structure
(Jing & Tian, 2021). Self-supervised learning is an under-explored
method in the field of ASC, although it performs remarkably well in
natural language processing and computer vision (Tripathi & Mishra,
2021). For instance, Tripathi and Mishra (Tripathi & Mishra, 2021)
presented a self-supervised learning-based deep classifier for ASC, where
they defined identification of the type of data augmentation as the
pretext task, and a well-trained model learned by solving the pretext task
is further fine-tuned for developing the deep model for ASC.
6.6. Result post-processing
The purpose of result post-processing is to process the prediction
results of ASC for improving performance. It can be considered a kind of
auxiliary task. We categorize it into three groups as follows:
1) Late fusion refers to the process of obtaining the final output by
combing the outputs of a set of individual classifiers, organized in a
parallel way, using a combination method (Santana et al., 2010).
Late fusion is also called model ensemble. The combination methods
have been summarized in (Kuncheva, 2004), detailing how they
combine several classifiers for performance improvement.
Late fusion makes the final decision for a test instance to improve
performance by fusing the predictions outputted by multiple well-
trained ASC models (Dong et al., 2018; Ding et al., 2022). The fusion
strategies include voting, averaging, weighted averaging, and stacking
(Jiang, Shi, & Li, 2019). The voting method gets the class with the most
votes in each sample as the final prediction. The averaging method
obtains the final prediction by averaging the prediction probabilities of
different systems. The weighted averaging method, a variant of aver­
aging, averages prediction probabilities with unequal weights for each
class. The stacking method combines prediction probabilities via a meta-
classifier, such as random forest (Liu et al., 2019), logistic regression,
B. Ding et al.
and softmax (Byttebier et al., 2021).
Generally, averaging and voting methods are comparatively simple,
and the stacking method is often more effective. In addition, other
strategies are also employed for late fusion, such as weighted sum
(Paseddula & Gangashetty, 2021), late fusion using swarm intelligence
(Ding et al., 2022), and late fusion based on CNNs and ensemble clas­
sifiers (Alamir, 2021). Moreover, Li et al. (Li et al., 2017) employed
random forest, extremely randomized trees, Adaboost, gradient tree
boosting, and weighted average probabilities to ensemble multiple ASC
models.
2) Calibration is another special result post-processing method that
processes the prediction result of a single ASC system for perfor­
mance improvement (Kong et al., 2020). For example, Platt directly
optimized the logarithmic loss standard through calibration based on
logistic regression (Guo et al., 2017; Alamir, 2021). Nguyen and
Pernkopf (Nguyen, Pernkopf, & Kosmider, 2020) used a heated-up
softmax to calibrate the predictions of the ASC model. The calibra­
tion method focuses on the prediction of a single system. The late
fusion method can be considered a special calibration process based
on multiple classification systems.
3) Two-stage fusion fuses the prediction results of 3-class (coarse
classes) and 10-class (fine classes) classifiers using an ad-hoc score
combination method (Hu et al., 2020; Hu et al., 2021). These two
classifiers are generally trained separately. It can also be integrated
into the model learning process, which is the joint training of two
classification tasks. For instance, Shim et al. (Shim et al., 2022)
jointly learned the 10-class and 3-class ASC in a model to implement
two-stage fusion for ASC. This method is relatively low-complexity
and competitive compared with the two-stage fusion in separate
training.
7. Open resources for ASC
7.1. Open-source code libraries for ASC
There are several commonly-used open-source projects and toolkits
dedicated to the ASC research community.
The Environmental Audio Recognition System (EARS9) is an open-
source project. It implements a CNN for live environmental audio pro­
cessing and recognition on low-power SoC devices (Raspberry Pi 3
Model B). EARS features a background thread for audio capture and
classification and a Bokeh server-based dashboard providing live visu­
alization and audio streaming from the device to the browser.
dcase_util10 is a collection of utilities for DCASE challenges. It was
bundled into a standalone library to allow its reuse in other research
projects. Most utilities are related to audio datasets for handling meta­
data and various forms of other structured data and providing a stan­
dardized usage API to audio datasets from different sources.
sed_eval11 is an open-source Python toolbox that provides a stan­
dardized and transparent way to evaluate SED and ASC systems
(Mesaros, Heittola, & Virtanen, 2016a).
openSMILE12 is an open-source feature extraction toolkit imple­
mented in C++ (Eyben, Wöllmer, & Schuller, 2010). It uses feature
extraction algorithms from the speech processing and music information
retrieval communities. It can be used to extract Low-Level Descriptors
(LLD) and apply various filters, functions, and transformations to them.
Various hand-crafted features are supported, such as CHROMA and
CENS features, loudness, MFCC, LPC, Perceptual Linear Predictive
Cepstral Coefficients (PLPCC), line spectral frequencies, fundamental
9
https://github.com/karolpiczak/EARS.
10
https://github.com/DCASE-REPO/dcase_util.
11
https://github.com/TUT-ARG/sed_eval.
12
https://opensmile.sourceforge.net/.
18
Expert Systems With Applications 238 (2024) 121902
frequencies, formant frequencies, and zero-crossings.
7.2. Available datasets for ASC
An audio classification system must be tested on standardized audio
datasets to evaluate its performance in a fair quantitative comparison
with similar studies. Therefore, we overview several available datasets
for ASC baseline testing in Table 8. The AudioSet dataset is generally
used to pre-train networks for transferring knowledge from SED to ASC
tasks. Other datasets, such as LITIS Rouen, ESC-50, ESC-10, MSOS,
UrbanSound8K, and the datasets for DCASE ASC tasks, have been
commonly applied in ASC.
All datasets for DCASE ASC tasks represent the development datasets
by default because reference labels are provided only for the develop­
ment datasets. In addition, there may be intersecting sets between these
ASC datasets. For example, TAU Urban Acoustic Scenes 2019 is a subset
of TAU Urban Acoustic Scenes 2020 Mobile. Audio Files for MSOS come
from the Freesound database,13 the ESC-50 dataset, and the Cambridge-
MT Multitrack Download Library.14
8. DCASE Challenges
Competitions and community activities play a significant role in
attracting excellent researchers and further promoting advanced
methods in the research field (Gharib et al., 2018a) because of their
benefits in terms of reproducibility of results, dissemination of open-
source code libraries, and documentation of results. CLEAR evaluation
and the DCASE challenge are ASC-related challenges.
The CLEAR evaluation (i.e., CLEAR 2006 (Stiefelhagen et al., 2007)
and CLEAR 2007 (Stiefelhagen et al., 2008)) involved acoustic, visual,
and audio-visual analysis. They mainly include tracking tasks (faces,
people, and vehicles), person identification tasks, head pose estimation,
and acoustic scene analysis (events and environment, i.e., SED, ASC
tasks). Although the ASC task is contained, it only accounts for a small
proportion. In particular, only one site participated in the ASC task of
CLEAR 2006. CLEAR 2007 even deleted the ASC task and only kept the
SED task.
The DCASE challenge is one of the competitions active in fields
related to auditory computational analysis. It provides researchers with
a fair comparison of available datasets and makes their research results
more specific for others to compare. It has been launching ASC tasks
annually since 2016. Therefore, we discuss ASC development and trends
by analyzing ASC task setup, results, and promising methods in DCASE.
8.1. Task setup of ASC tasks
The DCASE challenge started in 2013 and has been held annually
since 2016. We report all DCASE ASC tasks in Fig. 11 in terms of tasks,
number of scenes, dataset duration, segment duration, sampling rate,
baseline accuracy, and highest accuracy on the evaluation dataset. From
this figure, the dataset duration for DCASE ASC tasks increases from 50
min to 64 h. The segment duration ranges from 30 s for D13 and D16 to
10 s for D17-D21, and then to 1 s for D22 and D23. Summarily, the
development trends of ASC tasks mainly include increasing dataset
scale, external data, multiple devices, the open set, low complexity,
audio-visual fusion, and shorter segments.
Notably, all ASC tasks since the D18C task allow using external data.
Moreover, there exist ASC tasks associated with multiple devices each
year since 2018, such as D18B, D19B, D20A, D21A, D22, and D23. Their
development datasets have inclusion relationships shown in Fig. 12. It
shows the continuous expansion of development datasets in DCASE ASC
tasks, especially in the later stage, which is directly expanded on the
13
https://freesound.org/.
14
https://www.cambridge-mt.com/ms-mtk.htm.
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

Table 8 Table 8 (continued )

Available datasets for ASC. Notably, the data in brackets in the dataset duration Database name Classes Samples Dataset Usage, publications
column is the duration of an audio segment. Duration
Database name Classes Samples Dataset Usage, publications Scenes 2019
Duration Mobile
Dares G1 28 123 123 min Stress testing and TUT Urban 10 15,850 44 h (10 DCASE2019 Task1C (
developing recognition Acoustic s) Mesaros, Heittola, &
systems, (Computational) Scenes 2019 Virtanen, 2018b)
auditory scene analysis, Openset
Auditory cognition MSOS 5 2000 2.8 h (5 Making Sense of Sounds
research, and Hearing aid s) (MSOS) challenge (Kroos
validation (Van Grootel, et al., 2019)
Andringa, & Krijnders, HEAR-DS 14 10,226 28.4 h Hearing aid research data
2009) (10 s) for ASC (Martín-Morató
LITIS Rouen 19 3026 1513 min ASC based on image et al., 2021)
(30 s) features ( TAU Urban 10 23,040 64 h (10 DCASE2020&2021
Rakotomamonjy & Gasso, Acoustic s) Task1A (Heittola,
2015) Scenes 2020 Mesaros, & Virtanen,
YouTube-8 M 4716 >7M >27 M A large-scale video Mobile 2020)
classification benchmark TAU Urban 3 14,400 40 h (10 DCASE2020 Task1B (S.
(Abu-El-Haija et al., Acoustic s) Wang, Mesaros, Heittola,
2016) Scenes 2020 & Virtanen, 2021)
AudioSet 527 2.1 M 5800 h Audio event recognition ( 3Class
Gemmeke et al., 2017) TAU Urban 10 12,240 34 h (10 DCASE2021 Task1B (
ESC-50 50 200 166 min Environmental sound Audio Visual s) Gharib et al., 2018a)
(5 s) classification (Piczak, Scenes 2021
2015b) TAU Urban 10 230,350 64 h (1 s) DCASE2022&2023 Task1
ESC-10 10 400 33 min Environmental sound Acoustic (Mesaros, Heittola, &
(5 s) classification (the subset Scenes 2022 Virtanen, 2018b)
of ESC-50) Mobile
NYU Urban 10 8732 525 min Urban sound research
Sound8K (<=4s) (analysis, classification,
etc.) (Salamon, Jacoby, & original basis. The low-complexity ASC tasks are constrained by a
Bello, 2014) limited model complexity (e.g., 500 KB or 128 KB). The calculation rule
CHIME-Home 7 6137 409 min Sound source recognition of model complexity varies from ASC tasks. More details refer to DCASE
(4 s) in a domestic
environment (Foster
website.15
et al., 2015) We also report acoustic scenes for DACSE ASC tasks in Table 9. From
Freefield1010 7 7690 1282 min Audio recording archives this table, acoustic scenes are various before 2018, but scene classes
(Stowell & Plumbley, from 2018 are limited to urban acoustic scenes, mainly including three
2014)
major classes.
CICESE Sound 20 1367 92 min Scalable identification of
Events mixed environmental In general, the task complexity of ASC tasks increased over the years.
sounds, recorded from It is not only affected by dataset duration, segment duration, and scene
heterogeneous sources ( classes but also by the focus points of each ASC task (e.g., multiple de­
Beltrán, Chávez, & vices, use of external data, open set, low complexity, and audio-visual
Favela, 2015)
DCASE 2013 10 100 50 min DCASE2013 Task1 (
fusion). To quantify the task complexity, we attempt to calculate a
Scenes (30 s) Giannoulis et al., 2013b) task complexity factor based on these influence factors as follows.
DEMAND 18 18 (16 1.5 h (5 Multichannel
channels) min) environmental noise for (1) If CF , CC , CD , and CS present the task complexity factors of task
audio processing, such as
focus, the number of scene classes, datasets duration, and
source separation, signal
enhancement, and segment duration, The task complexity C is calculated by C =
acoustic noise CF ⋅CC ⋅CD ⋅CS .
suppression/removal ( (2) If CdF , CeF , CoF , ClF , and Cav
F correspond to task complexity factors of
Thiemann, Ito, & Vincent,
multiple devices, use of external data, open set, low complexity,
2013)
TUT Sound 15 1170 585 min DCASE2016 Task1 ( and audio-visual fusion, CF is their sum, but if it is calculated to 0,
Scenes 2016 (30 s) Mesaros, Heittola, & it will be assigned a value of 1; CdF is calculated by CdF =
Virtanen, 2016b) 5⋅(Ndevices − 1), where N devices is the number of recording de­
TUT Acoustic 15 4680 780 min DCASE2017 Task1 (
scenes 2017 (10 s) Mesaros et al., 2017) vices. Others is calculated by CeF , CoF , ClF , Cav F =
{
TUT Urban 10 86,400 24 h (10 DCASE2018 10, if the focus point exists
.
Acoustic s) Task1A&Task1C ( 0, others
Scenes 2018 Mesaros, Heittola, &
(3) CC and CD are equal to the number of scene classes and datasets
Virtanen, 2018b)
TUT Urban 10 10,080 28 h (10 DCASE2018 Task1B (
duration, respectively (e.g., CC = 10 for 10-class ASC or CD = 40
Acoustic s) Mesaros, Heittola, & when duration dataset is 40 h).
Scenes 2018 Virtanen, 2018b) (4) CS = 1 if segment duration is 30 s; CS = 3 if segment duration is
Mobile 10 s; CS = 30 if segment duration is 1. Because long-duration
TAU Urban 10 14,400 40 h (10 DCASE2019 Task1A (
sounds are generally most informative for ASC (Wu & Lee,
Acoustic s) Mesaros, Heittola, &
Scenes 2019 Virtanen, 2018b) 2020; Ebbers, Keyser, & Haeb-Umbach, 2021). Therefore, we
TAU Urban 10 16,560 46 h (10 DCASE2019 Task1B (
Acoustic s) Mesaros, Heittola, &
Virtanen, 2018b)

15
https://dcase.community/.

19
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

Fig. 11. The task setup of DCASE ASC tasks. The duration represents the duration of the development dataset and audio segment. The baseline and highest accuracies
are evaluated on the corresponding evaluation.

Fig. 12. The relationship between different development datasets of DCASE ASC tasks. The dataset for D22 and D23 has the same data with different segment
duration as D20A and D21A.

D21B.
Table 9
The acoustic scenes for DCASE ASC tasks.
9. Results of ASC tasks
DCASE Scene Acoustic Scenes
editions number
We also report the accuracies of the top-5 ranked systems of DCASE
D13 10 Bus, busystreet, office, open air market, park, quiet ASC tasks in Fig. 14. By observing this figure, we can draw the following
street, restaurant, supermarket, tube, tubestation conclusions:
D16-D17 15 Bus, cafe/restaurant, car, city center, forest path,
grocery store, home, lakeside beach, library, metro
station, office, residential area, train, tram, urban park (1) The accuracies of the top-5 systems for the D20B (3-class) and
D18-D23 10 Airport, shopping_mall, metro_station, D21B tasks (audio-visual) are more than 90 % due to their lower
street_pedestrian, public_square, street_traffic, tram, task complexity and advanced deep learning methods.
bus, metro, park
(2) The accuracy of the D13 task is very low because of the immature
D20B 3 Indoor (airport, metro_station, and shopping_mall),
outdoor (park, public_square, street_pedestrian, and ASC technologies. And the accuracy of D16 is higher because of
street_traffic), transportation (bus, metro, and tram) the introduction of deep learning methods into ASC.
(3) From Fig. 14 (a), the best accuracy of D16, D17, D18A, and D19A
is more than 80 %, while the highest accuracy of D18B, D19B,
consider that the shorter the segment duration, the higher the D20A, D21A, D22, and D23 is less than 80 % (even less than 70
task complexity. %). It might be caused by constraints of dataset size, multiple
devices, and low model complexity.
This calculation rule allows a qualitative comparison of task (4) The constraint of low complexity slightly affects ASC perfor­
complexity for DCASE ASC tasks shown in Fig. 13. This figure shows an mance by observing the results of D20A and D21A, where D21A
increasing trend of the overall task complexity over time. It is affected by only added a low-complexity constraint.
different influence factors, which is especially obvious in D20B and

20
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

Fig. 13. The task complexity of DCASE ASC tasks. Only the complexity of the entire task (C_tasks) uses the scale of the right Y-axis. It is noted that the D23 task is a
continuation of the D22 task, with modified memory limit and added measurement of energy consumption. Both D22 and D23 tasks focus on low-complexity
problems although there are subtle changes. With the calculation rule of task complexity, their task complexity factor are equal.

Fig. 14. The accuracies of the top-ranked systems submitted for ASC tasks in previous DCASE challenges. Notably, the vacancies in (b) mean no data.

21
B. Ding et al. Expert Systems With Applications 238 (2024) 121902

(5) The 10 s segment classification outperforms the 1 s by comparing
D21A (10 s) with D22/D23 (1 s), where they contain the same
audio data with different segment durations.
(6) By observing the results of D21A and D22/D23 in Fig. 14 (a), the
accuracy of the ranked low system may be higher than that of the
ranked high system because the ranking criteria changed from
macro-average accuracy to multiclass cross-entropy (e.g., D21A,
D21B, and D22) or weighted average rank of multiple criteria (e.
g., D23) since 2021.
(7) DCASE challenges play a critical role in stimulating the devel­
opment of ASC, as reflected in the accuracy improvement of top-
ranked systems compared with baseline systems, ranging from 7
% to 30 % with an average of about 20 %.
(8) The slight accuracy difference among the top 5 systems for the
same task (especially obvious since 2019) demonstrates that
high-performance ASC methods are diverse. The method di­
versity benefits ASC algorithm optimization under the constraints
of task complexity and limited model complexity.
(9) From Fig. 14 (c), some ASC systems are overconfident and per­
formed much worse on the evaluation dataset than on the
development dataset, corresponding to a maximum drop of 17.5
%. Some systems performed much better on the evaluation
dataset than the development dataset, corresponding to a
maximum increment of 10.9 %. The top system of each task
almost does not suffer from these two extreme problems; the
accuracy gap between the development and evaluation datasets is
less than 5.4 %.
Finally, we find that not a few teams have participated in the DCASE
ASC competition many times and ranked high by tracking the partici­
pation of top-ranked teams. For instance, the CP-JKU team participated
in almost all ASC competitions (Eghbal-Zadeh et al., 2016; Koutini et al.,
2020; Lehner et al., 2017; Dorfer et al., 2018; Koutini, Eghbal-zadeh,
Widmer, & Kepler, 2019, Koutini, Jan, & Widmer, 2021; Schmid
et al., 2022; Schmid et al., 2023), and won D16, D20B, D22, and D23.
Also, this team was ranked 2nd in D18A and D19A. The team created by
Park and Mun et al. was ranked 3rd and 1st in D16 and D17 (Park et al.,
2016; Mun et al., 2017a). The team formed by Yang et al. (Yang, Chen, &
Tao, 2018) got 5th and 2nd place in D18A and D18B. The team of
McDonnell and Gao was ranked 2nd, 3rd, and 3rd in D19B, D20A, and
D20B (Koutini et al., 2020; McDonnell & Gao, 2020; McDonnell &
UniSA, 2020). The team of Hu et al. was ranked 2nd in D20A and D20B
(Hu et al., 2020; Yang et al., 2021a). Then, they and Wang et al. got 2nd
place in D21A and D21B together as a team (Q. Wang et al., 2021).
9.1. The most promising methods in ASC competitions
We further analyze the most promising methods in DCASE ASC
competitions. Particularly, we report specifications of top systems in
DCASE ASC tasks and their accuracies on the development and evalu­
ation datasets (Eval. vs. Dev.) in Table 10. From this table, the following
points can be observed: (i) CNN was gradually introduced into ASC, and
the complexity of CNN increased, (ii) Data augmentation methods such
as mixup and GAN have been applied to boost performance for the
problem of requiring larger labeled data for model training in deep
learning (e.g., CNN), (iii) The top systems are stable and have good
generalization ability according to their slight accuracy difference be­
tween the development and evaluation datasets, (iv) The diversity of
features for ASC was increased, from the initial RQA, MFCC features,
and spectrograms to recent LMBE, as well as CQT features for supple­
menting LMBE features, (v) The effect of resampling and multichannel
in the input stage on performance improvement cannot be observed
from this table, and (vi) Late fusion methods are effective for perfor­
mance improvement because most top systems have used late fusion
except for D13 task.
Also, we summarize the promising ASC methods in terms of input,
sampling rate, date augmentation, features, classifier, and output by
analyzing DCASE technical reports. ASC systems commonly use
monaural, multichannel signals, or signal components obtained by
HPSS. The sampling rate is generally reset by resampling to adapt to the

Table 10
Top systems submitted for ASC tasks in previous DCASE challenges. CNN1 represents the use of traditional CNN architectures for classification. More details please
refer to (Roma, Nogueira, & Herrera, 2013; Eghbal-Zadeh et al., 2016; Mun et al., 2017a; Mesaros et al., 2018; Sakashita & Aono, 2018; Nguyen & Pernkopf, 2018;
Kosmider, 2019; Chen et al., 2019; Suh et al., 2020; Koutini et al., 2020; Kim et al., 2021; Schmid et al., 2022; Schmid et al., 2023).
DCASE Input Sampling Data Augmentation Features Classifier Output Eval. Dev.
ASC tasks Rate

D13 44.1 kHz RQA, MFCC SVM 76.0 % 71.0 %

D16 mono + 44.1 kHz MFCC, spectrogram i-vector, DCNN Averaging, 89.7 % 89.9 %
binaural calibration
D17 left, right, 22.05 kHz GAN LMBE, spectrogram MLP, RNN, CNN1, SVM Majority vote 83.3 % 87.1 %
mixed
D18A mono, 44.1 kHz mixup LMBE CNN1 Random forest 81.0 % 76.9 %
binaural
1
D18B 44.1 kHz LMBE and their nearest CNN Averaging vote 69.0 % 63.6 %
neighbor filtered version
1
D19A binaural 48 kHz GAN LMBE, CQT CNN Average vote 85.2 % 85.1 %
D19B 44.1 kHz SpecCorrect, LMBE CNN1 Soft-voting 75.3 % 70.0 %
specAugment, mixup
D20A 44.1 kHz temporal cropping, LMBE, deltas, delta-deltas ResNet, Snapshot Averaging, 76.5 % 74.2 %
mixup weighted
averaging
D20B stereo 22.05 kHz mixup Perceptually-weighted RF-regularized CNNs Averaging 96.5 % 97.3 %
LMBE
D21A 16 kHz mixup, time rolling, LMBE CNN1, BC-ResNet Maximum 76.1 % 75.9 %
specAugment likelihood
D21B 48 kHz mixup LMBE, CQTbark CNN1, EfficientNet, Swin Weighted vote 93.8 % 95.1 %
Transformer, ensemble
D22 32 kHz mixstyle, pitch shifting LMBE RF-regularized CNNs, Averaged logits 59.6 % 58.0 %
PaSST transformer
D23 32 kHz device impulse response LMBE RF-regularized CNNs, Averaged logits 58.7 % 58.4 %
augmentation, PaSST transformer
mixup,
freq-mixstyle,
pitch shifting

22
B. Ding et al.
ASC system. Next, data augmentation methods are often employed, such
as mixing, time shifting, pitch shifting, mixup, SpecAugment, frequency
shifting, random noise, and GAN. The popular features include LMBE,
MFCCs, CQT, raw waveform, deltas and delta-deltas of LMBE, HOG, etc.
The classifier is generally based on CNNs (e.g., ResNet), SVM, GMM, or
ensemble models. Finally, the output is usually processed by result post-
processing methods for higher performance.
9.2. Open challenges and future directions
This section discusses open challenges and future directions. Chal­
lenges are arose from developing and applying ASC algorithms in the
real world. Focusing on these challenges, we suggest several future di­
rections to address ASC problems.
9.3. Open challenges
The large progress in the ASC field is reflected in the increasing
datasets and significantly improved accuracy. However, the four ASC
problems mentioned in section 2.3 are still pending. Among these
problems, the unclear definition of acoustic scene and the lack of large-
scale comprehensive datasets are generalized problems that are hard to
address by a specific method. Because it is hard to provide a clear and
uniform acoustic scene definition for ASC. ASC datasets are closely
related to acoustic scene definition. Hence, the lack of large-scale
comprehensive datasets is hard to address as well, although there are
many ASC datasets.
We thus mainly discuss the challenges raised by the rest of the
problems (i.e., performance optimization and computational
complexity) in this section. Once they are overcome, the development of
ASC will make a great leap. Subsequently, various excellent ASC tech­
nologies would be applied to actual life and production, which can
significantly enhance production efficiency and improve people’s
quality of life.
9.3.1. Performance optimization
According to the influence factors of performance, we discuss per­
formance optimization challenges from the following four perspectives:
1) Multiple devices: Audio data is often recorded with distinct devices,
such as professional sound recording devices (Brezina & Jeseniˇcov́ a,
2018) and mobile devices (Goyal, Shukla, & Sarin, 2019). Varying
characteristics of recording devices lead to various qualities of audio
data, such as sampling rate, amplitude, frequency response, and data
distributions (Primus et al., 2019). However, it is challenging to train
models on an audio dataset with different data distributions caused
by mismatched devices (Nguyen & Pernkopf, 2019b). Various
methods, such as data augmentation (Nguyen & Pernkopf, 2019a; Hu
et al., 2021; Morocutti et al., 2023), spectrum correction (Nguyen,
Pernkopf, & Kosmider, 2020; Kosmider, 2020), and domain adap­
tation (Primus et al., 2019; Yang, Wang, & Zou, 2021; Gharib et al.,
2018b), have been used to address this problem.
Data augmentation works by increasing extra training data con­
taining device-related information. For example, mixup is used to
enlarge the training set recorded by devices with fewer samples to solve
the data imbalance problem between recording devices (Nguyen &
Pernkopf, 2019a). Spectrum correction reduces the difference among
mismatched devices by transforming a given input spectrum to that of a
reference (Nguyen, Pernkopf, & Kosmider, 2020). For instance, Kos­
mider (Kosmider, 2020) used it to transform the distortion of all known
devices into the distortion of a chosen reference device by matching
their spectra. Domain adaptation adjusts samples, features, or models
from the source data to the target data to minimize their discrepancy and
thus improve performance. It can be divided into three groups: instance
adaptation (e.g., instance weighting (Jiang & Zhai, 2007), sample
23
Expert Systems With Applications 238 (2024) 121902
reweighting (Huang et al., 2006), weighted local domain adaptive
method (He & Zhu, 2021), and DIDA (Deng et al., 2022)), feature
adaptation (feature alignment) (Wang et al., 2019), and model adapta­
tion (Primus et al., 2019; Zhao et al., 2022). Also, it can be categorized
into supervised (Abeßer, 2020; Ren et al., 2020) and unsupervised (e.g.,
MMD (Long et al., 2016), UADA (Gharib et al., 2018b), W-UADA
(Drossos, Magron, & Virtanen, 2019), DANN (Wang et al., 2019), MCD-
KD (Takeyama et al., 2021), and MTDA (Yang et al., 2021)) methods
according to whether target domain data are labeled.
Abeßer (Abeßer, 2020) argued that combining approaches for
domain adaptation and data augmentation jointly improved the
robustness of ASC algorithms against changes in acoustic conditions. In
addition, the problem of multiple devices can also be solved from the
perspective of feature extraction and classification. For instance, Jiang
et al. (Jiang et al., 2023) employed the class hierarchy relationship of
acoustic scenes to learn advanced features that are less susceptible to
differences in device information. Aryal and Lee (Aryal & Lee, 2023)
proposed a frequency-aware convolutional neural network (FACNN) to
solve the device mismatch problem by focusing on the frequency in­
formation of the audio samples. However, the performance of multiple-
device ASC tasks is still low, such as the D18B, D19B, D20A, D21A, D22,
and D23 tasks. Therefore, the multiple-device problem is still an open
challenge to be addressed for ASC development.
2) Deep learning methods: Although various deep learning methods
have been applied in ASC, there is still much space for performance
improvement. ASC algorithms can be optimized by novel deep
learning methods, such as attention mechanisms, transfer learning,
and multitask learning. The attention mechanism determines which
part needs to be focused on by estimating the contribution of the
input and thus boosts classification performance. Transfer learning
allows an ASC algorithm to retrain a pre-trained model by trans­
ferring existing knowledge from other tasks for performance
improvement. Multitask learning has achieved great success in task
assistance for ASC, such as the two-stage fusion by joint training and
multitask learning embeded SED.
A quantitative analysis of performance improvement and theoretical
guidance is still underexplored, although some papers have validated
the effectiveness of these deep learning methods for ASC. Therefore,
more research is needed.
3) Model Interpretability: Occasionally, evaluation metrics calculated
on only predictions and ground-truth labels do not suffice to char­
acterize the model. The demand for the reliability and interpret­
ability of a model arises when a mismatch occurs between the formal
objectives of supervised learning (testing performance) and the real-
world costs in a deployment setting. The desiderata of interpret­
ability research include trust, causality, transferability, informa­
tiveness, and fair and ethical decision-making (Lipton, 2018).
Model interpretability helps to explain the methods of model pre­
diction and internal feature representation. It will be useful to improve
model performance, confidence, and robustness when deploying the
model in the real world. For example, Wu and Lee (Wu & Lee, 2019)
proposed a CNN visualization method called Grad-CAM (Zhang et al.,
2021) to help people understand the relationship between the input and
output. However, it is still a challenge to develop methods that allow for
a better interpretation of the model predictions and internal feature
representations (Abeßer, 2020). Also, how to use model interpretability
to boost model performance and guide an efficient deployment of ASC
algorithms in the real world is another challenge that needs to be
overcome.
4) Result post-processing: result post-processing has become an essen­
tial part of state-of-the-art ASC algorithms recently (Abeßer, 2020).
B. Ding et al.
However, there are still two primary challenges that need to be
addressed. The first is how to improve performance steadily and
significantly with a general fusion method. Simple post-processing
methods (e.g., averaging) can only slightly improve performance.
While complex methods are designed for special tasks, which are
difficult to widely use in ASC tasks. For example, the optimization-
based late fusion method (Ding et al., 2022) requires various initial
ASC systems for optimization fusion. Two-stage fusion (Ren et al.,
2018; Hu et al., 2020; Hu et al., 2021) requires extra coarse class
labels and an auxiliary system. Second, result post-processing
requiring multiple models is often not feasible in the real world,
limited by available computational resources and real-time
requirements.
9.3.2. Computational complexity
The real-world solutions need to operate on devices with limited
computational capacity (Martín-Morató et al., 2021). Moreover, real-
time processing requirements often demand fast model prediction
with low latency. Therefore, we analyze computational complexity
challenges from the following two perspectives.
1) Model compression: Model compression refers to using a fast and
compact model to approximate the function learned by a slower,
larger, but better performing model (Buciluǎ, Caruana, & Niculescu-
Mizil, 2006). It is also widely concerned in ASC (Mohaimenuzzaman
et al., 2023), especially as DACSE challenges have released low-
complexity tasks for four consecutive years. Cheng et al. (Cheng
et al., 2018) summarized four types of model compression methods
as follows: (i) Parameter pruning and sharing remove inessential pa­
rameters in DNN without (little) performance loss (Gou et al., 2021).
It can be subdivided into model quantization (e.g., 8/16-bit quanti­
zation (Vanhoucke, Senior, & Mao, 2011; Gupta et al., 2015; Kim,
Yoo, & Kwak, 2020), ternary quantization (2-bit) (Zhu et al., 2017;
Chang et al., 2018), and binarization (1 bit) (Courbariaux, Bengio, &
David, 2015)), pruning and sharing, and designing the structural
matrix (Cheng et al., 2018). (ii) Low-rank factorization estimates the
informative parameters of the deep models by using matrix/tensor
decomposition (Cheng et al., 2018). (iii) Transferred/compact con­
volutional filters remove inessential parameters by transferring or
compressing the convolutional filters (Gou et al., 2021). And (iv)
Knowledge distillation (KD) (Gou et al., 2021; Kang et al., 2023; Tri­
pathi & Pandey, 2023) learns a smaller student model from a larger
network with good performance, which is implemented by learning
the output class distributions of the teacher model via softened
softmax.
Additionally, model simplification is also employed to compress a
model, implemented by reducing the model parameters (Pham et al.,
2023; Liang, Zhang, & Feng, 2020), scaling the network dimension (Li
et al., 2020), or using slim models, e.g., depth-wise separable CNNs
(Drossos et al., 2020; Lee et al., 2021). MobileNetV2 and EfficientNet
(Tan & Le, 2019) are successful examples. Also, some optimization
works to empower a small-size low-complexity CNN model are explored.
For example. Seresht and Mohammadi (Seresht & Mohammadi, 2023)
proposed a novel global pooling method called Sparse Salient Region
Pooling (SSRP), which computes the channel descriptors using a sparse
subset of features, and guides the model to effectively learn from the
more salient time–frequency regions.
Various model compression methods or their combinations (Yang
et al., 2021a; Kim et al., 2021; Han, Mao, & Dally, 2016) have efficiently
reduced the model complexity of ASC systems. However, the model
compression method for ASC is selected based on the researchers’
experience. There is still a lack of guidance for researchers to choose a
fast, simple, and efficient model compression scheme for ASC.
24
Expert Systems With Applications 238 (2024) 121902
2) Deployment based on collaborative end-edge-cloud computing: It
might still be a big challenge to deploy a compressed deep model on
the end device with limited resources, especially in complex appli­
cation scenarios. Although cloud computing can process
computation-intensive tasks (e.g., deep learning), it cannot guar­
antee low latency throughout the whole process of data generation,
transmission, and execution (Wang et al., 2020). Focusing on the
issue, a novel computing paradigm, collaborative end-edge-cloud
computing (Huang et al., 2018; Wang et al., 2020) was proposed,
which has potential for the deployment of complex ASC systems. By
using this computing paradigm, the high-complexity computation
will be reasonably decomposed and dispatched separately to the end,
edge, and cloud for execution, reducing the execution delay of the
task while ensuring the classification performance (G. Li, Liu, Wang,
Dong, Zhao, & Feng, 2018; Zhang & Yan, 2023).
However, how to reasonably decompose the ASC task and allocate
the end, edge, and cloud resources is still a challenge. It involves the
granularity determination of dividing tasks, computing resource evalu­
ation, and task scheduling optimization.
9.4. Future directions
As mentioned above, ASC faces four main problems. We present our
thoughts on research directions with aspects of these problems and their
challenges, as shown in Fig. 15. For the first problem, acoustic scenes
can be defined according to specific ASC applications combined with the
existing categorization of environmental sounds. Then, we summarize
three directions for the ASC dataset problem: (i) Building/collecting
specific ASC datasets directly, which is intuitive and effective but
complex and costly, (ii) Automatic audio tagging can be applied as an
alternative solution, which efficiently reduces the cost of manual
tagging for big data but relies on high-performance annotation algo­
rithms, (iii) Transfer learning (it requires less labeled data since it can
transfer knowledge from other related tasks) or few-shot learning (it
requires a few samples) can also be a feasible solution (Xie et al., 2023).
We provide several suggestions for the low-performance problem in
terms of corresponding open challenges. First, we suggest using data
augmentation, spectrum correction, or domain adaptation for the chal­
lenge of multiple devices. Second, we advise extracting multi-view
features and using efficient classification methods (such as DNN and
CNN) to optimize ASC performance from the perspective of deep
learning. Next, we recommend designing characterizable and inter­
pretable model components/structures, e.g., the attention model. Also,
appropriate result post-processing methods are suggested to boost ASC
performance, for example, a novel optimization algorithm can be
introduced into the late fusion for ASC (Zhao et al., 2023a; Zhao et al.,
2023b). Finally, the suggestion of designing a low-complexity ASC
model and using model compression is provided from the perspective of
model compression. Additionally, we suggest deploying ASC systems
based on collaborative end-edge-cloud in the real world.
Inspired by the application of reinforcement learning in speech
recognition to enhance speech (Shen et al., 2019) or optimize supervised
model training or adaptation methods (Kala & Shinozaki, 2018), rein­
forcement learning is also suggested to be used in ASC. In addition,
applications of ASC need to be explored, such as smart homes, smart
cities, driverless cars, virtual reality, etc. It is the key to improving the
application value of ASC and stimulating the development of ASC more
efficiently.
10. Conclusion
In recent years, a rapid increase of research papers have been pub­
lished in the ASC field. It means ASC has raised widespread attention.
There are several reasons for achieving this progress. Firstly, various
available dataset, code repositories, and tutorial materials make it easier
B. Ding et al.
Fig. 15. Research directions for the ASC field.
to start the research on ASC. Secondly, advanced audio signal processing
and deep learning techniques are proposed or introduced into ASC, such
as channel selection, data augmentation, feature acquisition, CNN
network designing, transfer learning, multitask learning, and attention
mechanism, etc. Finally, the annual DCASE challenge stimulates and
guides researchers to address complex problems in the ASC study (e.g.,
large-scale datasets, use of external data, open set, low-complexity,
multiple devices, audio-visual fusion, and shorter segments).
The recent state-of-the-art algorithms have achieved relatively high
performance on most available ASC datasets (refer to Table 2). For
example, the highest accuracy of ASC reach 99.0 % on UrbanSound8K,
97.0 % on ESC-10, 95.7 % on ESC-50, 98.9 % on LITIS, 96.0 % on MSOS,
and over 90 % on most ASC datasets from the DCASE challenge (e.g.,
D13, D16, D17, D18A, D18B, D19A, D19B). Recently, the ASC method
using multi-scale Mel spectrogram and designed RQNet network ach­
ieved the best accuracy of 88.0 % on the D21A datasets (focusing on
multiple devices and low complexity issues). However, the accuracy on
the newest D22 and D23 datasets (focusing multiple devices, low-
complexity issues, and shorter segments issues) is still low (under
75.0 %). It indicates that complex ASC tasks still face low- performance
issues.
We have summarized four problems in ASC, including unclear defi­
nition of acoustic scenes, lack of large-scale comprehensive datasets, low
classification performance, as well as computational efficiency and
practicability. Generally, performance optimization and computational
complexity are commonly focused by researchers. In terms of perfor­
mance optimization issue, various core technologies including data
processing, feature acquisition, and modeling methods have been pro­
posed or introduced into ASC. And the computational complexity issue
can addressed from aspects of model complexity controlling (e.g., model
compression, model optimization) and the optimization of deployment
strategy.
Summarily, this article provides a comprehensive survey for ASC,
covering the fundamentals and practice till August 2023. We summarize
the development history of ASC. We also overview core technologies for
ASC and discuss their promise and limitations. Then, we study the series
of ASC resources and challenges that are useful information for re­
searchers and engineers to start the ASC research. Finally, we analyze
open challenges for ASC and conclude with several potential future
25
Expert Systems With Applications 238 (2024) 121902
directions. We hope this survey will give researchers a deeper under­
standing of various ASC technologies and the relationships among them,
as a springboard for future research.
CRediT authorship contribution statement
Biyun Ding: Conceptualization, Methodology, Software, Investiga­
tion, Validation, Writing – original draft, Writing – review & editing.
Tao Zhang: Conceptualization, Resources, Supervision, Project admin­
istration, Funding acquisition. Chao Wang: Methodology, Writing –
review & editing. Ganjun Liu: Methodology, Writing – review & edit­
ing. Jinhua Liang: Writing – review & editing. Ruimin Hu: Writing –
review & editing. Yulin Wu: Writing – review & editing. Difei Guo:
Writing – review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Acknowledgment
This work was supported by the National Natural Science Foundation
of China (grant number 62271344).
References
Abeßer, J. (2020). A review of deep learning based methods for acoustic scene
classification. Applied Sciences, 10(6). https://doi.org/10.3390/app10062020
Abidin, S., Togneri, R., & Sohel, F. (2017). In Enhanced LBP texture features from time
frequency representations for acoustic scene classification (pp. 626–630). IEEE.
Abrol, V., & Sharma, P. (2020). Learning hierarchy aware embedding from raw audio for
acoustic scene classification. IEEE/ACM Transactions on Audio, Speech, and Language
Processing, 28, 1964–1973.
Abu-El-Haija, S., Kothari, N., Lee J., Natsev, P., Toderici, G., Varadarajan, B., &
Vijayanarasimhan, S (2016). Youtube-8m: A large-scale video classification
benchmark. arXiv preprint arXiv: 1609.08675. http://research.google.com/
youtube8m/.
Agrawal, D. M., Sailor, H. B., Soni, M. H., & Patil, H. A. (2017). Novel TEO-based
Gammatone features for environmental sound classification. In In 2017 25th
European Signal Processing Conference (EUSIPCO) (pp. 1809–1813).
B. Ding et al.
Alamir, M. A. (2021). A novel acoustic scene classification model using the late fusion of
convolutional neural networks and different ensemble classifiers. Applied Acoustics,
175, Article 107829. https://doi.org/10.1016/j.apacoust.2020.107829
Arniriparian, S., Freitag, M., Cummins, N., Gerczuk, M., Pugachevskiy, S., & Schuller, B.
(2018). A fusion of deep convolutional generative adversarial networks and
sequence to sequence autoencoders for acoustic scene classification. In In 2018 26th
European Signal Processing Conference (EUSIPCO) (pp. 977–981).
Aryal, N., & Lee, S. W. (2023). Frequency-based CNN and attention module for acoustic
scene classification. Applied Acoustics, 210. https://doi.org/10.1016/j.
apacoust.2023.109411
Aucouturier, J. J., Defreville, B., & Pachet, F. (2007). The bag-of-frames approach to
audio pattern recognition: A sufficient model for urban soundscapes but not for
polyphonic music. The Journal of the Acoustical Society of America, 122(2), 881–891.
Aytar, Y., Vondrick, C., & Torralba, A. (2016). SoundNet: Learning sound representations
from unlabeled video. In 30th Annual Conference on Neural Information Processing
Systems (NIPS) (pp. 892–900).
Aziz, S., Awais, M., Akram, T., Khan, U., Alhussein, M., & Aurangzeb, K. (2019).
Automatic Scene Recognition through Acoustic Classification for Behavioral
Robotics. Electronics, 2019, 8(5), Article 483. https://doi.org/10.3390/
electronics8050483.
Baelde, M., Biernacki, C., & Greff, R. (2017). A mixture model-based real-time audio
sources classification method. In In 2017 IEEE International Conference on Acoustics,
Speech and Signal Processing (ICASSP) (pp. 2427–2431). https://doi.org/10.1109/
ICASSP.2017.7952592
Bahmei, B., Birmingham, E., & Arzanpour, S. (2022). CNN-RNN and Data Augmentation
Using Deep Convolutional Generative Adversarial Network for Environmental Sound
Classification. IEEE Signal Processing Letters, 29, 682–686.
Bai, X., Du, J., Pan, J., Zhou, H. S., Tu, Y. H., & Lee, C. H. (2020). High-resolution
attention network with acoustic segment model for acoustic scene classification. In
In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing
(ICASSP) (pp. 656–660). https://doi.org/10.1109/ICASSP40776.2020.9053519
Bai, X., Du, J., Wang, Z. R., & Lee, C. H. (2019). A Hybrid Approach to Acoustic Scene
Classification Based on Universal Acoustic Models. In Proc. Interspeech, 3619–3623.
https://doi.org/10.21437/Interspeech.2019-2171.
Barchiesi, D., Giannoulis, D., Stowell, D., & Plumbley, M. D. (2015). Acoustic scene
classification: Classifying environments from the sounds they produce. IEEE Signal
Processing Magazine, 32(3), 16–34. https://doi.org/10.1109/MSP.2014.2326181
Battaglino, D., Lepauloux, L., Pilati, L., & Evans, N. (2015). Acoustic context recognition
using local binary pattern codebooks. In In 2015 IEEE Workshop on Applications of
Signal Processing to Audio and Acoustics (WASPAA) (pp. 1–5). https://doi.org/
10.1109/WASPAA.2015.7336886
Bear, H. L., Nolasco, I., & Benetos, E. (2019). Towards joint sound scene and polyphonic
sound event recognition. In Proc. Interspeech (pp. 4594-4598). https://doi.org/
10.21437/Interspeech.2019-2169.
Beltrán, J., Chávez, E., & Favela, J. (2015). Scalable identification of mixed
environmental sounds, recorded from heterogeneous sources. Pattern Recognition
Letters, 68, 153–160.
Berland, A., Gaillard, P., Guidetti, M., & Barone, P. (2015). Perception of everyday
sounds: a developmental study of a free sorting task. PLoS One, 10(2), Article
e0115557. https://doi.org/10.1371/journal.pone.0115557.
Bisot, V., Essid, S., & Richard, G. (2015). HOG and subband power distribution image
features for acoustic scene classification. In In 2015 23rd European Signal Processing
Conference (EUSIPCO) (pp. 719–723). https://doi.org/10.1109/
EUSIPCO.2015.7362477
Bisot, V., Serizel, R., Essid, S., & Richard, G. (2016). In Acoustic scene classification with
matrix factorization for unsupervised feature learning (pp. 6445–6449). IEEE.
Bisot, V., Serizel, R., Essid, S., & Richard, G. (2017a). Feature learning with matrix
factorization applied to acoustic scene classification. IEEE/ACM Transactions on
Audio, Speech, and Language Processing, 25(6), 1216–1229.
Bisot, V., Serizel, R., Essid, S., & Richard, G. (2017b). Nonnegative feature learning
methods for acoustic scene classification. In In Proc. DCASE 2017 Workshop (pp.
1142–1158).
Bregman, A. S. (1990). Auditory scene analysis: The perceptual organization of sound ((1st
ed.),, p. 1).). The MIT Press (Chapter.
Brezina, P., & Jeseniˇcov́ a, S. (2018). Sound recording technologies and music education.
Ad Alta: Journal of Interdisciplinary Research, 8(2), 13–18.
Brown, A. L., Kang, J., & Gjestland, T. (2011). Towards standardization in soundscape
preference assessment. Applied acoustics, 72(6), 387–392. https://doi.org/10.1016/j.
apacoust.2011.01.001
Brown, G. J., & Cooke, M. (1994). Computational auditory scene analysis. Computer
Speech & Language, 8(4), 297–336.
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006). Model Compression. In In
Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery
and data mining (pp. 535–541). https://doi.org/10.1145/1150402.1150464
Byttebier, L., Desplanques, B., Thienpondt, J., Song, S., Demuynck, K., & Madhu, N.
(2021). Small-Footprint acoustic scene classification through 8-Bit Quantization-
Aware training and pruning of ResNet models. Tech. Rep. DCASE2021 Challenge.
Cao, B., Wang, N., Li, J., & Gao, X. (2019). Data Augmentation-Based Joint Learning for
Heterogeneous Face Recognition. IEEE Transactions on Neural Networks and Learning
Systems, 30(6), 1731–1743.
Carey, M. J., Parris, E. S., & Lloyd-Thomas, H. (1999). A comparison of features for
speech music discrimination. In 1999 IEEE International Conference on Acoustics,
Speech, and Signal Processing (ICASSP) (pp. 149–152). https://doi.org/10.1109/
ICASSP.1999.758084
Caruana, R. (1998). Multitask Learning. Autonomous Agents and Multi-Agent Systems, 27
(1), 95–133.
26
Expert Systems With Applications 238 (2024) 121902
Casey, M. (2001). General sound classification and similarity in MPEG-7. Organised
Sound, 6(2), 153–164.
Chachada, S., & Kuo, C. C. J. (2014). Environmental sound recognition: A survey.
APSIPA Transactions on Signal and Information Processing, 3(1), Article e14.
https://doi.org/10.1017/ATSIP.2014.12.
Chandrakala, S., & Jayalakshmi, S. L. (2019). Environmental audio scene and sound
event recognition for autonomous surveillance: A survey and comparative studies.
ACM Computing Surveys, 52(3), 1–34. https://doi.org/10.1145/3322240
Chang, Y., Wu, X., Zhang, S., & Yan, J. (2018). Ternary weighted networks with equal
quantization levels. In 25th Asia-Pacific Conference on Communications (APCC) (pp.
126–130). https://doi.org/10.1109/APCC47188.2019.9026483
Chen, C., Wang, M., & Zhang, P. (2022). Audio-visual scene classification using a transfer
learning based joint optimization strategy. arXiv preprint arXiv: 2204.11420.
Chen, H., Liu, Z., Liu, Z., Zhang, P., & Yan, Y. (2019). Integrating the data augmentation
scheme with various classifiers for acoustic scene modeling. Tech. Rep. DCASE2019
Challenge.
Chen, H., Zhang, P., & Yan, Y. (2019). An audio scene classification framework with
embedded filters and a DCT-Based temporal module. In 2019 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 835–839).
https://doi.org/10.1109/ICASSP.2019.8683636
Chen, H., Liu, Z., Liu, Z., & Zhang, P. (2021). Long-term scalogram integrated with an
iterative data augmentation scheme for acoustic scene classification. The Journal of
the Acoustical Society of America, 149(6), 4198–4213.
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018). Model compression and acceleration
for deep neural networks: The principles, progress, and challenges. IEEE Signal
Processing Magazine, 35(1), 126–136.
Chollet, F. (2017). In Xception: Deep learning with depthwise separable convolutions (pp.
1251–1258). IEEE Computer Society.
Chorowski, J. K., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015). Attention-
based models for speech recognition. Advances in Neural Information Processing
Systems, 28, 577–585.
Chu, S., Narayanan, S., & Kuo, C. C. J. (2009). Environmental sound recognition with
time-frequency audio features. IEEE Transactions on Audio, Speech, and Language
Processing, 17(6), 1142–1158.
Chu, S., Narayanan, S., Kuo, C. C. J., & Mataric, M. J. (2006). Where am I? Scene
recognition for mobile robots using audio features. In 2006 IEEE International
conference on multimedia and expo (pp. 885–888). https://doi.org/10.1109/
ICME.2006.262661
Clarkson, B., & Pentland, A. (1999). Unsupervised clustering of ambulatory audio and
video. In 1999 IEEE International Conference on Acoustics, Speech, and Signal
Processing. Proceedings (ICASSP) (Vol. 6, pp. 3037-3040). IEEE, https://doi.org/
10.1109/ICASSP.1999.757481.
Clarkson, B., Sawhney, N., & Pentland, A. (1998). Auditory context awareness via
wearable computing. In Proceedings of The 1998 Workshop On Perceptual User
Interfaces (pp. 1–6).
Courbariaux, M., Bengio, Y., & David, J. P. (2015). Binaryconnect: Training deep neural
networks with binary weights during propagations. In Proceedings of the 28th
International Conference on Neural Information Processing Systems, 2, 3123-3131.
Couvreur, C., Fontaine, V., Gaunard, P., & Mubikangiey, C. G. (1998). Automatic
classification of environmental noise events by hidden Markov models. Applied
Acoustics, 54(3), 187–206.
Cramer, J., Wu, H. H., Salamon, J., & Bello, J. P. (2019). Look, listen, and learn more:
Design choices for deep audio embeddings. In 2019 IEEE International Conference on
Acoustics, Speech and Signal Processing (ICASSP) (pp. 3852–3856). IEEE. https://doi.
org/10.1109/ICASSP.2019.8682475.
Cui, X., Goel, V., & Kingsbury, B. (2015). Data augmentation for deep neural network
acoustic modeling. IEEE/ACM Transactions on Audio, Speech, and Language Processing,
23(9), 1469–1477.
Dai, W., Dai, C., Qu, S., Li, J., & Das, S. (2017). Very deep convolutional neural networks
for raw waveforms. In 2017 IEEE international conference on acoustics, speech and
signal processing (ICASSP) (pp. 421–425). IEEE. https://doi.org/10.1109/
ICASSP.2017.7952190.
Dandashi, A., & AlJaam, J. (2017). A survey on audio content-based classification. In
2017 International conference on computational science and computational intelligence
(CSCI) (pp. 408–413). IEEE. https://doi.org/10.1109/CSCI.2017.69.
Deng, Z., Zhou, K., Li, D., He, J., Song, Y. Z., & Xiang, T. (2022). Dynamic instance
domain adaptation. IEEE Transactions on Image Processing, 31, 4585–4597. https://
doi.org/10.1109/TIP.2022.3186531
Dennis, J., Tran, H. D., & Chng, E. S. (2013). Image feature representation of the subband
power distribution for robust sound event classification. IEEE Transactions on Audio,
Speech, and Language Processing, 21(2), 367–377.
Devalraju, D. V., & Rajan, P. (2022). Multiview Embeddings for Soundscape
Classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30,
1197–1206.
Ding, B., Zhang, T., Liu, G., Kong, L., & Geng, Y. (2022). Late fusion for acoustic scene
classification using swarm intelligence. Applied Acoustics, 192, Article 108698.
https://doi.org/10.1016/j.apacoust.2022.108698
Dong, X., Yan, Y., Tan, M., Yang, Y., & Tsang, I. W. (2018). Late fusion via subspace
search with consistency preservation. IEEE Transactions on Image Processing, 28(1),
518–528. https://doi.org/10.1109/TIP.2018.2867747
Dorfer, M., Lehner, B., Eghbal-zadeh, H., Christop, H., Fabian, P., & Gerhard, W. (2018).
Acoustic scene classification with fully convolutional neural networks and i-vectors.
Tech. Rep. DCASE2018 Challenge.
Drossos, K., Magron, P., & Virtanen, T. (2019). Unsupervised adversarial domain
adaptation based on the wasserstein distance for acoustic scene classification. In
B. Ding et al.
2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics
(WASPAA) (pp. 259–263). IEEE. https://doi.org/10.1109/WASPAA.2019.8937231.
Drossos, K., Mimilakis, S. I., Gharib, S., Li, Y., & Virtanen, T. (2020). Sound event
detection with depthwise separable and dilated convolutions. In 2020 International
Joint Conference on Neural Networks (IJCNN) (pp. 1–7). IEEE. https://doi.org/
10.1109/IJCNN48605.2020.9207532.
Droumeva, M. (2005). Understanding immersive audio: a historical and socio-cultural
exploration of auditory displays. In International Conference on Auditory Display
(ICAD) (pp. 162-168).
Dubois, D., Guastavino, C., & Raimbault, M. (2006). A cognitive approach to urban
soundscapes: Using verbal data to access everyday life auditory categories. Acta
Acustica United with Acustica, 92(6), 865–874.
Dwyer, R. (1983). Detection of non-Gaussian signals by frequency domain Kurtosis
estimation. In IEEE International Conference on Acoustics, Speech, and Signal Processing
(ICASSP) (pp. 607–610). IEEE. https://doi.org/10.1109/ICASSP.1983.1172264.
Ebbers, J., Keyser, M. C., & Haeb-Umbach, R. (2021). Adapting sound recognition to a
new environment via self-training. In 2021 29th European Signal Processing
Conference (EUSIPCO) (pp. 1135–1139). IEEE.
Eghbal-Zadeh, H., Lehner, B., Dorfer, M., & Widmer, G. (2016). CP-JKU submissions for
DCASE-2016: a hybrid approach using binaural i-vectors and deep convolutional
neural networks. Tech. Rep. DCASE2016 Challenge.
Eghbal-zadeh, H., Lehner, B., Dorfer, M., & Widmer, G. (2017). A hybrid approach with
multi-channel i-vectors and convolutional neural networks for acoustic scene
classification. In 2017 25th European Signal Processing Conference (EUSIPCO) (pp.
2749–2753). IEEE.
Ellis, D. P. W. (1996). Pediction-driven computational auditory scene analysis [Doctoral
dissertation]. Doctoral dissertation, Columbia University. https://doi.org/10.7916/
D84J0N13
El-Maleh, K., Samouelian, A., & Kabal, P. (1999). Frame level noise classification in
mobile environments. In 1999 IEEE International Conference on Acoustics, Speech,
and Signal Processing (ICASSP), 1, 237-240. IEEE, https://doi.org/10.1109/
ICASSP.1999.758106.
Eronen, A. J., Peltonen, V. T., Tuomi, J. T., Klapuri, A. P., Fagerlund, S., Sorsa, T., et al.
(2006). Audio-based context recognition. IEEE Transactions on Audio, Speech, and
Language Processing, 14(1), 321–329.
Eronen, A., Tuomi, J., Klapuri, A., Fagerlund, S., Sorsa, T., Lorho, G., et al. (2003). Audio-
based context awareness—Acoustic modeling and perceptual evaluation. In 2003
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) (pp.
529–532). IEEE. https://doi.org/10.1109/ICASSP.2003.1200023.
Commision, E. (2014). EAR-IT: Using sound to picture the world in a new way. Retrieved
from https://digital-strategy.ec.europa.eu/en/news/ear-it-using-sound-picture-
world-new-way. Accessed December 1, 2022.
Eyben, F., Wöllmer, M., & Schuller, B. (2010). OpenSMILE: The munich versatile and fast
open-source audio feature extractor. In Proceedings of the 18th ACM international
conference on Multimedia (pp. 1459-1462).
Foote, J. (1999). An overview of audio information retrieval. Multimedia Systems, 7(1),
2–10.
Foster, P., Sigtia, S., Krstulovic, S., Barker, J., & Plumbley, M. D. (2015). Chime-home: A
dataset for sound source recognition in a domestic environment. In 2015 IEEE
Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) (pp.
1–5). IEEE. https://doi.org/10.1109/WASPAA.2015.7336899.
Gaver, W. W. (1993). What in the world do we hear? An ecological approach to auditory
event perception. Ecological Psychology, 5(1), 1–29. https://doi.org/10.1207/
s15326969eco0501_1
Geiger, J. T., Schuller, B., & Rigoll, G. (2013). Large-scale audio feature extraction and
SVM for acoustic scene classification. In 2013 IEEE Workshop on Applications of Signal
Processing to Audio and Acoustics (pp. 1–4). IEEE. https://doi.org/10.1109/
WASPAA.2013.6701857.
Gemmeke, J. F., Ellis, D. P., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., et al.
(2017). Audio set: An ontology and human-labeled dataset for audio events. In 2017
IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp.
776–780). IEEE. https://doi.org/10.1109/ICASSP.2017.7952261.
Gerhard, D. (2003). Audio signal classification: History and current techniques. Regina, SK,
Canada: Department of Computer Science, University of Regina.
Gharib, S., Derrar, H., Niizumi, D., Senttula, T., Tommola, J., Heittola, T., et al. (2018).
Acoustic scene classification: A competition review. In 28th International Workshop
on Machine Learning for Signal Processing (MLSP) (pp. 1–6). IEEE. https://doi.org/
10.1109/MLSP.2018.8517000.
Gharib, S., Drossos, K., Cakir, E., Serdyuk, D., & Virtanen, T. (2018b). Unsupervised
Adversarial Domain Adaptation for Acoustic Scene Classification. In Proc.
DCASE2018 Workshop (pp. 1-5).
Giannoulis, D., Benetos, E., Stowell, D., Rossignol, M., Lagrange, M., & Plumbley, M. D.
(2013). Detection and classification of acoustic scenes and events: An IEEE AASP
challenge. In 2013 IEEE Workshop on Applications of Signal Processing to Audio and
Acoustics (pp. 1–4). IEEE. https://doi.org/10.1109/WASPAA.2013.6701819.
Giannoulis, D., Stowell, D., Benetos, E., Rossignol, M., Lagrange, M., & Plumbley, M. D.
(2013). A database and challenge for acoustic scene classification and event
detection. In 21st European Signal Processing Conference (EUSIPCO) (pp. 1–5). IEEE.
Giordano, B. L., McDonnell, J., & McAdams, S. (2010). Hearing living symbols and
nonliving icons: Category specificities in the cognitive processing of environmental
sounds. Brain and Cognition, 73(1), 7–19. https://doi.org/10.1016/j.
bandc.2010.01.005
Gong, Y., Chung, Y. A., & Glass, J. (2021). AST: Audio spectrogram transformer. In Proc.
Interspeech (pp. 571–575). https://doi.org/10.21437/Interspeech.2021-698.
27
Expert Systems With Applications 238 (2024) 121902
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S.,
Courville, A. & Bengio, Y. (2014). Generative Adversarial Nets. In Proceedings of the
27th International Conference on Neural Information Processing Systems, 2, 2672-2680.
Götz, P., Tuna, C., Walther, A., & Habets, E. A. (2023). Contrastive Representation
Learning for Acoustic Parameter Estimation. In 2023 IEEE International Conference on
Acoustics, Speech and Signal Processing (ICASSP) (pp. 1–5). IEEE. https://doi.org/
10.1109/ICASSP49357.2023.10095279.
Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021). Knowledge Distillation: A Survey.
International Journal of Computer Vision, 129(6), 1789–1819. https://doi.org/
10.1007/s11263-021-01453-z
Goyal, A., Shukla, S. K., & Sarin, R. K. (2019). Identification of source mobile hand sets
using audio latency feature. Forensic Science International, 298, 332–335.
Grollmisch, S., & Cano, E. (2021). Improving semi-supervised learning for audio
classification with FixMatch. Electronics, 10(15), 1807.
Guastavino, C. (2007). Categorization of environmental sounds. Canadian Journal of
Experimental Psychology/Revue canadienne de psychologie expérimentale, 61(1), 54–63.
https://psycnet.apa.org/doi/10.1037/cjep2007006.
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural
networks. In International conference on machine learning (ICML) (pp. 1321–1330).
PMLR.
Guo, M. H., Xu, T. X., Liu, J. J., Liu, Z. N., Jiang, P. T., Mu, T. J., Zhang, S. H.,
Martin, R. R., Cheng, M. M., & Hu, S. M. (2022). Attention mechanisms in computer
vision: A survey. Computational Visual Media, 8, 331–368.
Gupta, S., Agrawal, A., Gopalakrishnan, K., & Narayanan, P. (2015). Deep learning with
limited numerical precision. In International conference on machine learning (pp.
1737–1746). PMLR.
Guzhov, A., Raue, F., Hees, J., & Dengel, A. (2021). ESResNet: Environmental Sound
Classification Based on Visual Domain Models. In 2020 25th International Conference
on Pattern Recognition (ICPR) (pp. 4933–4940). IEEE. https://doi.org/10.1109/
ICPR48806.2021.9413035.
Gygi, B., Kidd, G. R., & Watson, C. S. (2007). Similarity and categorization of
environmental sounds. Perception & Psychophysics, 69(6), 839–855. https://doi.org/
10.3758/BF03193921
Hajihashemi, V., Gharahbagh, A. A., Cruz, P. M., Ferreira, M. C., Machado, J. J., &
Tavares, J. M. R. (2022). Binaural Acoustic Scene Classification Using Wavelet
Scattering, Parallel Ensemble Classifiers and Nonlinear Fusion. Sensors, 22(4), Article
1535.
Han, J., Matuszewski, M., Sikorski, O., Sung, H., & Cho, H. (2023). Randmasking
Augment: A Simple and Randomized Data Augmentation For Acoustic Scene
Classification. In 2023 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP) (pp. 1–5). IEEE. https://doi.org/10.1109/
ICASSP49357.2023.10095001.
Han, S., Mao, H., & Dally, W. J. (2016). Deep compression: Compressing deep neural
networks with pruning, trained quantization and Huffman coding. In 4th
International Conference on Learning Representations (ICLR).
Han, Y., Park, J., & Lee, K. (2017). Convolutional neural networks with binaural
representations and background subtraction for acoustic scene classification. In Proc.
DCASE2017 Workshop (pp. 46-50).
Hasan, N. W., Saudi, A. S., Khalil, M. I., & Abbas, H. M. (2023). A Genetic Algorithm
Approach to Automate Architecture Design for Acoustic Scene Classification. IEEE
Transactions on Evolutionary Computation, 27(2), 222–236. https://doi.org/10.1109/
TEVC.2022.3185543
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition.
In Proceedings of the IEEE conference on computer vision and pattern recognition (pp.
770-778).
He, N., & Zhu, J. (2021). A weighted partial domain adaptation for acoustic scene
classification and its application in fiber optic security system. IEEE Access, 9,
2244–2250. https://doi.org/10.1109/ACCESS.2020.3044153
Heittola, T., Mesaros, A., & Virtanen, T. (2020). Acoustic scene classification in DCASE
2020 challenge: generalization across devices and low complexity solutions. In Proc.
DCASE2020 Workshop (pp. 56-60).
Heo, H. S., Jung, J. W., Shim, H. J., & Yu, H. J. (2019). Acoustic scene classification using
teacher-student learning with soft-labels. In Proc. Interspeech (pp. 614-618). https://
doi.org/10.21437/Interspeech.2019-1989.
Hershey, S., Chaudhuri, S., Ellis, D. P., Gemmeke, J. F., Jansen, A., Moore, R. C., et al.
(2017). CNN architectures for large-scale audio classification. In 2017 IEEE
international conference on acoustics, speech and signal processing (ICASSP) (pp.
131–135). IEEE. https://doi.org/10.1109/ICASSP.2017.7952132.
Houix, O., Lemaitre, G., Misdariis, N., Susini, P., & Urdapilleta, I. (2012). A lexical
analysis of environmental sound categories. Journal of Experimental Psychology:
Applied, 18(1), 52. https://psycnet.apa.org/doi/10.1037/a0026240.
Hu, H., Yang, C. H. H., Xia, X., Bai, X., Tang, X., Wang, Y., Niu, S., Chai, L., Li, J., Zhu, H.,
Bao, F., Zhao, Y., Siniscalchi, S. M., Wang, Y., Du, J. & Lee, C. H. (2020). Device-
robust acoustic scene classification based on two-stage categorization and data
augmentation. Tech. Rep. DCASE2020 Challenge.
Hu, H., Yang, C. H. H., Xia, X., Bai, X., Tang, X., Wang, Y., et al. (2021). A two-stage
approach to device-robust acoustic scene classification. In 2021 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 845–849). IEEE.
https://doi.org/10.1109/ICASSP39728.2021.9414835.
Huang, J., Gretton, A., Borgwardt, K., Schölkopf, B., & Smola, A. (2006). Correcting
sample selection bias by unlabeled data. Advances in Neural Information Processing
Systems, 19, 601–608.
Huang, J., Lu, H., Meyer, P., Cordourier, H., & Ontiveros, J. (2019). Acoustic scene
classification using deep learning-based ensemble averaging. In Proc. DCASE2019
Workshop (pp. 94-98).
B. Ding et al.
Huang, Y., Zhu, Y., Fan, X., Ma, X., Wang, F., Liu, J., et al. (2018). Task scheduling with
optimized transmission time in collaborative cloud-edge learning. In 27th
International Conference on Computer Communication and Networks (ICCCN) (pp. 1–9).
IEEE. https://doi.org/10.1109/ICCCN.2018.8487352.
Huang, Z., Liu, C., Fei, H., Li, W., Yu, J., & Cao, Y. (2020). Urban sound classification
based on 2-order dense convolutional network using dual features. Applied Acoustics,
164, Article 107243.
Hüwel, A., Adiloğlu, K., & Bach, J. H. (2020). Hearing aid research data set for acoustic
environment recognition. In 2020 IEEE International Conference on Acoustics, Speech
and Signal Processing (ICASSP) (pp. 706–710). IEEE. https://doi.org/10.1109/
ICASSP40776.2020.9053611.
Imoto, K. (2021). Acoustic Scene Classification Using Multichannel Observation with
Partially Missing Channels. In 2021 29th European Signal Processing Conference
(EUSIPCO) (pp. 875–879). IEEE.
Imoto, K., & Ono, N. (2017). Spatial Cepstrum as a Spatial Feature Using a Distributed
Microphone Array for Acoustic Scene Analysis. IEEE/ACM Transactions on Audio,
Speech, and Language Processing, 25(6), 1335–1343.
Imoto, K., Tonami, N., Koizumi, Y., Yasuda, M., Yamanishi, R., & Yamashita, Y. (2020).
Sound event detection by multitask learning of sound events and scenes with soft
scene labels. In 2020 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP) (pp. 621–625). IEEE. https://doi.org/10.1109/
ICASSP40776.2020.9053912.
Jaiswal, A., Babu, A. R., Zadeh, M. Z., Banerjee, D., & Makedon, F. (2021). A survey on
contrastive self-supervised learning. Technologies, 2020, 9(1), Article 2.
Jansen, A., Gemmeke, J. F., Ellis, D. P., Liu, X., Lawrence, W., & Freedman, D. (2017).
Large-scale audio event discovery in one million youtube videos. In 2017 IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp.
786–790). IEEE. https://doi.org/10.1109/ICASSP.2017.7952263.
Jati, A., Nadarajan, A., Peri, R., Mundnich, K., Feng, T., Girault, B., et al. (2021).
Temporal dynamics of workplace acoustic scenes: Egocentric analysis and
prediction. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29,
756–769. https://doi.org/10.1109/TASLP.2021.3050265
Jiang, G., Ma, Z., Mao, Q., & Zhang, J. (2023). Multi-level distance embedding learning
for robust acoustic scene classification with unseen devices. Pattern Analysis and
Applications, 26, 1089–1099.
Jiang, J., & Zhai, C. X. (2007). Instance weighting for domain adaptation in NLP. In 45th
Annual Meeting of the Association for Computational Linguistics, ACL 2007 (pp. 264-
271).
Jiang, S., Shi, C., & Li, H. (2019). Acoustic scene classification using ensembles of
convolutional neural networks and spectrogram decompositions. In Tech. Rep.
DCASE2019 Challenge Task1.
Jing, L., & Tian, Y. (2021). Self-supervised visual feature learning with deep neural
networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43
(11), 4037–4058.
Jung, J. W., Shim, H. J., Kim, J. H., & Yu, H. J. (2021). DCASENet: An integrated
pretrained deep neural network for detecting and classifying acoustic scenes and
events. In 2021 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP) (pp. 621–625). IEEE. https://doi.org/10.1109/
ICASSP39728.2021.9414406.
Kacprzak, S., & Kowalczyk, K. (2021). Adversarial Domain Adaptation with Paired
Examples for Acoustic Scene Classification on Different Recording Devices. In 2021
29th European Signal Processing Conference (EUSIPCO) (pp. 1030–1034). IEEE.
Kala, T., & Shinozaki, T. (2018). Reinforcement learning of speech recognition system
based on policy gradient and hypothesis selection. In 2018 IEEE international
conference on acoustics, speech and signal processing (ICASSP) (pp. 5759–5763). IEEE.
https://doi.org/10.1109/ICASSP.2018.8462656.
Kang, Z., He, Y., Wang, J., Peng, J., Qu, X., & Xiao, J. (2023). Feature-Rich Audio Model
Inversion for Data-Free Knowledge Distillation Towards General Sound
Classification. In 2023 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP) (pp. 1–5). IEEE. https://doi.org/10.1109/
ICASSP49357.2023.10096079.
Kawamoto, M., & Hamamoto, T. (2020). Building health monitoring using computational
auditory scene analysis. In 2020 16th International Conference on Distributed
Computing in Sensor Systems (DCOSS) (pp. 144–146). IEEE. https://doi.org/10.1109/
DCOSS49796.2020.00033.
Kawamura, T., Kinoshita, Y., Ono, N., & Scheibler, R. (2023). Effectiveness of Inter- and
Intra-Subarray Spatial Features for Acoustic Scene Classification. In 2023 IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1–5).
IEEE. https://doi.org/10.1109/ICASSP49357.2023.10096935.
Kim, B., Yang, S., Kim, J., & Chang, S. (2021). QTI submission to DCASE 2021: Residual
normalization for device-imbalanced acoustic scene classification with efficient
design. Tech. Rep. DCASE2021 Challenge.
Kim, J., Hyun, M., Chung, I., & Kwak, N. (2021). Feature fusion for online mutual
knowledge distillation. In 2020 25th International Conference on Pattern Recognition
(ICPR) (pp. 4619–4625). IEEE. https://doi.org/10.1109/ICPR48806.2021.9412615.
Kim, J., Yoo, K. Y., & Kwak, N. (2020). Position-based scaled gradient for model
quantization and pruning. Advances in Neural Information Processing Systems, 33,
20415–20426.
Komatsu, T., Imoto, K., & Togami, M. (2020). Scene-dependent acoustic event detection
with scene conditioning and fake-scene-conditioned loss. In 2020 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 646–650). IEEE.
https://doi.org/10.1109/ICASSP40776.2020.9053702.
Kong, Q., Cao, Y., Iqbal, T., Wang, Y., Wang, W., & Plumbley, M. D. (2020). PANNs:
Large-scale pretrained audio neural networks for audio pattern recognition. IEEE/
ACM Transactions on Audio, Speech, and Language Processing, 28, 2880–2894.
28
Expert Systems With Applications 238 (2024) 121902
Kosmider, M. (2020). Spectrum correction: Acoustic scene classification with
mismatched recording devices. In Proc. Interspeech 2020 (pp. 4641-4645). https://
doi.org/10.21437/Interspeech.2020-3088.
Kosmider, M. (2019). Calibrating neural networks for secondary recording devices. Tech.
Rep. DCASE2019 Challenge.
Koutini, K., Eghbal-zadeh, H., & Widmer, G. (2019). Receptive-field-regularized CNN
variants for acoustic scene classification. In Proc. DCASE2019 Workshop (pp. 124-
128). https://doi.org/10.33682/cjd9-kc43.
Koutini, K., Eghbal-zadeh, H., Widmer, G., & Kepler, J. (2019). CP-JKU submissions to
DCASE’19: acoustic scene classification and audio tagging with receptive-field-
regularized CNNs. Tech. Rep. DCASE2019 Challenge.
Koutini, K., Henkel, F., Eghbal-zadeh, H., & Widmer, G. (2020). CP-JKU Submissions to
DCASE’20: Low-Complexity Cross-Device Acoustic Scene Classification with RF-
Regularized CNNs. Tech. Rep. DCASE2020 Challenge.
Koutini, K., Jan, S., & Widmer, G. (2021). CPJKU submission to DCASE21: cross-device
audio scene classification with wide sparse frequency-damped CNNs. Tech. Rep.
DCASE2021 Challenge.
Kroos, C., Bones, O., Cao, Y., Harris, L., Jackson, P. J., Davies, W. J., et al. (2019).
Generalisation in environmental sound classification: The making sense of sounds
data set and challenge. In 2019 IEEE International Conference on Acoustics, Speech and
Signal Processing (ICASSP) (pp. 8082–8086). IEEE. https://doi.org/10.1109/
ICASSP.2019.8683292.
Kumar, A., Khadkevich, M., & Fügen, C. (2018). Knowledge transfer from weakly labeled
audio using convolutional neural network for sound events and scenes. In 2018 IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp.
326–330). IEEE. https://doi.org/10.1109/ICASSP.2018.8462200.
Kumar, T. V., Sundar, R. S., Purohit, T., & Ramasubramanian, V. (2020). End-to-end
audio-scene classification from raw audio: Multi time-frequency resolution CNN
architecture for efficient representation learning. In International Conference on Signal
Processing and Communications (SPCOM) (pp. 1–5). IEEE. https://doi.org/10.1109/
SPCOM50965.2020.9179600.
Kuncheva, L. I. (2004). Combining Pattern Classifiers: Methods and Algorithms (1st ed.).
Wiley-Interscience (Chapter 4).
Lasseck, M. (2018). Acoustic bird detection with deep convolutional neural networks. In
Proc. DCASE2018 Workshop (pp. 143-147).
Lee, K., Hyung, Z., & Nam, J. (2013). Acoustic scene classification using sparse feature
learning and event-based pooling. In 2013 IEEE Workshop on Applications of Signal
Processing to Audio and Acoustics (pp. 1–4). IEEE. https://doi.org/10.1109/
WASPAA.2013.6701893.
Lee, Y., Lim, S., & Kwak, I. Y. (2021). CNN-based acoustic scene classification system.
Electronics, 10(4), Article 371. https://doi.org/10.3390/electronics10040371.
Lehner, B., Eghbal-Zadeh, H., Dorfer, M., Korzeniowski, F., Koutini, K., & Widmer, G.
(2017). Classifying short acoustic scenes with i-vectors and CNNs: Challenges and
optimisations for the 2017 DCASE ASC task. Tech. Rep. DCASE2017 Challenge.
Lemaitre, G., Houix, O., Misdariis, N., & Susini, P. (2010). Listener expertise and sound
identification influence the categorization of environmental sounds. Journal of
Experimental Psychology: Applied, 16(1), 16–32. https://psycnet.apa.org/doi/10.1037
/a0018762.
Leng, Y., Zhao, W., Lin, C., Sun, C., Wang, R., Yuan, Q., et al. (2020). LDA-based data
augmentation algorithm for acoustic scene classification. Knowledge-Based Systems,
195, Article 105600.
Li, D., Sethi, I. K., Dimitrova, N., & McGee, T. (2001). Classification of general audio data
for content-based retrieval. Pattern Recognition Letters, 22(5), 533–544.
Li, G., Liu, L., Wang, X., Dong, X., Zhao, P., & Feng, X. (2018). In Auto-tuning neural
network quantization framework for collaborative inference between the cloud and edge
(pp. 402–411). Cham: Springer.
Li, J., Dai, W., Metze, F., Qu, S., & Das, S. (2017). A comparison of deep learning methods
for environmental sound detection. In 2017 IEEE International conference on acoustics,
speech and signal processing (ICASSP) (pp. 126–130). https://doi.org/10.1109/
ICASSP.2017.7952131
Li, S., Gu, Y., Luo, Y., Chambers, J., & Wang, W. (2019). Enhanced streaming based
subspace clustering applied to acoustic scene data clustering. In 2019 IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp.
11–15). https://doi.org/10.1109/ICASSP.2019.8682593
Li, S., & Wang, W. (2018). In Randomly sketched sparse subspace clustering for acoustic
scene clustering (pp. 2489–2493). IEEE.
Li, Y., Li, X., Zhang, Y., Wang, W., Liu, M., & Feng, X. (2018). Acoustic scene
classification using deep audio feature and BLSTM network. In 2018 International
Conference on Audio, Language and Image Processing (ICALIP) (pp. 371–374). https://
doi.org/10.1109/ICALIP.2018.8455765
Li, Y., Liu, M., Drossos, K., & Virtanen, T. (2020). Sound event detection via dilated
convolutional recurrent neural networks. In 2020 IEEE International Conference on
Acoustics, Speech and Signal Processing (ICASSP) (pp. 286–290). https://doi.org/
10.1109/ICASSP40776.2020.9054433
Li, Y., Liu, M., Wang, W., Zhang, Y., & He, Q. (2019). Acoustic scene clustering using
joint optimization of deep embedding learning and clustering iteration. IEEE
Transactions on Multimedia, 22(6), 1385–1394.
Li, Z., Hou, Y., Xie, X., Li, S., Zhang, L., Du, S., et al. (2019). Multi-level attention model
with deep scattering spectrum for acoustic scene classification. In 2019 IEEE
International Conference on Multimedia and Expo Workshops (ICMEW) (pp. 396–401).
https://doi.org/10.1109/ICMEW.2019.00074
Liang, J., Zhang, T., & Feng, G. (2020). Channel compression: Rethinking information
redundancy among channels in CNN architecture. IEEE Access, 8, 147265–147274.
Lipton, Z. C. (2018). The mythos of model interpretability. Communications of the ACM,
61(10), 35–43.
B. Ding et al.
Liu, Y., Jiang, S., Shi, C., & Li, H. (2019). Acoustic Scene Classification Using Ensembles
of Deep Residual Networks and Spectrogram Decompositions. In Proc. DCASE2019
Workshop (pp. 25–26).
Long, M., Wang, J., Cao, Y., Sun, J., & Philip, S. Y. (2016). Deep learning of transferable
representation for scalable domain adaptation. IEEE Transactions on Knowledge and
Data Engineering, 28(8), 2027–2040.
Lopez-Meyer, P., del Hoyo Ontiveros, J. A., Lu, H., & Stemmer, G. (2021). Efficient end-
to-end audio embeddings generation for audio classification on target applications.
In 2021 IEEE International Conference on Acoustics, Speech and Signal Processing
(ICASSP) (pp. 601–605). https://doi.org/10.1109/ICASSP39728.2021.9414229
Lostanlen, V., Salamon, J., Cartwright, M., McFee, B., Farnsworth, A., Kelling, S., et al.
(2018). Per-channel energy normalization: Why and how. IEEE Signal Processing
Letters, 26(1), 39–43.
Lu, J., Ma, R., Liu, G., & Qin, Z. (2021). Deep convolutional neural network with transfer
learning for environmental sound classification. In 2021 International Conference on
Computer, Control and Robotics (ICCCR) (pp. 242–245). https://doi.org/10.1109/
ICCCR49711.2021.9349393
Madhu, A., & Suresh, K. (2023). RQNet: Residual Quaternion CNN for Performance
Enhancement in Low Complexity and Device Robust Acoustic Scene Classification.
IEEE Transactions on Multimedia. https://doi.org/10.1109/TMM.2023.3241553
Madhu, A. (2022). EnvGAN: A GAN-based augmentation to improve environmental
sound classification. Artificial Intelligence Review, 55, 6301–6320.
Malkin, R. G., & Waibel, A. (2005). Classifying user environment for mobile applications
using linear autoencoding of ambient audio. In IEEE International Conference on
Acoustics, Speech, and Signal Processing (ICASSP) (pp. 509–512). https://doi.org/
10.1109/ICASSP.2005.1416352
Marcell, M. M., Borella, D., Greene, M., Kerr, E., & Rogers, S. (2000). Confrontation
naming of environmental sounds. Journal of Clinical and Experimental
Neuropsychology, 22(6), 830–864. https://doi.org/10.1076/jcen.22.6.830.949
Martin, K. D. (1999). Sound Source Recognition: A theory and computational model.
Massachusetts Institute of Technology [Doctoral dissertation]. Doctoral dissertation.
Martín-Morató, I., Heittola, T., Mesaros, A., & Virtanen, T. (2021). Low-complexity
acoustic scene classification for multi-device audio: Analysis of DCASE 2021
challenge systems. In Proc. DCASE2021 Workshop (pp. 85–89).
McDonnell, M. D., & Gao, W. (2020). Acoustic scene classification using deep residual
networks with late fusion of separated high and low frequency paths. In 2020 IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp.
141–145). https://doi.org/10.1109/ICASSP40776.2020.9053274
McDonnell, M., & UniSA, S. (2020). Low-complexity acoustic scene classification using
one-bit-per-weight deep convolutional neural networks. Tech. Rep. DCASE2020
Challenge.
Mesaros, A., Heittola, T., & Virtanen, T. (2016a). Metrics for polyphonic sound event
detection. Applied Sciences, 6(6), Article 162.
Mesaros, A., Heittola, T., & Virtanen, T. (2016b). TUT database for acoustic scene
classification and sound event detection. In 2016 24th European Signal Processing
Conference (EUSIPCO) (pp. 1128–1132). https://doi.org/10.1109/
EUSIPCO.2016.7760424
Mesaros, A., Heittola, T., & Virtanen, T. (2018a). Acoustic scene classification: An
overview of DCASE 2017 challenge entries. In 2018 16th International Workshop on
Acoustic Signal Enhancement (IWAENC) (pp. 411–415). https://doi.org/10.1109/
IWAENC.2018.8521242
Mesaros, A., Heittola, T., & Virtanen, T. (2018b). A multi-device dataset for urban
acoustic scene classification. In Proc., DCASE2018, 9–13.
Mesaros, A., Heittola, T., Benetos, E., Foster, P., Lagrange, M., Virtanen, T., et al. (2018).
Detection and Classification of Acoustic Scenes and Events: Outcome of the DCASE
2016 Challenge. IEEE/ACM Transactions on Audio, Speech, and Language Processing,
26(2), 379–393.
Mesaros, A., Heittola, T., Diment, A., et al. (2017). DCASE 2017 challenge setup: Tasks,
datasets and baseline system. In Proc. DCASE2017 Workshop (pp. 85–92).
Mesaros, A., Heittola, T., Virtanen, T., & Plumbley, M. D. (2021). Sound event detection:
A tutorial. IEEE Signal Processing Magazine, 38(5), 67–83. https://doi.org/10.1109/
MSP.2021.3090678
Mitrović, D., Zeppelzauer, M., & Breiteneder, C. (2010). Features for content-based audio
retrieval. Advances in Computers, 78, 71–150. https://doi.org/10.1016/S0065-2458
(10)78003-7
Mogi, R., & Kasai, H. (2013). Noise-Robust environmental sound classification method
based on combination of ICA and MP features. Artificial Intelligence Research, 2(1),
107–121.
Mohaimenuzzaman, M., Bergmeir, C., West, I., & Meyer, B. (2023). Environmental Sound
Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely
Resource-Constrained Devices. Pattern Recognition, 133, Article 109025.
Morocutti, T., Schmid, F., Koutini, K., & Widmer, G. (2023). Device-Robust Acoustic
Scene Classification via Impulse Response Augmentation. arXiv preprint arXiv:
2305.07499.
Muhammad, G., Alotaibi, Y. A., Alsulaiman, M., & Huda, M. N. (2010). Environment
Recognition Using Selected MPEG-7 Audio Features and Mel-Frequency Cepstral
Coefficients. In 2010 Fifth international conference on digital telecommunications (pp.
11–16). https://doi.org/10.1109/ICDT.2010.10
Mulimani, M., & Koolagudi, S. G. (2021). Acoustic scene classification using deep
learning architectures. In 2021 6th International Conference for Convergence in
Technology (I2CT) (pp. 1–6). https://doi.org/10.1109/I2CT51068.2021.9418177
Mun, S., Park, S., Han, D. K., & Ko, H. (2017a). Generative Adversarial Network Based
Acoustic Scene Training Set Augmentation and Selection Using SVM Hyper-Plane. In
Proc. DCASE2017 Workshop (pp. 93-102).
Mun, S., Shon, S., Kim, W., Han, D. K., & Ko, H. (2017). Deep neural network based
learning and transferring mid-level audio features for acoustic scene classification. In
29
Expert Systems With Applications 238 (2024) 121902
2017 IEEE international conference on acoustics, speech and signal processing (ICASSP)
(pp. 796–800). https://doi.org/10.1109/ICASSP.2017.7952265
National Science Foundation of China (NSFC). (2019). Major research of Cognitive
computing of visual and auditory information. Retrieved from https://www.nsfc.
gov.cn/publish/portal0/tab448/info75524.htm, Accessed December 1, 2022.
Nguyen, T., & Pernkopf, F. (2018). Acoustic scene classification using a convolutional
neural network ensemble and nearest neighbor filters. In Proc. DCASE2018 Workshop
(pp. 34–38).
Nguyen, T., & Pernkopf, F. (2019a). Acoustic Scene Classification with Mismatched
Devices Using CliqueNets and Mixup Data Augmentation. Proc. Interspeech,
2330–2334. https://doi.org/10.21437/Interspeech.2019-3002.
Nguyen, T., & Pernkopf, F. (2019b). Acoustic scene classification with mismatched
recording devices using mixture of experts layer. In 2019 IEEE International
Conference on Multimedia and Expo (ICME) (pp. 1666–1671). https://doi.org/
10.1109/ICME.2019.00287
Nguyen, T., Pernkopf, F., & Kosmider, M. (2020). Acoustic scene classification for
mismatched recording devices using heated-up softmax and spectrum correction. In
2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
(pp. 126–130). https://doi.org/10.1109/ICASSP40776.2020.9053582
Nwe, T. L., Dat, T. H., & Ma, B. (2017). Convolutional neural network with multi-task
learning scheme for acoustic scene classification. In 2017 Asia-Pacific Signal and
Information Processing Association Annual Summit and Conference (APSIPA ASC) (pp.
1347–1350).
Olvera, M., Vincent, E., & Gasso, G. (2022). On the impact of normalization strategies in
unsupervised adversarial domain adaptation for acoustic scene classification. In
2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
(pp. 631–635). https://doi.org/10.1109/ICASSP43922.2022.9747540
Ono, N., Miyamoto, K., Le Roux, J., Kameoka, H., & Sagayama, S. (2008). In Separation of
a monaural audio signal into harmonic/percussive components by complementary
diffusion on spectrogram (pp. 1–4). IEEE.
Özseven, T., & Arpacioglu, M. (2023). Classification of Urban Sounds with PSO and WO
Based Feature Selection Methods. In 2023 5th International Congress on Human-
Computer Interaction, Optimization and Robotic Applications (HORA) (pp. 1–3).
https://doi.org/10.1109/HORA58378.2023.10156803
Park, D. S., Chan, W., Zhang, Y., Chiu, C. C., Zoph, B., Cubuk, E. D., et al. (2019).
SpecAugment: A Simple Data Augmentation Method for Automatic Speech
Recognition. Proc. Interspeech, 2613–2617. https://doi.org/10.21437/Interspeech.
2019-2680.
Park, J., Shin, J., & Lee, K. (2017). Exploiting continuity/discontinuity of basis vectors in
spectrogram decomposition for harmonic-percussive sound separation. IEEE/ACM
Transactions on Audio, Speech, and Language Processing, 25(5), 1061–1074.
Park, S., Mun, S., Lee, Y., & Ko, H. (2016). Score Fusion of Classification Systems for
Acoustic Scene Classification. Tech. Rep. DCASE2016 Challenge.
Paseddula, C., & Gangashetty, S. V. (2021). Late fusion framework for Acoustic Scene
Classification using LPCC, SCMC, and LogMel band energies with Deep Neural
Networks. Applied Acoustics, 172, Article 107568. https://doi.org/10.1016/j.
apacoust.2020.107568
Peltonen, V. T., Eronen, A. J., Parviainen, M. P., & Klapuri, A. P. (2001). Recognition of
everyday auditory scenes: potentials, latencies and cues. In Proceedings of the 110th
audio engineering society convention. Hall, Amsterdam.
Peltonen, V., Tuomi, J., Klapuri, A., Huopaniemi, J., & Sorsa, T. (2002). Computational
auditory scene recognition. In 2002 IEEE International Conference on Acoustics,
Speech, and Signal Processing (Vol. 2, pp. II-1941-II-1944). IEEE, https://doi.org/
10.1109/ICASSP.2002.5745009.
Pham, L., McLoughlin, I., Phan, H., Palaniappan, R., & Mertins, A. (2020). Deep feature
embedding and hierarchical classification for audio scene classification. In 2020
International Joint Conference on Neural Networks (IJCNN) (pp. 1–7). https://doi.org/
10.1109/IJCNN48605.2020.9206866
Pham, L., Phan, H., Nguyen, T., Palaniappan, R., Mertins, A., & McLoughlin, I. (2021).
Robust acoustic scene classification using a multi-spectrogram encoder-decoder
framework. Digital Signal Processing, 110, Article 102943.
Pham, L., Ngo, D., Salovic, D., Jalali, A., Schindler, A., Nguyen, P. X., et al. (2023).
Lightweight deep neural networks for acoustic scene classification and an effective
visualization for presenting sound scene contexts. Applied Acoustics, 211. https://doi.
org/10.1016/j.apacoust.2023.109489
Phan, H., Chén, O. Y., Pham, L., Koch, P., De Vos, M., McLoughlin, I., & Mertins, A.
(2019). Spatio-temporal attention pooling for audio scene classification. In Proc.
Interspeech 2019 (pp. 3845-3849). https://doi.org/10.21437/Interspeech.2019-
3040.
Phaye, S. S. R., Benetos, E., & Wang, Y. (2019). Subspectralnet–using sub-spectrogram
based convolutional neural networks for acoustic scene classification. In 2019 IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp.
825–829). https://doi.org/10.1109/ICASSP.2019.8683288
Piczak, K. J. (2015a). Environmental sound classification with convolutional neural
networks. In 2015 IEEE 25th international workshop on machine learning for signal
processing (MLSP) (pp. 1–6). https://doi.org/10.1109/MLSP.2015.7324337
Piczak, K. J. (2015b). ESC: Dataset for environmental sound classification. In Proceedings
of the 23rd ACM international conference on Multimedia (pp. 1015–1018).
Primus, P., Eghbal-zadeh, H., Eitelsebner, D., Koutini, K., Arzt, A., & Widmer, G. (2019).
Exploiting parallel audio recordings to enforce device invariance in CNN-based
acoustic scene classification. In Proc. DCASE2019 Workshop (pp. 204–208).
Rakotomamonjy, A. (2017). Supervised Representation Learning for Audio Scene
Classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25
(6), 1253–1265.
B. Ding et al.
Rakotomamonjy, A., & Gasso, G. (2015). Histogram of Gradients of Time-Frequency
Representations for Audio Scene Classification. IEEE/ACM Transactions on Audio,
Speech, and Language Processing, 23(1), 142–153.
Ren, Z., Kong, Q., Han, J., Plumbley, M. D., & Schuller, B. W. (2019). Attention-based
atrous convolutional neural networks: Visualisation and understanding perspectives
of acoustic scenes. In 44th IEEE International Conference on Acoustics, Speech, and
Signal Processing (ICASSP) (pp. 56–60). https://doi.org/10.1109/
ICASSP.2019.8683434
Ren, Z., Kong, Q., Han, J., Plumbley, M. D., & Schuller, B. W. (2020). CAA-Net:
Conditional atrous CNNs with attention for explainable device-robust acoustic scene
classification. IEEE Transactions on Multimedia, 23, 4131–4142.
Ren, Z., Kong, Q., Qian, K., Plumbley, M. D., & Schuller, B. W. (2018). Attention-based
convolutional neural networks for acoustic scene classification. In Proc. DCASE2018
Workshop (pp. 39–43).
Ren, Z., Pandit, V., Qian, K., Yang, Z., Zhang, Z., & Schuller, B. (2017). Deep sequential
image features for acoustic scene classification. In Proc. DCASE2017 Workshop (pp.
113–117).
Richard, G., Sundaram, S., & Narayanan, S. (2013). An Overview on Perceptually
Motivated Audio Indexing and Classification. Proceedings of the IEEE, 101(9),
1939–1954.
Roma, G., Nogueira, W., & Herrera, P. (2013). Recurrence quantification analysis
features for environmental sound recognition. In 2013 IEEE Workshop on Applications
of Signal Processing to Audio and Acoustics (pp. 1–4). https://doi.org/10.1109/
WASPAA.2013.6701890
Ruder, S. (2017). An overview of multi-task learning in deep neural networks. arXiv
preprint arXiv:1706.05098.
Sakashita, Y., & Aono, M. (2018). Acoustic scene classification by ensemble of
spectrograms based on adaptive temporal divisions. Tech. Rep. DCASE2018
Challenge.
Salamon, J., & Bello, J. P. (2015). Unsupervised feature learning for urban sound
classification. In 2015 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP) (pp. 171–175). https://doi.org/10.1109/ICASSP.2015.7177954
Salamon, J., & Bello, J. P. (2017). Deep convolutional neural networks and data
augmentation for environmental sound classification. IEEE Signal processing letters, 24
(3), 279–283.
Salamon, J., Jacoby, C., & Bello, J. P. (2014). A dataset and taxonomy for urban sound
research. In Proceedings of the 22nd ACM international conference on Multimedia (pp.
1041–1044). https://doi.org/10.1145/2647868.2655045
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L. (2018). MobileNetV2:
Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on
computer vision and pattern recognition (pp. 4510–4520).
Santana, L. E., Silva, L., Canuto, A. M., Pintro, F., & Vale, K. O. (2010). A comparative
analysis of genetic algorithm and ant colony optimization to select attributes for an
heterogeneous ensemble of classifiers. In IEEE congress on evolutionary computation
(pp. 1–8). https://doi.org/10.1109/CEC.2010.5586080
Sawhney, N., & Maes, P. (1997). Situational Awareness from Environmental Sounds.
Project Rep. for Pattie Maes, 1–7.
Schafer, R. M. (1977). The tuning of the world. New York, NY: Knopf.
Schafer, R. M. (1993). The soundscape: Our sonic environment and the tuning of the world
(1st ed., pp. 1–11). Simon and Schuster (Chapter).
Schmid, F., Masoudian, S., Koutini, K., & Widmer, G. (2022). CPJKU submission to
dcase22: Distilling knowledge for lowcomplexity convolutional neural networks
from a patchout audio transformer. Tech. Rep, DCASE2022 Challenge.
Schmid, F., Morocutti, T., Masoudian, S., Koutini, K., & Widmer, G. (2023). CP-JKU
Submission to Dcase23: Efficient Acoustic Scene Classification with Cp-Mobile,
DCASE2023 Challenge. Tech. Rep, DCASE2023 Challenge.
Schröder, J., Goetze, S., & Anemüller, J. (2015). Spectro-Temporal Gabor Filterbank
Features for Acoustic Event Detection. IEEE/ACM Transactions on Audio, Speech, and
Language Processing, 23(12), 2198–2208.
Schröder, J., Moritz, N., Anemüller, J., Goetze, S., & Kollmeier, B. (2017). Classifier
architectures for acoustic scenes and events: Implications for DNNs, TDNNs, and
perceptual features from DCASE 2016. IEEE/ACM Transactions on Audio, Speech, and
Language Processing, 25(6), 1304–1314.
Schröder, J., Moritz, N., Schädler, M. R., Cauchi, B., Adiloglu, K., Anemüller, J., et al.
(2013). On the use of spectro-temporal features for the IEEE AASP challenge
‘detection and classification of acoustic scenes and events’. In 2013 IEEE Workshop
on Applications of Signal Processing to Audio and Acoustics (pp. 1–4). https://doi.org/
10.1109/WASPAA.2013.6701868
Seo, H., Park, J., & Park, Y. (2019). Acoustic scene classification using various pre-
processed features and convolutional neural networks. In Proc. DCASE2019
Workshop (pp. 3–6).
Seresht, H. R., & Mohammadi, K. (2023). Environmental Sound Classification With Low-
Complexity Convolutional Neural Network Empowered by Sparse Salient Region
Pooling. IEEE Access, 11, 849–862. https://doi.org/10.1109/ACCESS.2022.3232807
Shen, Y. L., Huang, C. Y., Wang, S. S., Tsao, Y., Wang, H. M., & Chi, T. S. (2019).
Reinforcement Learning Based Speech Enhancement for Robust Speech Recognition.
In 2019 IEEE International Conference on Acoustics, Speech and Signal Processing
(ICASSP) (pp. 6750–6754). https://doi.org/10.1109/ICASSP.2019.8683648
Shim, H. J., Jung, J. W., Kim, J. H., & Yu, H. J. (2022). Attentive max feature map and
joint training for acoustic scene classification. In 2022 IEEE International Conference
on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1036–1040). https://doi.org/
10.1109/ICASSP43922.2022.9746091
Sigtia, S., Stark, A. M., Krstulović, S., & Plumbley, M. D. (2016). Automatic
environmental sound recognition: Performance versus computational cost. IEEE/
ACM Transactions on Audio, Speech, and Language Processing, 24(11), 2096–2107.
30
Expert Systems With Applications 238 (2024) 121902
Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale
image recognition. Computational and Biological Learning Society.
Singh, A. (2022). 1-D CNN based Acoustic Scene Classification via Reducing Layer-wise
Dimensionality. arXiv preprint arXiv: 2204.00555.
Singh, A., Rajan, P., & Bhavsar, A. (2019) Deep Multi-View Features from Raw Audio for
Acoustic Scene Classification. In Proc. DCASE2019 Workshop (pp. 229-233).
Singh, A., Rajan, P., & Bhavsar, A. (2020). SVD-based redundancy removal in 1-D CNNs
for acoustic scene classification. Pattern Recognition Letters, 131, 383–389.
Singh, A., Thakur, A., Rajan, P., & Bhavsar, A. (2018). A layer-wise score level ensemble
framework for acoustic scene classification. In 2018 26th European Signal Processing
Conference (EUSIPCO) (pp. 837–841).
Singh, V. K., Sharma, K., & Sur, S. N. (2023). A survey on preprocessing and classification
techniques for acoustic scene. Expert Systems with Applications, 229. https://doi.org/
10.1016/j.eswa.2023.120520
Song, H., Han, J., & Deng, S. (2018). A compact and discriminative feature based on
auditory summary statistics for acoustic scene classification. Proc. Interspeech,
3294–3298. https://doi.org/10.21437/Interspeech.2018-1299.
Steffens, J., Steele, D., & Guastavino, C. (2017). Situational and person-related factors
influencing momentary and retrospective soundscape evaluations in day-to-day life.
The Journal of the Acoustical Society of America, 141(3), 1414–1425.
Stiefelhagen, R., Bernardin, K., Bowers, R., Garofolo, J., Mostefa, D., & Soundararajan, P.
(2007). The CLEAR 2006 evaluation. In International evaluation workshop on
classification of events, activities and relationships (pp. 1–44). Berlin Heidelberg:
Springer. https://doi.org/10.1007/978-3-540-69568-4_1.
Stiefelhagen, R., Bernardin, K., Bowers, R., Rose, R. T., Michel, M., & Garofolo, J. (2008).
The CLEAR 2007 evaluation. In International Evaluation Workshop on Classification of
Events, Activities and Relationships (pp. 3-34). Berlin, Heidelberg: Springer Berlin
Heidelberg, https://doi.org/10.1007/978-3-540-68585-2_1.
Stiefelhagen, R., Bowers, R., & Fiscus, J. (2007). Multimodal Technologies for Perception of
Humans (1st ed.), Berlin, Heidelberg, Germany: Springer. https://doi.org/10.1007/
978-3-540-68585-2.
Stowell, D., & Plumbley, M. (2014). An Open Dataset for Research on Audio Field
Recording Archives: freefield1010. In 53rd AES International Conference 2014:
Semantic Audio (pp. 80-86).
Stowell, D., Giannoulis, D., Benetos, E., Lagrange, M., & Plumbley, M. D. (2015).
Detection and classification of acoustic scenes and events. IEEE Transactions on
Multimedia, 17(10), 1733–1746.
Suh, S., Park, S., Jeong, Y., & Lee, T. (2020). Designing acoustic scene classification
models with CNN variants. In Tech. Rep. DCASE2020 Challenge Task1.
Summers, C., & Dinneen, M. J. (2019). Improved mixed-example data augmentation. In
2019 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp.
1262–1270). IEEE. https://doi.org/10.1109/WACV.2019.00139.
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016). Rethinking the
Inception Architecture for Computer Vision. In Proceedings of the IEEE conference on
computer vision and pattern recognition (pp. 2818-2826).
Takahashi, G., Yamada, T., Ono, N., & Makino, S. (2017). Performance evaluation of
acoustic scene classification using DNN-GMM and frame-concatenated acoustic
features. In 2017 Asia-Pacific Signal and Information Processing Association Annual
Summit and Conference (APSIPA ASC) (pp. 1739–1743). IEEE. https://doi.org/
10.1109/APSIPA.2017.8282314.
Takeyama, S., Komatsu, T., Miyazaki, K., Togami, M., & Ono, S. (2021). Robust acoustic
scene classification to multiple devices using maximum classifier discrepancy and
knowledge distillation. In 2020 28th European Signal Processing Conference
(EUSIPCO) (pp. 36–40). IEEE.
Tan, M., & Le, Q. (2019). EfficientNet: Rethinking Model Scaling for Convolutional
Neural Networks. In International conference on machine learning (ICML) (pp.
6105–6114). PMLR.
Tang, Z., Gao, Y., Karlinsky, L., Sattigeri, P., Feris, R., & Metaxas, D. (2020).
OnlineAugment: Online Data Augmentation with Less Domain Knowledge. In
Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28,
2020, Proceedings, Part VII 16 (pp. 313-329). Springer International Publishing.
Tardieu, J., Susini, P., Poisson, F., Lazareff, P., & McAdams, S. (2008). Perceptual study
of soundscapes in train stations. Applied Acoustics, 69(12), 1224–1239.
Temko, A., Malkin, R., Zieger, C., Macho, D., Nadeu, C., & Omologo, M. (2007). CLEAR
evaluation of acoustic event detection and classification systems. In International
Evaluation Workshop on Classification of Events, Activities and Relationships (pp.
311–322). Berlin, Heidelberg: Springer.
Thiemann, J., Ito, N., & Vincent, E. (2013). The Diverse Environments Multi-channel
Acoustic Noise Database (DEMAND): A database of multichannel environmental
noise recordings. In Proceedings of Meetings on Acoustics ICA2013. Acoustical Society of
America, 19(1), Article 035081.
Tokozume, Y., Ushiku, Y., & Harada, T. (2018). Learning from between-class examples
for deep sound recognition. In International Conference on Learning Representations
(ICLR) (pp. 1-13).
Tripathi, A. M., & Pandey, O. J. (2023). Divide and Distill: New Outlooks on Knowledge
Distillation for Environmental Sound Classification. IEEE/ACM Transactions on
Audio, Speech, and Language Processing, 31, 1100–1113. https://doi.org/10.1109/
TASLP.2023.3244507
Tripathi, A. M., & Mishra, A. (2021). Self-supervised learning for Environmental Sound
Classification. Applied Acoustics, 2021(182), Article 108183. https://doi.org/
10.1016/j.apacoust.2021.108183
Truax, B. (2001). Acoustic Communication (1st ed.). Greenwood Publishing Group
(Chapter 1).
Tsalera, E., Papadakis, A., & Samarakou, M. (2021). Comparison of pre-trained CNNs for
audio classification using transfer learning. Journal of Sensor and Actuator.
Networks, 10(4), Article 72.
B. Ding et al.
Tsau, E. S., Kim, S. H., & Kuo, C. C. J. (2011). Environmental sound recognition with
CELP-based features. In International Symposium on Signals, Circuits and Systems (pp.
1–4). IEEE. https://doi.org/10.1109/ISSCS.2011.5978729.
Van Engelen, J. E., & Hoos, H. H. (2020). A survey on semi-supervised learning. Machine
Learning, 109(2), 373–440. https://doi.org/10.1007/s10994-019-05855-6
Van Grootel, M. W. W., Andringa, T. C., & Krijnders, J. D. (2009). DARES-G1: Database
of annotated real-world everyday sounds. In Proceedings of the NAG/DAGA Meeting
(pp. 996-999).
VanDerveer, N. J. (1979). Ecological acoustics: Human perception of environmental
sounds [Doctoral dissertation]. UMI No. 804002), ProQuest Dissertations & Theses
Global (302934163 Dissertation Abstracts International, 40/(9-B), 4543B htt
ps://www.proquest.com/dissertations-theses/ecological-acoustics-human-perceptio
n/docview/302934163/se-2?accountid=10424.
Vanhoucke, V., Senior, A., & Mao, M. Z. (2011). Improving the speed of neural networks
on CPUs. In Proc. Conf. Neural Information Processing Systems Deep Learning and
Unsupervised Feature Learning Workshop (pp. 1-8).
Varma, D. D., Padmanabhan, R., & Dileep, A. D. (2021). Learning to separate:
Soundscape classification using foreground and background. In 2020 28th European
Signal Processing Conference (EUSIPCO) (pp. 21–25). IEEE.
Venkatesh, S., Mulimani, M., & Koolagudi, S. G. (2023). Acoustic Scene Classification
using Deep Fisher network. Digital Signal Processing, 139. https://doi.org/10.1016/j.
dsp.2023.104062
Vij, D., Yogesh, Y., Srivastava, D., et al. (2023). Detection of Acoustic Scenes and Events
using Audio Analysis – A Survey. In 2023 3rd International Conference on Advance
Computing and Innovative Technologies in Engineering (ICACITE) (pp. 316–320). IEEE.
https://doi.org/10.1109/ICACITE57410.2023.10183195.
Virtanen, T., Plumbley, M. D., & Ellis, D. (2018). Computational analysis of sound scenes
and events (1st ed.). Springer (Chapter 1-9). https://doi.org/10.1007/978-3-319-
63450-0.
Vivek, V. S., Vidhya, S., & Madhanmohan, P. (2020). Acoustic scene classification in
hearing aid using deep learning. In 2020 International Conference on Communication
and Signal Processing (ICCSP) (pp. 0695–0699). IEEE. https://doi.org/10.1109/
ICCSP48568.2020.9182160.
Waibe,l A. & Stiefelhagen, R. (2009). Computers in the human interaction loop (1st ed.).
Springer (Chapter 7).
Waldekar, S., & Saha, G. (2020a). Analysis and classification of acoustic scenes with
wavelet transform-based mel-scaled features. Multimedia Tools and Applications, 79
(11), 7911–7926.
Waldekar, S., & Saha, G. (2020b). Two-level fusion-based acoustic scene classification.
Applied Acoustics, 170, Article 107502. https://doi.org/10.1016/j.
apacoust.2020.107502
Wang, C. Y., Santoso, A., & Wang, J. C. (2017). Acoustic scene classification using self-
determination convolutional neural network. In 2017 Asia-Pacific Signal and
Information Processing Association Annual Summit and Conference (APSIPA ASC) (pp.
19–22). IEEE. https://doi.org/10.1109/APSIPA.2017.8281995.
Wang, D. L., & Brown, G. J. (2006). Computational auditory scene analysis: Principles,
algorithms, and applications (1st ed.). Wiley-IEEE press (Chapter 1).
Wang, H., Zou, Y., Chong, D., & Wang, W. (2020). Environmental sound classification
with parallel temporal-spectral attention. In Proc. Interspeech (pp. 821-825). https://
doi.org/10.21437/Interspeech.2020-1219.
Wang, J. C., Wang, J. F., He, K. W., & Hsu, C. S. (2006). Environmental Sound
Classification using Hybrid SVM/KNN Classifier and MPEG-7 Audio Low-Level
Descriptor. In The 2006 IEEE international joint conference on neural network
proceedings (pp. 1731–1735). IEEE. https://doi.org/10.1109/IJCNN.2006.246644.
Wang, M., Chen, C., Xie, Y., Chen, H., Liu, Y., & Zhang, P. (2021). Audio-visual scene
classification using transfer learning and hybrid fusion strategy. Tech. Rep.
DCASE2021 Challenge.
Wang, Q., Zheng, S., Li, Y., Wang, Y., Wu, Y., Hu, H., Yang, C. H. H., Siniscalchi, S. M.,
Wang, Y., Du, J. & Lee, C. H. (2021). A model ensemble approach for audio-visual
scene classification. Tech. Rep. DCASE2021 Challenge.
Wang, Q., Du, J., Wu, H. X., Pan, J., Ma, F., & Lee, C. H. (2023). A Four-Stage Data
Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound
Event Localization and Detection. IEEE/ACM Transactions on Audio, Speech, and
Language Processing, 31, 1251–1264. https://doi.org/10.1109/TASLP.2023.3256088
Wang, R., Wang, M., Zhang, X. L., & Rahardja, S. (2019). Domain adaptation neural
network for acoustic scene classification in mismatched conditions. In 2019 Asia-
Pacific Signal and Information Processing Association Annual Summit and Conference
(APSIPA ASC) (pp. 1501–1505). IEEE. https://doi.org/10.1109/
APSIPAASC47483.2019.9023057.
Wang, S., Mesaros, A., Heittola, T., & Virtanen, T. (2021c). A curated dataset of urban
scenes for audio-visual scene analysis. In 2021 IEEE International Conference on
Acoustics, Speech and Signal Processing (ICASSP) (pp. 626–630). IEEE. https://doi.
org/10.1109/ICASSP39728.2021.9415085.
Wang, X., Han, Y., Leung, V. C., Niyato, D., Yan, X., & Chen, X. (2020). Convergence of
edge computing and deep learning: A comprehensive survey. IEEE Communications
Surveys & Tutorials, 22(2), 869–904.
Wang, Y. (2018). Polyphonic sound event detection with weak labeling. Carnegie Mellon
University [Doctoral dissertation]. Doctoral dissertation,.
Wang, Y., Getreuer, P., Hughes, T., Lyon, R. F., & Saurous, R. A. (2017). Trainable
frontend for robust and far-field keyword spotting. In 2017 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 5670–5674).
IEEE. https://doi.org/10.1109/ICASSP.2017.7953242.
Weiss, K., Khoshgoftaar, T. M., & Wang, D. D. (2016). A survey of transfer learning.
Journal of Big data, 3(1), 1–40.
31
Expert Systems With Applications 238 (2024) 121902
Wilkinghoff, K. (2021). On Open-Set Classification with L3-Net Embeddings for Machine
Listening Applications. In 2020 28th European Signal Processing Conference
(EUSIPCO) (pp. 800–804). IEEE.
Wu, Y., & Lee, T. (2019). Enhancing sound texture in CNN-based acoustic scene
classification. In 2019 IEEE International Conference on Acoustics, Speech and Signal
Processing (ICASSP) (pp. 815–819). IEEE. https://doi.org/10.1109/
ICASSP.2019.8683490.
Wu, Y., & Lee, T. (2020). Time-Frequency Feature Decomposition based on sound
duration for acoustic scene classification. In 2020 IEEE International Conference on
Acoustics, Speech and Signal Processing (ICASSP) (pp. 716–720). IEEE. https://doi.
org/10.1109/ICASSP40776.2020.9053194.
Xia, X., Togneri, R., Sohel, F., Zhao, Y., & Huang, D. (2019). A survey: Neural network-
based deep learning for acoustic event detection. Circuits, Systems, and Signal
Processing, 38(8), 3433–3453.
Xie, J., & Zhu, M. (2019). Investigation of acoustic and visual features for acoustic scene
classification. Expert Systems with Applications, 126, 20–29.
Xie, W., He, Q., Yan, H., & Li, Y. (2021). Acoustic Scene Classification Using Deep CNNs
with Time-Frequency Representations. In 21st International Conference on
Communication Technology (ICCT) (pp. 1325–1329). IEEE. https://doi.org/10.1109/
ICCT52962.2021.9658004.
Xie, W., Li, Y., He, Q., & Cao, W. (2023). Few-shot class-incremental audio classification
via discriminative prototype learning. Expert Systems with Applications, 255, Article
120044. https://doi.org/10.1016/j.eswa.2023.120044
Xu, Y., Huang, Q., Wang, W., & Plumbley, M. D. (2016). Hierarchical learning for DNN-
Based acoustic scene classification. In Proc. DCASE2016 Workshop (pp. 105-109).
Yang, C. H. H., Hu, H., Siniscalchi, S. M., Wang, Q., Wang, Y., Xia, X., Zhao, Y., Wu, Y.,
Wang, Y., Du, J. & Lee, C. H. (2021). A lottery ticket hypothesis framework for low-
complexity device-robust neural acoustic scene classification. Tech. Rep.
DCASE2021 Challenge.
Yang, D., Wang, H., & Zou, Y. (2021). Unsupervised multi-target domain adaptation for
acoustic scene classification. In Proc. Interspeech 2021 (pp. 1159-1163). https://doi.
org/10.21437/Interspeech.2021-300.
Yang, L., Chen, X., & Tao, L. (2018). Acoustic scene classification using multi-scale
features. Tech. Rep. DCASE2018 Challenge.
Yang, L., Tao, L., Chen, X., & Gu, X. (2020). Multi-scale semantic feature fusion and data
augmentation for acoustic scene classification. Applied Acoustics, 163, Article
107238.
Yang, Y., Zhang, H., Tu, W., Ai, H., Cai, L., Hu, R., et al. (2019). Kullback-Leibler
divergence frequency warping scale for acoustic scene classification using
convolutional neural network. In 2019 IEEE International Conference on Acoustics,
Speech and Signal Processing (ICASSP) (pp. 840–844). IEEE. https://doi.org/10.1109/
ICASSP.2019.8683000.
Yao, K., Yang, J., Zhang, X., Zheng, C., & Zeng, X. (2019). Robust deep feature extraction
method for acoustic scene classification. In 2019 IEEE 19th International Conference
on Communication Technology (ICCT) (pp. 198–202). IEEE. https://doi.org/10.1109/
ICCT46805.2019.8947252.
Ye, M., Zhong, H., Song, X., Huang, S., & Cheng, G. (2019). Acoustic scene classification
using deep convolutional neural network via transfer learning. In 2019 International
Conference on Asian Language Processing (IALP) (pp. 19–22). IEEE. https://doi.org/
10.1109/IALP48816.2019.9037692.
Yin, Y., Shah, R. R., & Zimmermann, R. (2018). Learning and fusing multimodal deep
features for acoustic scene categorization. In Proceedings of the 26th ACM
international conference on Multimedia (pp. 1892–1900).
Zhang, C., Zhan, H., Hao, Z., & Gao, X. (2023). Classification of Complicated Urban
Forest Acoustic Scenes with Deep Learning Models. Forests, 14(2), 206.
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2018). Mixup: Beyond empirical
risk minimization. In International Conference on Learning Representations (ICLR)
(pp.1-13).
Zhang, J., & Yan, H. (2023). Application and Implementation of Convolutional Neural
Network Accelerator Based on FPGA in Environmental Sound Classification. In 2023
8th International Conference on Computer and Communication Systems (ICCCS) (pp.
22–27). IEEE. https://doi.org/10.1109/ICCCS57501.2023.10151442.
Zhang, L., Han, J., & Shi, Z. (2020). Learning Temporal Relations from Semantic
Neighbors for Acoustic Scene Classification. IEEE Signal Processing Letters, 27,
950–954.
Zhang, T., & Kuo, C. C. J. (2001). Audio content analysis for online audiovisual data
segmentation and classification. IEEE Transactions on speech and audio processing, 9
(4), 441–457.
Zhang, T., & Kuo, C. C. J. (2001b). Generic audio data segmentation and indexing (1st ed.).
Springer (Chapter 4).
Zhang, T., Feng, G., Liang, J., & An, T. (2021). Acoustic scene classification based on Mel
spectrogram decomposition and model merging. Applied Acoustics, 182, Article
108258.
Zhang, T., Liang, J., & Ding, B. (2020). Acoustic scene classification using deep CNN with
fine-resolution feature. Expert Systems with Applications, 143, Article 113067. https://
doi.org/10.1016/j.eswa.2019.113067
Zhang, T., Liang, J., & Feng, G. (2022). Adaptive time-frequency feature resolution
network for acoustic scene classification. Applied Acoustics, 195, Article 108819.
https://doi.org/10.1016/j.apacoust.2022.108819
Zhang, T., Zhang, K., & Wu, J. (2018). Temporal Transformer Networks for Acoustic
Scene Classification. In Proc. Interspeech (pp. 1349-1353). https://doi.org/
10.21437/Interspeech.2018-1152.
Zhao, H., Hu, L., Peng, X., Wang, G., Yu, F., & Xu, C. (2011). An Improving MFCC
Features Extraction Based on FastICA Algorithm plus RASTA Filtering. Journal of
Computers, 6(7), 1477–1484.
B. Ding et al.
Zhao, J., Kong, Q., Song, X., Feng, Z., & Wu, X. (2022). Feature alignment for robust
acoustic scene classification across devices. IEEE signal processing letters, 29,
578–582.
Zhao, X., Jia, X., Zhang, T., Cao, Y., & Liu, T. (2023). Evolutionary Algorithms with Blind
Fitness Evaluation for Solving Optimization Problems with Only Fuzzy Fitness
Information. IEEE Transactions on Fuzzy Systems. https://doi.org/10.1109/
TFUZZ.2023.3273308
Zhao, X., Jia, X., Zhang, T., Liu, T., & Cao, Y. (2023). A Supervised Surrogate-Assisted
Evolutionary Algorithm for Complex Optimization Problems. IEEE Transactions on
Instrumentation and Measurement, 72, 1–14.
32
Expert Systems With Applications 238 (2024) 121902
Zheng, W., Mo, Z., & Zhao, G. (2022). Clustering by Errors: A Self-Organized Multitask
Learning Method for Acoustic Scene Classification. Sensors, 22(1), Article 36.
Zhong, Z., Zheng, L., Kang, G., Li, S., & Yang, Y. (2020). Random erasing data
augmentation. In 34th AAAI Conference on Artificial Intelligence (Vol. 34, No. 07,
pp. 13001-13008).
Zhu, C., Han, S., Mao, H., & Dally, W. J. (2017). Trained ternary quantization. In
International Conference on Learning Representations (ICLR). (pp. 1-10).
Zieliński, S. K. (2018). Feature extraction of surround sound recordings for acoustic scene
classification. In International conference on artificial intelligence and soft
computing (pp.475-486).