Electric Guitar Playing Style Feature

fusion using Filter Banks

-Aryan S Nair (CH.EN. U4ECE22010), Priyanshu Kumar Singh (CH.EN. U4ECE22036),
Sankeerthana Sanikommu (CH.EN. U4ECE22061)

Abstract
We, in our project aim to Develop an effective and versatile CNN model for classifying
electric guitar playing styles into nine different categories. The model is trained on a dataset
of audio recordings, where the audio files are converted into various types of spectrograms,
such as Mel-Frequency Cepstral Coefficients (MFCC), Linear Frequency Cepstral
Coefficients (LFCC) and Triangular Filter Banks. These spectrograms serve as input features
to the CNN model, which identifies sequences in the audio data that are characteristic of
different playing styles. We calculate the efficiency metrics such as accuracy, loss, confusion
matrix, and F1 scores. Additionally, the project explores the potential of combining Mel
Spectrogram +Filter Banks and LFCC+MFCC to potentially improve classification accuracy.

Introduction:
Guitar playing style classification is a challenging task due to the diverse range of techniques
employed by guitarists. The ability to automatically categorize guitar playing styles can have
practical applications not in music analysis, genre identification, and music recommendation
systems but also in other applications involving a spectral analysis of sound to use in
practical scenarios like to detect machine malfunctions from sound, or in healthcare to
identify respiratory issues through cough analysis. Environmental applications include
recognizing gunshots or animal sounds for security or wildlife monitoring. Automatic
instrument recognition can even analyse background noise to optimize audio settings on
smart devices.

In this project, we use Convolutional Neural Networks (CNNs), to tackle this classification
problem. CNNs have proven to be highly effective in various domains, including image and
audio processing, due to their ability to capture local patterns and extract relevant features
from raw data. We sort the Audio into 9 distinct classes:

• Picking Styles: Alternate picking creates a smooth sound, while sweep picking
delivers a rapid flow.
• Legato: Notes are played smoothly, transitioning smoothly.
• Tapping: Both hands are used on the fretboard for a percussive sound
• Vibrato: Contains subtle pitch fluctuations.

So, we analyse their spectral characteristics to try to understand and classify their
variance from each other.

Seeing their Mean Filter Bank for each class across each different class helps us get a
perspective:

LFCC AND MFCC calculation by Librosa:

Literature Survey:
Author Title Methodology/P Findings
arameters used
Alexandros A The research introduces a
Mitsou, Antonia multimodal audio+video dataset for
recognising electric guitar
Petrogianni, dataset for playing styles, comprising 549
Eleni Amvrosia electric mp4 files and their respective
Vakalaki, guitar audio files. Recordings were
Christos Nikou, playing made using an Android
Theodoros technique smartphone device in a home
studio setup, featuring three
Psallidas, recognition different instruments and
Theodoros utilised simulated amplifiers
Giannakopoulos to extract sound. The dataset
includes MuseScore files for
educational purposes and
future expansion. Support
Vector Machine (SVM) and
Convolutional Neural
Network (CNN) models were
employed to classify the files
into different styles.
Vincent Extended The research conducted by
Lostanlen, playing Lostanlen, Andén, and
techniques: the Lagrange focused on
Joakim Anden, advancing automatic
next milestone
Mathieu in musical identification of instrumental
Lagrange, instrument playing techniques (IPTs) in
recognition music. They conducted a
benchmark study utilizing the
Studio On Line (SOL) dataset,
which contains samples of
extended IPTs for 16 different
instruments. The study
implemented machine
listening systems for query-
by-example browsing,
evaluating 143 extended IPTs.
Three key conditions for
improving performance
beyond the traditional mel-
frequency cepstral coefficient
(MFCC) baseline were
identified: incorporation of
second-order scattering
coefficients, integration of
long-range temporal
Their information collected
addresses a notable gap in
comprehensive datasets for
recognising different playing styles,
offering diverse techniques beyond
conventional ones. Its inclusion of
video and audio data enables
exploration of new deep learning
architectures. Machine learning
experiments using SVM and CNN
models showed promising
performance spanning 67.8% to
84.2% for SVM and 76.5% to 81.1%
for CNN. However, limitations exist
due to reliance on recordings from
a single player, suggesting potential
improvements through
diversification of samples from
multiple players to capture broader
playing styles.
The research findings revealed
significant advancements in both
instrument and IPT recognition
tasks. Evaluation on the SOL
dataset demonstrated remarkable
precision, with a precision at rank 5
of 99.7% for instrument
recognition and 61.0% for IPT
recognition. The incorporation of
second-order scattering
coefficients, consideration of long-
range temporal dependencies, and
application of metric learning using
LMNN contributed to substantial
improvements in accuracy
compared to the traditional MFCC
baseline. These findings indicate
promising opportunities for
enhancing machine listening
systems, particularly in the context
of music information retrieval and
recommendation systems. Overall,
the study underscores the
importance of considering gesture
and playing techniques in music
 dependencies, and utilization
of metric learning with large-
margin nearest neighbors
(LMNN) to reduce intra-class
variability. The methodology
involved comparing the
performance of different
feature sets, including MFCCs
and scattering transforms,
while varying parameters
such as time scale for
amplitude modulation and
distance metrics.
Qinyang Xi, Guitarset: A The research presented by Xi
Rachel M dataset for et al. introduces GuitarSet, a
comprehensive dataset
Bittner, Johan guitar designed for guitar
Pauwels, transcription transcription research,
Xuzhou Ye, addressing the challenges
Juan P Bello posed by the instrument's
polyphonic nature and varied
playing styles. The dataset
comprises high-quality
recordings of acoustic guitar
excerpts played by six
experienced guitarists,
utilizing a hexaphonic pickup
to capture individual string
recordings and automate
annotation processes.
Detailed annotations include
string and fret positions,
chords, beats, downbeats,
and playing style, enabling a
wide range of analysis tasks
beyond transcription, such as
performance analysis and
chord estimation. The
methodology involves careful
data collection using
hexaphonic pickups and
condenser microphones,
coupled with an automated
annotation process leveraging
tools like pYIN-note for pitch
estimation and custom
annotation tools for precise
note-level transcription.
Christian Automatic The paper introduces a new
Kehling, Jakob algorithm designed for the
Tablature auto-analysis,transcribing and
Abeßer, Transcription
analysis, offering valuable insights
for future research in the field.
The findings from the baseline
experiments demonstrate the
effectiveness of GuitarSet in
evaluating algorithm performance
for guitar transcription tasks.
Evaluation results on note
transcription accuracy using the
Deep Salience algorithm reveal an
overall accuracy of approximately
46%, with variations observed
across genres, recording modes,
tempos, and players. Similarly,
chord recognition performance,
assessed against state-of-the-art
algorithms, showcases variations in
accuracy across different genres,
indicating challenges associated
with chord complexity and
variations in playing styles. Beat
and downbeat detection
algorithms, evaluated on GuitarSet,
exhibit substantial differences in
performance across players,
highlighting the impact of
individual playing characteristics on
algorithmic accuracy. Overall,
GuitarSet serves as a valuable
resource for advancing guitar
transcription research, offering
researchers a standardized dataset
with rich annotations to explore
various aspects of guitar playing
and inform the development of
robust transcription algorithms.
The proposed algorithm represents
a significant advancement in guitar
transcription technology,
Christian of Electric feature fusion from
Dittmar, Gerald Guitar polyphonic instrument audio.
The algorithm focuses on
Schuller Recordings retrieving both score-related
by information such as note
Estimation onset, duration, and pitch, as
well as instrument-specific
of Score- details like the plucked string
and and applied playing
Instrument- techniques. To achieve this,
Related the researchers adapted
cutting edge techniques for
Parameters string recognition, multipitch
estimation, feature
extraction, and start and
offset detection. They also
looked into a powerful partial
tracking algorithm that could
handle inharmonicity, extract
a large amount of data
features, and use instrument-
based knowledge to make
predictions that were more
accurate. The method
demonstrated remarkable
precision, achieving 98% in
offset and onset detection
and multipitch estimation. It
also shown strong
performance in terms of
instrument-related
characteristics, obtaining 82%
in string number, 93% in
plucking style, and 83% in
expressive style.
Methodology:
demonstrating outstanding
accuracy in automatically
extracting essential parameters
crucial for creating guitar scores
from audio recordings. Through
meticulous experimentation and
optimization, the system achieved
remarkable results, with accuracy
values exceeding 98% for onset and
offset detection and multipitch
estimation. Moreover, the
algorithm displayed strong
performance in estimating
instrument-related parameters,
with accuracy values surpassing
80% for string number, plucking
style, and expression style. These
concludes the robustness of the
project in accurately transcribing
complex guitar recordings, paving
the way for applications in music
education software, music games,
and expressive performance
analysis. Additionally, the creation
of a novel dataset facilitates further
research and evaluation in the field
of guitar transcription, highlighting
the paper's contributions to
advancing the state-of-the-art in
music information retrieval for
guitar recordings.
Step Description
Data Preprocessing Audio Files are converted to different types
of spectrograms (MFCC, Triangular Filter
Banks) using librosa library.

The Spectrograms are resized to a fixed

input shape for the CNN model
Feature Extraction The input features comprise of mel-
spectrograms extracted from 1-second
audio segments using a STFT window fifty
miliseconds with no overlapping. The mel-
spectrograms are computed with 128 mel-
frequency bins, resulting in a two
dimensional form of frame 20 × 128
(timestamps x frequency bins) for each one
second sound part.
Model Architecture Four convolutional layers and a three-layer
linear classifier make up the CNN model.
The latter converts the fed graphs are
converted into 9-D data segments that
represents various electric guitar playing
styles. Every convolutional layer uses five x
five kernels with a padding of two and a
stride of one by one to preserve the spatial
dimensions. After 32 channels, each
convolutional layer has 256 channels, which
is a twofold increase. After every
convolutional layer, Batch Normalization
and LeakyReLU activation functions are
implemented. With a 2 × 2 kernel size, 2D
Max Pooling is used in each convolutional
layer to halve the spatial dimensions. This
2048D data segment is mapped to the final
9D data segments utilising 3 paths.
Every linear layer is made up of a
LeakyReLU activation after a linear
mapping. These linear layers have output
dimensions of 1048, 256, and 9, in that
order.

Model training Then train the CNN model using Adam

optimizer and sparse categorical cross-
entropy loss. Monitor training and
validation metrics (loss, accuracy)
Model Evaluation Then we evaluate the trained model on a test
dataset- Compute metrics: accuracy, loss,
confusion matrix, F1 scores
 Concatenations Combine MFCC and Filter Banks as input
features - Train a separate CNN model on
the combined features - Evaluate the
concatenated model's performance
Visualisations Plot the graphs for each method - Visualize
the spectrograms (MFCC, Filter Banks) for
sample audio files

Conclusion:
Data Split Method Accuracy F1 score (%)
(%) [avg]
1 MFCC 92.77 90.66
2 Triangular filter- 91.18 90.55
bank+ Mel
Spectrogram
3 LFCC 90.57 88.66
4 LFCC+MFCC 91.79 90.22

The Electric Guitar playing style classification by Filter Bank Feature Fusion project
successfully develops a CNN model capable of classifying electric guitar playing styles into
nine different categories. The model used a dataset of audio recordings, where the audio files
were converted into various types of spectrograms, such as MFCC and Triangular Filter
Banks.

Through extensive experimentation, we demonstrate the effectiveness of using spectrograms

as input features to the CNN model, enabling it to identify patterns in the audio data that are
characteristic of different playing styles.

The project also explored the potential of combining Mel Spectrograms,Filter Banks and
LFCCs to create an ensemble model, which showed promising results accuracy tests by
leveraging the slightly complementary information provided by the both feature
representations. While Mel spectrograms represent the energy distribution across different
frequencies (mimicking the human auditory perception), filter banks provide a more detailed
representation of the spectral envelope. By concatenating these two features, the model can
potentially learn from the complementary information they provide, leading to a more
comprehensive representation of the audio signal. Concatenating multiple features can make
the model more robust to variations in the input data. If one type of feature fails to capture
certain aspects of the audio signal due to noise or other factors, the other feature type may
still provide useful information, thereby increasing the overall robustness of the model.
It's worth noting that the effectiveness of using concatenated features can vary depending on
the specific task and dataset. In some cases, using only one type of feature (e.g., Mel
spectrogram or filter bank) may be sufficient (like in the case here), or even better than using
concatenated features.

The trained model achieved satisfactory performance, as evidenced by the reported metrics
such as accuracy, loss, confusion matrix, and F1 scores. Additionally, the project included
visualizations of the spectrograms and mean Filter Banks for each class, providing insights
into the underlying patterns learned by the model.

Overall, this project contributes to the field of music information retrieval and paves the way
for various applications, such as automatic genre classification, music recommendation
systems, and assisting musicians in refining their techniques.