2016 IEEE 12th International Colloquium on Signal Processing & its Applications (CSPA2016), 4 - 6 March 2016, Melaka, Malaysia

Abnormal Sound Analytical Surveillance System

Using Microcontroller
Tan Teng Teng #1, Lim Tien Sze #2, Ong Lee Yeng #3
#
Centre For Remote Sensing And Surveillance Technologies (CRSST),
Faculty of Engineering and Technology, Multimedia University,
Jalan Ayer Keroh Lama 75450 Bukit Beruang, Melaka, Malaysia
1
tengteng77@yahoo.com, 2 tslim@mmu.edu.my, 3 lyong@mmu.edu.my

Abstract—Analytical surveillance can perform the surveillance
tasks much more efficient comparing to operator manual
monitoring. This had made it getting increased market’s interest
in recent years. Commonly, closed circuit television (CCTV) is
used for security surveillance. However, CCTVs are purely vision
output. These silent videos may not provide complete picture of
the happening. Sound detection is incorporate into vision
surveillance for enhancement. Sound detection is able to detect
abnormal sound although happen at camera blind spots or due to
intentional blocking. In this paper, we propose to use
microcontroller embedded system to enhance current CCTV
system. Proposed abnormal sound embedded system is to carry
out the sound detection, audio processing and analysis. This
study is using only single microphone for sound detection. Audio
amplitude and frequency range are targeted feature extracted
from Fast Fourier Transform (FFT). Abnormal sound of human
screaming and glass breaking were classified using decision tree.
From experiment, proposed abnormal sound analytical
surveillance system test yield average of 88% accuracy detection.
We can consider our work is simple and cost effective for field
implementation.
Index Terms—Abnormal sound, audio analytical,
microcontroller.
I. INTRODUCTION
Crime’s index has increase trend and had been identified as
main concern of people and to the economy. From the statistic
data, property theft contributing 82% of overall reported
crimes[1]. CCTVs are introduced to perform the surveillance
tasks in street and residential areas but blind spot had lead to
imperfection of the CCTV surveillance system. Sound is
another important environment stimulus for surveillance
alertness. Glass breaking sound could be considered as action
of try break into a premise that needs attention. Human
screaming when needs help and attention. These 2 types of
sound will be our main focus in this paper. We are proposing
an abnormal sound analytical surveillance using
microcontroller to enhance existing CCTV. The audio-assisted
vision surveillance system will be catered with comprehensive
capability to recognize the happening. Thus, surveillance
system will be more efficient to prevent and detect theft
happening.
Mel Frequency Cepstral Coefficients (MFCC) is one of
popular audio feature used for recognition. MFCC had been
used as one of the audio features for scream detection[2][3]. It
also result a good classification. MFCC takes human
perception sensitivity with respect to frequencies into
consideration. This include of mel frequency filtering which
reflect of human ear perceived frequency/pitch. Human
hearing is more sensitive to low frequencies. Another step in
MFCC extraction is logarithm. This is to mimic human
perception to loudness. There are also other audio features
including pitch range[2][5], energy[2][4] and zero crossing
rate(ZCR)[4] had been exploited to recognize scream and
glass breaking sound.
Microcontroller had been used in voice and speech
recognition[7][8][9][10]. Clayder and Nils had compared that
audio signal processed by microcontroller is compatible with
computer[8]. Consideration on using microcontroller is its
limitation on speed and memory capacity.
Audio also plays an important role other than vision on
analytic surveillance detection. Start with CCTV surveillance
and now to be developed with abnormal sound detection for
surveillance enhancement. Proposed abnormal sound analytical
surveillance is designed to detect, analyze, recognize and
trigger for further action.
II. METHODOLOGY
Since CCTV is commonly used for surveillance, we are
proposing and developing an add-on module to enhance the
current CCTV surveillance system.
We start with theoretical and mathematic analysis on
interested sound to be detected. Here, we perform characteristic
comparison between abnormal sound and no abnormality
sound. Thus, we selected features that could recognize between
abnormal and no abnormality sound for classification. Here,
we develop our abnormal sound analytical algorithm.
Next step we continue with modeling the platform using
microcontroller embedded board for our proposed abnormal
sound analytical surveillance system. The proposed abnormal
sound analytical microcontroller embedded system is used for
audio signal acquisition and proposed algorithm execution.

978-1-4673-8780-4/16/$31.00 ©2016 IEEE 162

Authorized licensed use limited to: Monash University. Downloaded on August 29,2020 at 01:40:14 UTC from IEEE Xplore. Restrictions apply.
 2016 IEEE 12th International Colloquium on Signal Processing & its Applications (CSPA2016), 4 - 6 March 2016, Melaka, Malaysia
Finally our proposed abnormal sound analytical surveillance
system will be assessed at real environment to evaluate the
effectiveness on scream and glass breaking sound detection.
A. Feature Selection
Fig. 1. Time domain plot for ‘Tolong saya’ normal speech scenario.
Fig. 2. Time domain plot for ‘Tolong saya’ screaming scenario.
Fig. 3. Frequency domain plot for ‘Tolong saya’ normal speech scenario.
163
Authorized licensed use limited to: Monash University. Downloaded on August 29,2020 at 01:40:14 UTC from IEEE Xplore. Restrictions apply.
Fig. 4. Frequency domain plot for ‘Tolong saya’ screaming scenario.
Fig. 5. Frequency domain plot for glass breaking.
For audio, there are three physical properties: frequency,
intensity and envelope to be deployed for the next step of
classification. The selection of appropriate features will
determine on the classification result. Human brain percept
that high pitch sound carry the abnormal scenario and will
draw attention to it.
For surveillance, we are proposed a method to detect the
abnormal scenario. The content of the audio will not be main
core to be analyzed. Figure 1 normal speech and Figure 2
screaming is from the same Malay word ‘Tolong saya’. We
clearly observed that screaming’s audio spectrum envelope is
wider than normal speech. Screaming’s audio amplitude is
higher than normal speech. Amplitude is selected as one of
our classification feature to differentiate normal speech and
screaming.
The audio signal could be analysed in frequency domain by
performing Fourier transform to decompose a time function
audio signal into frequency function signal. For periodic
signal, discrete Fourier Transform (DFT) was applied to
convert time function into a finite combination of complex
sinusoids correspondence to frequency. Fast Fourier
Transform (FFT) is an algorithm to perform DFT. In
frequency domain, the magnitude represents the amplitude for
 2016 IEEE 12th International Colloquium on Signal Processing & its Applications (CSPA2016), 4 - 6 March 2016, Melaka, Malaysia
each frequency. For analog signals with periodic sequence, N
can be represented by a discrete Fourier Transform as :-
∑ ⁄
(1)
From frequency domain chart Fig. 4 and Fig. 5 abnormal
scenarios, high amplitude mostly accumulate at higher
frequency compare to Fig. 3 normal speech. Frequency will be
another feature selected for our classification algorithm.
Figure 6, Figure 7 and Figure 8 show that sound source
distance from sensing microphone has a significant impact on
amplitude. Magnitude value is reduce when source move
further away from microphone but the high frequency
characteristic still remain useable for recognition. The
influence of source distance will be taken into consideration in
our abnormal sound analytic algorithm. Pre-emphasis step was
added to minimize the influence of source distance.
The intensity, I is related to source distance by :-
(2)
(3)
where by Pav=average power emitted, A=coverage area
and r=radius distance from source.
Fig. 6. Source distance at 120cm from microphone on frequency domain plot.
Fig. 7. Source distance at 280cm from microphone on frequency domain plot.
164
Authorized licensed use limited to: Monash University. Downloaded on August 29,2020 at 01:40:14 UTC from IEEE Xplore. Restrictions apply.
Fig. 8. Source distance at 600cm from microphone on frequency domain plot.
B. Classification
For audio classification, we proposed to use decision tree.
Classification tree is decision tree learning to map
observations base on decision rules to conclude the target
categories. From the root, decision node will specific an
attribute test. The decision node will either branch to leaf node
which indicate the target category or path to another decision
node. These processes will continue until come to the end of
leaf node. Our decision ended with three categories: glass
breaking, human screaming and no abnormality.
Typically human voice is approximate 300-3400Hz.
Normal speech range from 85-255Hz. Generally, female’s
voice frequency is higher than male. Screaming generally
generated high pitch and loud. High pitch is corresponding to
high frequency. Meanwhile, the louder the sound, the bigger is
the amplitude.
The classification tree root start from the 1st leaf node with
recognise soften audio. To mitigate the intensity lost, audio
with magnitude lower than defined amplitude threshold will
be emphasized with a given vector. Audio with acceptable
level amplitude will proceed to next stage without additional
processing. Next step is to filter off the spectrum with
magnitude less than amplitude filter threshold. We continue
leaf node with higher frequency threshold to classify glass
breaking sound. Lastly leaf node will differentiate the human
screaming sound or no abnormality scenario.
III. HARDWARE AND SYSTEM IMPLEMENTATION
The stimulus signal will be passed to brain for processing
and recognition. Finally, brain will trigger refection action
accordingly. These series of flow are developed in automation
machine using intelligent computing. The idea of this paper is
to convert human recognition on abnormal sound into an
automatic computing analytic surveillance system. This is
show in Fig. 9
For implementation, there are two main things to be
considered which are sound acquisition and processing. For
sound acquisition, there are two types set-up: one is single
microphone and another type is microphone arrays[6]. The
processing could be carried out using PC or as simple as
 2016 IEEE 12th International Colloquium on Signal Processing & its Applications (CSPA2016), 4 - 6 March 2016, Melaka, Malaysia

embedded system[6]. In this paper, we are presenting the

abnormal sound analytical surveillance using microcontroller
embedded system. This audio detection system could be added
to current video surveillance system with minimum software
modification.
In this paper, we proposed a set-up of our abnormal sound
analytical surveillance system consist of a single microphone
and a microcontroller embedded system. The set-up is shown
in Fig. 11. There are only 3 items required: proposed
microcontroller embedded system (Fig 10), microphone and
connection cable to CCTV processor to set-up our abnormal
sound analytical surveillance. PIC32MX250F128D was
Fig. 11. Microcontroller embedded system setup.
selected as processor of proposed abnormal sound analytical
surveillance system. PIC32MX250F128D microchip is
commonly acquired in market and cost effective for
implementation. The processor could be easy be programmed Analog Audio
Signal
Microphone

by using free development software tool provided by Acquisition

Microchip[11]. A codec WM8940 was used for analog-digital

conversion(ADC). We are using Li-ion battery for locally
power supply. The microcontroller interface including of a Analog Digital
Convertor
Codec
VM8974
3.5mm audio jack socket for sensing microphone connection
and a micro USB for alertness signal transmission through
UART to main processor of CCTV surveillance. Fig. 10 is Feature Selection/
Extraction
appearance of our microcontroller embedded system. Proposed (Frequency
Domain)
embedded system was used to perform sound capturing, feature
extraction and classification. Subsequently trigger surveillance
results for awareness labeling. Proposed abnormal sound YES

analytical surveillance system algorithm flow was presented in Audio Is minimum

Fig. 12. Analytical

Algorithm
amplitude threshold
meet?

Pre- Microcontroller
emphasize Embedded
NO System

Filtering

Classification

Fig. 9. Conversion from human recognition to computing analytical Classification

Results
UART
Transmission
classification on abnormal sound.

Fig. 12. Flow chart for abnormal sound analytical surveillance system.

Fig. 10. Microcontroller PIC32MX250F128D embedded system.

165

Authorized licensed use limited to: Monash University. Downloaded on August 29,2020 at 01:40:14 UTC from IEEE Xplore. Restrictions apply.
 2016 IEEE 12th International Colloquium on Signal Processing & its Applications (CSPA2016), 4 - 6 March 2016, Melaka, Malaysia
Proposed abnormal sound analytic algorithm is following:
INITIALIZE the system
SETUP serial port
SETUP ADC
INITIALIZE codec WM8974
START surveillance LOOP
START codec WM8974
FOR frame size
Codec WM8974 read audio input
ENDFOR
Arrange the input signal in array
Signal extraction FFT
IF Minimum Amplitude < Defined value
Amplify the input signal
END IF
IF Maximum Frequency > Defined Frequency
Classify the signal into Glass Breaking
ELSE
IF Maximum Frequency > Defined Frequency
Classify the signal into Screaming
ELSE
No Abnormality
END ELSE_IF
Transmit classification result through UART
REPEAT Surveillance LOOP
IV. EXPERIMENT AND EVALUATION RESULTS
Proposed abnormal sound analytical surveillance system is
target for indoor environment. Office was selected as our
experiment locations. Sound samples were downloaded and
played from website[12][13]. A total of 50 sounds including
of human screaming, glass breaking and no abnormality
scenario sounds were tested on our system. The performance
of proposed system was measured base on misdetection.
Indicator is measured in percentage. Misdetection is
calculating quantity of abnormal sound scenario that not
detected from the total test sound samples. No abnormality
sound types are human speech, sneeze, cough and laughing.
Our initial test result was shown as per TABLE I. Our
proposed abnormal sound analytical surveillance system is
having better detection on human screaming compare to glass
breaking sound. In overall, the detection’s accuracy is average
88%.
There are still future works to be carried out on our
proposed system. Performance assessment will be held at
different surveillance locations to collect more data to support
our proposed system’s detection accuracy.
V. CONCLUSION
In this study, an abnormal sound analytic surveillance
system using microcontroller prototype has successfully
166
Authorized licensed use limited to: Monash University. Downloaded on August 29,2020 at 01:40:14 UTC from IEEE Xplore. Restrictions apply.
developed. Proposed abnormal sound analytic system is
capable to detect human screaming and glass breaking.
ACKNOWLEDGMENT
We would like to acknowledge CREST for the project
funding. We also appreciate our industry partner Myreka Sdn
Bhd for technical support on this project.
TABLE I. ASSESSMENT RESULT AT OFFICE
Sound Type Error Rate (%)
Misdetection
Human scream 4%
Glass breaking 22%
Normal condition 14%
REFERENCES
[1] http://onlineapps.epu.gov.my/rmke10/rmke10_english.html
[2] Weimin Huang, Tuan Kiang Chiew, Haizhou Li, Tian Shiang
Kok and Jit Biswas, “Scream Detection for Home
Applications,” IEEE Industrial Electronics and Applicationsis,
pp.2115-2120, 2010.
[3] Chuan-Yu Chang and Yi-Ping Chang, “Application of
Abnormal Sound Recognition System for Indoor Environment,”
IEEE ICICS, pp., 2013.
[4] Cheung-Fat Chan and Eric W.M. Yu, “An abnormal Sound
Detection and Classification System for Surveillance
Applications,” EURASIP, pp.1851-1855, 2010.
[5] Burak Uzkent and Buket D. Barkana, “Pitch-Range Based
Feature Extraction for Audio Surveillance Systems,” IEEE
International Conference on Information technology, pp.476-
480, 2011.
[6] Laurentiu Frangu, Marius Mazarel, Claudiu Chiculita and Silviu
Epure, “Embedded System for Audio Source Separation,” IEEE
Design and Technology in Electronic Packaging, pp.185-188,
2010.
[7] Nihat Ozturk and Ulvi Unozkan, “Microprocessor Based Voice
Recognition System Realization,” Application of Information
and Communication Technologies, pp. 1-3, 2010.
[8] Clayder Gonzalez-Cadenillas and Nils Murrugarra-Llerene,
“Isolated Words Recognition Using a Low Cost
Microcontroller,” Computing Systems Engineering, pp. 77-82,
2013.
[9] Nitin Kandpal, Yashodhan Mandke and Amit Patwardhan,
“Implementation of Voice Recognition in Low Power
Microcontroller,” IACSIT Hong Kong, pp. 111-115, 2012.
[10] Wang Zhenhai, “Application for Realizing Voice Recording
Using MCU,” Fuzzy Systems and Knowledge Discovery, pp.
82-85, 2011.
[11] Microchip PIC32MX Family Reference Manual, 2008.
[12] The Freesound Project, http://www.freesound.org, 2015.
[13] www.freesound.org, 2015.