Professional Documents
Culture Documents
Textbook Intelligent Video Surveillance Systems An Algorithmic Approach First Edition Maheshkumar H Kolekar Ebook All Chapter PDF
Textbook Intelligent Video Surveillance Systems An Algorithmic Approach First Edition Maheshkumar H Kolekar Ebook All Chapter PDF
https://textbookfull.com/product/fault-tolerant-message-passing-
distributed-systems-an-algorithmic-approach-michel-raynal/
https://textbookfull.com/product/computational-network-science-
an-algorithmic-approach-1st-edition-hexmoor/
https://textbookfull.com/product/intelligent-robotic-systems-
first-edition-tzafestas/
https://textbookfull.com/product/reuse-in-intelligent-
systems-1st-edition-stuart-h-rubin-editor/
Cases and Concepts in Comparative Politics An
Integrated Approach First Edition Patrick H. O'Neil
https://textbookfull.com/product/cases-and-concepts-in-
comparative-politics-an-integrated-approach-first-edition-
patrick-h-oneil/
https://textbookfull.com/product/introduction-to-intelligent-
surveillance-surveillance-data-capture-transmission-and-
analytics-wei-qi-yan/
https://textbookfull.com/product/abnormal-psychology-an-
integrative-approach-eighth-edition-david-h-barlow/
https://textbookfull.com/product/intelligent-image-and-video-
compression-communicating-pictures-2nd-edition-david-bull/
https://textbookfull.com/product/business-process-reengineering-
an-ict-approach-first-edition-chen/
Intelligent Video
Surveillance Systems
An Algorithmic Approach
Intelligent Video
Surveillance Systems
An Algorithmic Approach
Maheshkumar H. Kolekar
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors and
publishers have attempted to trace the copyright holders of all material reproduced in this publication
and apologize to copyright holders if permission to publish in this form has not been obtained. If any
copyright material has not been acknowledged please write and let us know so we may rectify in any
future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or
hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access
www.copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc.
(CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization
that provides licenses and registration for a variety of users. For organizations that have been granted
a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.
List of Figures xv
Foreword xxi
Preface xxv
vii
viii Contents
2.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 46
2.2.1.1 Issues in Motion Segmentations . . . . . . . 46
2.2.1.2 Main Attributes of a Motion Segmentation Al-
gorithm . . . . . . . . . . . . . . . . . . . . . 47
2.2.2 Motion Segmentation Algorithms . . . . . . . . . . . . 48
2.2.2.1 Image Difference . . . . . . . . . . . . . . . . 48
2.2.2.2 Statistical Theory . . . . . . . . . . . . . . . 48
2.2.2.3 Optical Flow . . . . . . . . . . . . . . . . . . 49
2.2.2.4 Layers . . . . . . . . . . . . . . . . . . . . . . 49
2.2.2.5 Factorization Technique . . . . . . . . . . . . 50
2.3 Optical Flow Methods . . . . . . . . . . . . . . . . . . . . . . 50
2.3.1 Estimation of Optical Flow . . . . . . . . . . . . . . . 50
2.3.1.1 Horn-Schunck Optical Flow Estimation . . . 51
2.3.1.2 Lucas Kanade Optical Flow Estimation . . . 53
2.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
2.4.1 Surveillance and Security . . . . . . . . . . . . . . . . 56
2.4.2 Content-Based Video Indexing and Retrieval . . . . . 56
2.4.3 Automatic Highlight Generation of Sports Videos . . . 57
2.4.4 Traffic Monitoring . . . . . . . . . . . . . . . . . . . . 57
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
3 Background Modeling 59
3.1 What Is Background Modeling? . . . . . . . . . . . . . . . . 59
3.2 Background Modeling Techniques . . . . . . . . . . . . . . . 59
3.2.1 Non-Statistical Background Modeling Methods . . . . 61
3.2.1.1 Background Modeling Independent of Time . 61
3.2.1.2 Improved Basic Background Modeling . . . . 61
3.2.1.3 Long-Term Average Background Modeling . 62
3.2.2 Statistical Background Modeling Methods . . . . . . . 63
3.2.2.1 Example of GMM . . . . . . . . . . . . . . . 63
3.2.2.2 GMM Model . . . . . . . . . . . . . . . . . . 63
3.2.2.3 Expectation Maximization GMM Algorithm 65
3.2.2.4 GMM-Based Background Detection . . . . . 66
3.3 Shadow Detection and Removal . . . . . . . . . . . . . . . . 67
3.3.1 Shadow Removal for Traffic Flow Detection . . . . . . 69
3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
II Object Tracking 73
4 Object Classification 75
4.1 Shape-Based Object Classification . . . . . . . . . . . . . . . 76
4.2 Motion-Based Object Classification . . . . . . . . . . . . . . 76
4.2.1 Approaches . . . . . . . . . . . . . . . . . . . . . . . . 76
4.2.2 Applications . . . . . . . . . . . . . . . . . . . . . . . 77
4.3 Viola Jones Object Detection Framework . . . . . . . . . . . 77
x Contents
Bibliography 173
Index 179
List of Figures
xv
xvi List of Figures
xix
Foreword
In recent years, the scourge of terrorism has been affecting the whole world.
India has suffered a number of casualties as a result of bomb blasts in market
areas, indiscriminate firing in crowded places and suicide attacks. The ter-
rorist attack on the Parliament in December 2001 and later in Mumbai on
November 26, 2008 have resulted in intensifying counter-terrorism operations
to thwart such acts in the future. In order to effectively counter the terrorist
operations, there is the need for accurate and timely intelligence commonly
known as actionable intelligence, or, as I would like call it, predictive intelli-
gence.
There are different types of intelligence inputs, which are needed to form
a complete and reliable information package on which the security agencies
can take pre-emptive action. These can be communication intercepts, photo
imagery, human-sourced inputs, etc. However, in proximity to or within impor-
tant area such as official premises and airports, video surveillance has become
a very important tool to counter terrorists or anti-social acts. While many
cities in the country have established video-based networks as part of Smart
City projects, the suggestions from this book will greatly help the city ad-
ministration in many ways, such as control of traffic, crowd control and crime
control and detection, to name a few.
With the recently developed IP-based systems and smart cameras, which
also have filters for human facial and behavioral identification, the intelligent
video surveillance will go a long way in enhancing various applications. Un-
doubtedly, these require complex algorithmic solutions and are a daunting
task for scientists. Dr. Maheshkumar Kolekar has therefore taken a vital step
in writing this book, explaining certain concepts and solutions as well as giv-
ing illustrations, figures, and photographs to explain his hypothesis.
I am therefore delighted to introduce the book, Intelligent Video Surveil-
lance Systems, written by Dr. Maheshkumar Kolekar. I am very happy that as
a follow-up to the research project being undertaken by him at IIT Patna, he
has decided to compile the details into a book. It is also noteworthy that this
xxi
xxii Foreword
book on video surveillance is coming out in the early stages of the field, which
is under rapid development. This book therefore can play a significant role in
bringing about the understanding and subsequent maturity of the field.
Though many research papers are available, they provide limited perspec-
tives about the field, its application potential, and the techniques. This book
gives a broad view about the available techniques, implementation aspects,
emerging techniques, and algorithms. I have particularly liked the idea that
instead of searching for concepts from a variety of papers, the author has at-
tempted to introduce and compile different implementation details, which will
allow ease in further research.
Over the past few years, the author was working on a project, “Abnormal
Human Activity Recognition to Enhance Security,” which was sponsored by
the principal scientific advisor to the government of India. Being a member
of the project monitoring committee, I had the opportunity to evaluate his
project work. Since this topic is very important from a security point view, I
have always encouraged Dr. Kolekar to continue his research work in this field
and asked him to motivate more students to work in this area. As a result
of that he has initiated an elective course on intelligent video surveillance in
IIT Patna and I am happy to know that this is now a very popular course in
the institute. I came to know that many students of different disciplines such
as electrical engineering, computer science and engineering, and mechatronics
engineering joined this course in 2018. Seeing this trend, I am sure that in the
coming years, this course and resultant research will become very popular in
other technical institutes and universities, and this book will be very useful for
such courses. Also, I urge the faculty members of other institutes to initiate
similar courses, since this is a very important area in today’s scenario.
These days, many public or open places are equipped with cameras to mon-
itor the security of that area. This increased adaptation of security cameras to
survey public areas has caused a digital data explosion that human operators
have difficulty keeping up with. Even though humans are the most intelligent
creatures in this world, there are many shortcomings in human-operated sys-
tems. At the moment, the best solution to attain a safe environment requires
automatic tools for human operators to monitor the digital camera images.
Due to the fatigue factor, human observers cannot pay full attention to several
video screens for periods of long duration. In order to support human oper-
ators, automatic abnormal human activity recognition and tracking systems
are introduced in the surveillance video for security in public places such as
railway stations, airports, and banks.
Normally, for working on an intelligent video surveillance systems course,
understanding of digital image processing, digital video processing, and ma-
chine learning are pre-requisites. But to allow easier adaptation, the author
has devoted the first and second chapters to digital image processing and dig-
ital video processing, respectively, and explains machine-learning algorithms
such as HMM with implementation details in a chapter on human activity
recognition to make this book self-sufficient. The flow of the chapters is also
Foreword xxiii
logical. The first two chapters are on basic concepts in image and video pro-
cessing, to create background for the readers in this area. Chapter 3 explains
background modelling, which is the first building block of any surveillance
system. Object classification, human activity recognition, and video object
tracking are discussed in Chapters 4, 5, and 6, respectively. Chapter 7 de-
scribes various camera models used for surveillance, and in the final Chapter
8, the author discusses some techniques and applications that have been im-
plemented in various domains.
The chapters in the book describe various machine learning concepts and
demonstrate practical applications of advanced techniques, such as hidden
Markov models, support vector machine, convolutional neural network, and
deep learning. Instead of explaining many concepts in brief, the author has
tried to explain a few important concepts at length, with required imple-
mentation details for surveillance application. I am sure that by reading this
book, readers can easily implement various algorithms and understand video
surveillance-related concepts. I wish Dr. Maheshkumar Kolekar, the author
of this book, fellow researchers working in related areas, and other readers of
the book further success in their endeavors in enhancing better security and
to be of service to humanity.
Jai Hind.
xxv
xxvi Preface
daughters Samruddhi and Anwita, for their constant support and encourage-
ment. Today the dream of my mother came true.
JUSTIINA. Kysy mutkalta, kyllä väärä vastaa. Niin, Ansu saa nyt
periä tätinsä ja on se Ansu yhtä hyvä kuin joku muukin, vaikkei se
osaakaan veistellä makkaratikkuja.
EPRA. Vai synnin unesta! En minä ole kirkon kipee enkä sanan
vaivanen!
KAIKKI. Ohoh!
MESAKKI. Pois uni poikkeaa, sillä sydän lyöpi ja tykkää, kuin mato
koukkuaan itsens taas suoraks lykkää.
MESAKKI. Vaimo, älä rupea minuun, sillä kirjoitettu on, sinä olet
kiusaus.
MESAKKI. Mitä!
MESAKKI. Pläsivalakka!
JASKA (Tulee juosten takaisin). Molskis vaan, kun minä heitin sen
veteen! Herrastuomari!
JASKA. Petkutettu?
JASKA. Mutta mikset sinä epäillyt miestä vaan annoit vetää itseäsi
nenästä, sinä olet niin herkkäuskoinen, höömeli sinä olet, Epra.
EPRA. Ha, kolme, ai, ai, ai, ja se juutas vei vielä kuistin alta pussin
ja kolme taalaria. Sinä olet tehnyt minusta keppikerjäläisen, sinä,
sinä Jaska, miksi otit sen yöksi taloosi? Mesakki, herrastuomari,
minä vetoon teihin, kaikki on Jaskan syy, Jaska saa maksaa kaikki
takaisin.
EPRA. Kuules mies, älä häntä noin ylvästele, tästä tuleekin toista
vaahtoa.
Hölmöläisiä tulee.
HÖLMÖLÄISET. Ei kukaan!
EPRA. Ha!
HÖLMÖLÄISET. Veisaavat.
HÖLMÖLÄISET. Johorimuikkus!
Esirippu.