You are on page 1of 54

Computer Vision Using Deep Learning

Neural Network Architectures with


Python and Keras 1st Edition Vaibhav
Verdhan
Visit to download the full and correct content document:
https://textbookfull.com/product/computer-vision-using-deep-learning-neural-network-
architectures-with-python-and-keras-1st-edition-vaibhav-verdhan/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Supervised Learning with Python: Concepts and Practical


Implementation Using Python Vaibhav Verdhan

https://textbookfull.com/product/supervised-learning-with-python-
concepts-and-practical-implementation-using-python-vaibhav-
verdhan/

Deep Learning Projects Using TensorFlow 2: Neural


Network Development with Python and Keras 1st Edition
Vinita Silaparasetty

https://textbookfull.com/product/deep-learning-projects-using-
tensorflow-2-neural-network-development-with-python-and-
keras-1st-edition-vinita-silaparasetty/

Supervised Learning with Python Concepts and Practical


Implementation Using Python 1st Edition Vaibhav Verdhan

https://textbookfull.com/product/supervised-learning-with-python-
concepts-and-practical-implementation-using-python-1st-edition-
vaibhav-verdhan/

Deep Learning with Python Develop Deep Learning Models


on Theano and TensorFLow Using Keras Jason Brownlee

https://textbookfull.com/product/deep-learning-with-python-
develop-deep-learning-models-on-theano-and-tensorflow-using-
keras-jason-brownlee/
Beginning Anomaly Detection Using Python-Based Deep
Learning: With Keras and PyTorch Sridhar Alla

https://textbookfull.com/product/beginning-anomaly-detection-
using-python-based-deep-learning-with-keras-and-pytorch-sridhar-
alla/

Learn Keras for Deep Neural Networks: A Fast-Track


Approach to Modern Deep Learning with Python 1st
Edition Jojo John Moolayil

https://textbookfull.com/product/learn-keras-for-deep-neural-
networks-a-fast-track-approach-to-modern-deep-learning-with-
python-1st-edition-jojo-john-moolayil/

Number Systems for Deep Neural Network Architectures


1st Edition Alsuhli

https://textbookfull.com/product/number-systems-for-deep-neural-
network-architectures-1st-edition-alsuhli/

Deep Learning with Applications Using Python Chatbots


and Face, Object, and Speech Recognition With
TensorFlow and Keras Springerlink (Online Service)

https://textbookfull.com/product/deep-learning-with-applications-
using-python-chatbots-and-face-object-and-speech-recognition-
with-tensorflow-and-keras-springerlink-online-service/

Reinforcement Learning: With Open AI, TensorFlow and


Keras Using Python 1st Edition Abhishek Nandy

https://textbookfull.com/product/reinforcement-learning-with-
open-ai-tensorflow-and-keras-using-python-1st-edition-abhishek-
nandy/
Computer
Vision Using
Deep Learning
Neural Network Architectures with
Python and Keras

Vaibhav Verdhan
Computer Vision
Using Deep Learning
Neural Network Architectures
with Python and Keras

Vaibhav Verdhan
Computer Vision Using Deep Learning: Neural Network Architectures
with Python and Keras
Vaibhav Verdhan
Limerick, Ireland

ISBN-13 (pbk): 978-1-4842-6615-1 ISBN-13 (electronic): 978-1-4842-6616-8


https://doi.org/10.1007/978-1-4842-6616-8

Copyright © 2021 by Vaibhav Verdhan


This work is subject to copyright. All rights are reserved by the Publisher, whether the whole
or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical
way, and transmission or information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a
trademark symbol with every occurrence of a trademarked name, logo, or image we use the
names, logos, and images only in an editorial fashion and to the benefit of the trademark
owner, with no intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms,
even if they are not identified as such, is not to be taken as an expression of opinion as to
whether or not they are subject to proprietary rights.
While the advice and information in this book are believed to be true and accurate at the
date of publication, neither the authors nor the editors nor the publisher can accept any
legal responsibility for any errors or omissions that may be made. The publisher makes no
warranty, express or implied, with respect to the material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Aaron Black
Development Editor: James Markham
Coordinating Editor: Jessica Vakili
Distributed to the book trade worldwide by Springer Science+Business Media New York,
1 NY Plazar, New York, NY 10014. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail
orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a
California LLC and the sole member (owner) is Springer Science + Business Media Finance
Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.
For information on translations, please e-mail booktranslations@springernature.com; for
reprint, paperback, or audio rights, please e-mail bookpermissions@springernature.com.
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook
versions and licenses are also available for most titles. For more information, reference our
Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is
available to readers on GitHub via the book’s product page, located at www.apress.com/
978-1-4842-6615-1. For more detailed information, please visit http://www.apress.com/
source-code.
Printed on acid-free paper
To Yashi, Pakhi and Rudra
Table of Contents
About the Author���������������������������������������������������������������������������������xi

About the Technical Reviewer�����������������������������������������������������������xiii


Acknowledgments������������������������������������������������������������������������������xv
Introduction��������������������������������������������������������������������������������������xvii
Foreword�������������������������������������������������������������������������������������������xix

Chapter 1: Introduction to Computer Vision and Deep Learning����������1


1.1 Technical requirements�����������������������������������������������������������������������������������2
1.2 Image Processing using OpenCV��������������������������������������������������������������������3
1.2.1 Color detection using OpenCV����������������������������������������������������������������4
1.3 Shape detection using OpenCV�����������������������������������������������������������������������6
1.3.1 Face detection using OpenCV�����������������������������������������������������������������9
1.4 Fundamentals of Deep Learning�������������������������������������������������������������������12
1.4.1 The motivation behind Neural Network������������������������������������������������14
1.4.2 Layers in a Neural Network������������������������������������������������������������������15
1.4.3 Neuron��������������������������������������������������������������������������������������������������16
1.4.4 Hyperparameters����������������������������������������������������������������������������������17
1.4.5 Connections and weight of ANN�����������������������������������������������������������18
1.4.6 Bias term����������������������������������������������������������������������������������������������18
1.4.7 Activation functions������������������������������������������������������������������������������19
1.4.8 Learning rate����������������������������������������������������������������������������������������25
1.4.9 Backpropagation����������������������������������������������������������������������������������26
1.4.10 Overfitting�������������������������������������������������������������������������������������������28

v
Table of Contents

1.4.11 Gradient descent��������������������������������������������������������������������������������29


1.4.12 Loss functions������������������������������������������������������������������������������������31
1.5 How Deep Learning works?��������������������������������������������������������������������������32
1.5.1 Popular Deep Learning libraries�����������������������������������������������������������36
1.6 Summary������������������������������������������������������������������������������������������������������38
1.6.1 Further readings�����������������������������������������������������������������������������������39

Chapter 2: Nuts and Bolts of Deep Learning for Computer Vision������41


2.1 Technical requirements���������������������������������������������������������������������������������42
2.2 Deep Learning using TensorFlow and Keras�������������������������������������������������42
2.3 What is a tensor?������������������������������������������������������������������������������������������43
2.3.1 What is a Convolutional Neural Network?��������������������������������������������45
2.3.2 What is convolution?����������������������������������������������������������������������������46
2.3.3 What is a Pooling Layer?����������������������������������������������������������������������51
2.3.4 What is a Fully Connected Layer?���������������������������������������������������������52
2.4 Developing a DL solution using CNN�������������������������������������������������������������53
2.5 Summary������������������������������������������������������������������������������������������������������64
2.5.1 Further readings�����������������������������������������������������������������������������������66

Chapter 3: Image Classification Using LeNet��������������������������������������67


3.1 Technical requirements���������������������������������������������������������������������������������68
3.2 Deep Learning architectures�������������������������������������������������������������������������68
3.3 LeNet architecture����������������������������������������������������������������������������������������69
3.4 LeNet-1 architecture�������������������������������������������������������������������������������������70
3.5 LeNet-4 architecture�������������������������������������������������������������������������������������71
3.6 LeNet-5 architecture�������������������������������������������������������������������������������������72
3.7 Boosted LeNet-4 architecture�����������������������������������������������������������������������75
3.8 Creating image classification models using LeNet���������������������������������������76
3.9 MNIST classification using LeNet�����������������������������������������������������������������77

vi
Table of Contents

3.10 German traffic sign identification using LeNet�������������������������������������������84


3.11 Summary��������������������������������������������������������������������������������������������������100
3.11.1 Further readings�������������������������������������������������������������������������������101

Chapter 4: VGGNet and AlexNet Networks����������������������������������������103


4.1 Technical requirements�������������������������������������������������������������������������������104
4.2 AlexNet and VGG Neural Networks��������������������������������������������������������������104
4.3 What is AlexNet Neural Network?���������������������������������������������������������������105
4.4What is VGG Neural Network?���������������������������������������������������������������������107
4.5 VGG16 architecture�������������������������������������������������������������������������������������107
4.6 Difference between VGG16 and VGG19�������������������������������������������������������110
4.7 Developing solutions using AlexNet and VGG����������������������������������������������111
4.8 Working on CIFAR-10 using AlexNet�����������������������������������������������������������113
4.9 Working on CIFAR-10 using VGG�����������������������������������������������������������������128
4.10 Comparing AlexNet and VGG���������������������������������������������������������������������136
4.11 Working with CIFAR-100���������������������������������������������������������������������������137
4.12 Summary��������������������������������������������������������������������������������������������������138
4.12.1 Further readings�������������������������������������������������������������������������������139

Chapter 5: Object Detection Using Deep Learning����������������������������141


5.1 Technical requirements�������������������������������������������������������������������������������142
5.2 Object Detection������������������������������������������������������������������������������������������142
5.2.1 Object classification vs. object localization vs. object detection�������143
5.2.2 Use cases of Object Detection������������������������������������������������������������144
5.3 Object Detection methods���������������������������������������������������������������������������146
5.4 Deep Learning frameworks for Object Detection����������������������������������������147
5.4.1 Sliding window approach for Object Detection����������������������������������148
5.5 Bounding box approach������������������������������������������������������������������������������150

vii
Table of Contents

5.6 Intersection over Union (IoU)�����������������������������������������������������������������������152


5.7 Non-max suppression���������������������������������������������������������������������������������154
5.8 Anchor boxes����������������������������������������������������������������������������������������������155
5.9 Deep Learning architectures�����������������������������������������������������������������������157
5.9.1 Region-based CNN (R-CNN)����������������������������������������������������������������157
5.10 Fast R-CNN�����������������������������������������������������������������������������������������������160
5.11 Faster R-CNN��������������������������������������������������������������������������������������������162
5.12 You Only Look Once (YOLO)�����������������������������������������������������������������������165
5.12.1 Salient features of YOLO�������������������������������������������������������������������166
5.12.2 Loss function in YOLO�����������������������������������������������������������������������167
5.12.3 YOLO architecture�����������������������������������������������������������������������������169
5.13 Single Shot MultiBox Detector (SSD)��������������������������������������������������������172
5.14 Transfer Learning��������������������������������������������������������������������������������������177
5.15 Python implementation�����������������������������������������������������������������������������179
5.16 Summary��������������������������������������������������������������������������������������������������182
5.16.1 Further readings�������������������������������������������������������������������������������184

Chapter 6: Face Recognition and Gesture Recognition���������������������187


6.1 Technical toolkit������������������������������������������������������������������������������������������188
6.2 Face recognition�����������������������������������������������������������������������������������������188
6.2.1 Applications of face recognition���������������������������������������������������������190
6.2.2 Process of face recognition����������������������������������������������������������������192
6.2.3 DeepFace solution by Facebook���������������������������������������������������������194
6.2.4 FaceNet for face recognition��������������������������������������������������������������199
6.2.5 Python implementation using FaceNet�����������������������������������������������206
6.2.6 Python solution for gesture recognition���������������������������������������������208
6.3 Summary����������������������������������������������������������������������������������������������������217
6.3.1 Further readings���������������������������������������������������������������������������������219

viii
Table of Contents

Chapter 7: Video Analytics Using Deep Learning������������������������������221


7.1 Technical toolkit������������������������������������������������������������������������������������������222
7.2 Video processing�����������������������������������������������������������������������������������������222
7.3 Use cases of video analytics�����������������������������������������������������������������������223
7.4 Vanishing gradient and exploding gradient problem����������������������������������225
7.5 ResNet architecture������������������������������������������������������������������������������������230
7.5.1 ResNet and skip connection���������������������������������������������������������������230
7.5.2 Inception network�������������������������������������������������������������������������������234
7.5.3 GoogLeNet architecture����������������������������������������������������������������������237
7.5.4 Improvements in Inception v2������������������������������������������������������������239
7.6 Video analytics��������������������������������������������������������������������������������������������243
7.7 Python solution using ResNet and Inception v3������������������������������������������244
7.8 Summary����������������������������������������������������������������������������������������������������254
7.8.1 Further readings���������������������������������������������������������������������������������255

Chapter 8: End-to-End Model Development��������������������������������������257


8.1 Technical requirements�������������������������������������������������������������������������������258
8.2 Deep Learning project requirements����������������������������������������������������������258
8.3 Deep Learning project process�������������������������������������������������������������������262
8.4 Business problem definition�����������������������������������������������������������������������263
8.4.1 Face detection for surveillance����������������������������������������������������������265
8.4.2 Source data or data discovery phase�������������������������������������������������268
8.5 Data ingestion or data management�����������������������������������������������������������270
8.6 Data preparation and augmentation�����������������������������������������������������������272
8.6.1 Image augmentation��������������������������������������������������������������������������274
8.7 Deep Learning modeling process���������������������������������������������������������������279
8.7.1 Transfer learning��������������������������������������������������������������������������������282
8.7.2 Common mistakes/challenges and boosting performance����������������284

ix
Table of Contents

8.8 Model deployment and maintenance����������������������������������������������������������289


8.9 Summary����������������������������������������������������������������������������������������������������294
8.9.1 Further readings���������������������������������������������������������������������������������296

References����������������������������������������������������������������������������������������297
Major activation functions and layers used in CNN������������������������������������������297
Google Colab�����������������������������������������������������������������������������������������������������298

Index�������������������������������������������������������������������������������������������������303

x
About the Author
Vaibhav Verdhan is a seasoned data science
professional with rich experience spanning
across geographies and domains. He is
a hands-on technical expert and has led
multiple engagements in machine learning
and artificial intelligence. He is a leading
industry expert, is a regular speaker at
conferences and meetups, and mentors
students and professionals. Currently, he
resides in Ireland and is working as a Principal
Data Scientist.

xi
About the Technical Reviewer
Vishwesh Ravi Shrimali graduated from BITS Pilani in 2018, where he
studied mechanical engineering. Since then, he has worked with Big
Vision LLC on Deep Learning and computer vision and was involved in
creating official OpenCV AI courses. Currently, he is working at Mercedes-­
Benz Research and Development India Pvt. Ltd. He has a keen interest
in programming and AI and has applied that interest in mechanical
engineering projects. He has also written multiple blogs on OpenCV and
Deep Learning on Learn OpenCV, a leading blog on computer vision. He
has also coauthored Machine learning for OpenCV 4 (second edition) by
Packt. When he is not writing blogs or working on projects, he likes to go
on long walks or play his acoustic guitar.

xiii
Acknowledgments
I would like to express my thanks to the following people. It is the results of
their hard work and passion that are advancing this field:
Ross Girshick
Jeff Donahue
Trevor Darrell
Jitendra Malik
Shaoqing Ren
Kaiming He
Jian Sun
Christian Szegedy
Wei Liu
Yangqing Jia
Pierre Sermanet
Scott Reed
Dragomir Anguelov
Dumitru Erhan
Vincent Vanhoucke
Andrew Rabinovich
Sergey Ioffe
Jonathon Shlens
Xiangyu Zhang
Omkar M. Parkhi
Andrea Vedaldi
Andrew Zisserman
Yaniv Taigman
Ming Yang

xv
Acknowledgments

Marc’Aurelio Ranzato
Lior Wolf
Yann LeCun
Leon Bottou
Yoshua Bengio
Patrick Haffner
Sefik Ilkin Serengil

xvi
Introduction
Innovation distinguishes between a leader and a follower.
—Steve Jobs

How good is your driving? Will you drive better than an autonomous
driving system? Or do you think an algorithm will perform better than a
specialist in classifying medical images? It can be a tricky question. But
artificial intelligence has outperformed doctors in detecting lung cancer
and breast cancer by analyzing images! Ouch!
Nature has been very kind to grant us powers of sight, taste, smell,
touch, and hearing. Out of these senses, the power of sight allows us to
appreciate the beauty of the world, enjoy the colors, recognize the faces
of our family and loved ones, and above all relish this beautiful world and
life. With time, humans amplified the power of the brain and made path-­
breaking inventions and discoveries. The wheel or airplane, printing press
or clock, light bulb or personal computers – innovations have changed the
way we live, work, travel, decide, and progress. These innovations make life
simpler, easier, and far enjoyable and safe.
Data science and Deep Learning are allowing us to further enhance
the innovative buckets. Using Deep Learning, we are able to replicate
the power of vision given by nature. The computers are being trained to
perform the same tasks done by a human being. It can be detection of
colors or shape or size, classifying between a cat or a dog or a horse, or
driving on a road – the use cases are many. The solutions are applicable
for all the sectors like retail, manufacturing, BFSI, agriculture, security,
transport, pharmaceuticals, and so on.
This book is an attempt to explain the concepts of Deep Learning
and Neural Network for computer vision problems. We are examining
xvii
Introduction

convolutional Neural Networks in detail, and their various components


and attributes. We are exploring various Neural Network architectures
like LeNet, AlexNet, VGG, R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO,
ResNet, Inception, DeepFace, and FaceNet in detail. We are also developing
pragmatic solutions to tackle use cases of binary image classification,
multiclass image classification, object detection, face recognition, and
video analytics. We will use Python and Keras for the solutions. All the
codes and datasets are checked into the GitHub repo for quick access.
In the final chapter, we are studying all the steps in a Deep Learning
project – right from defining the business problem to deployment. We
are also dealing with major errors and issues faced while developing the
solutions. Throughout the book, we are providing tips and tricks for training
better algorithms, reducing the training time, monitoring the results, and
improving the solution. We are also sharing prominent research papers and
datasets which you should use to gain further knowledge.
The book is suitable for researchers and students who want to explore
and implement computer vision solutions using Deep Learning. It is
highly useful and hence recommended for professionals who intend to
explore cutting-edge technology, grasp the advanced concepts, develop
a thorough understanding of Deep Learning architectures, and get the
best practices and solutions to common computer vision challenges. It
is directed toward the business leaders who wish to implement Deep
Learning solutions in their business and gain confidence while they
communicate with their teams and clientele. Above all, for a curious
person trying to explore how Deep Learning algorithms work for solving
computer vision problems and would like to try Python.
I would like to thank Apress, Aaron, Jessica, and Vishwesh for believing
in me and giving me the chance to work on this subject. And a special
word of thanks to my family – Yashi, Pakhi, and Rudra – for the excellent
support without which it would be impossible to complete this work.
Vaibhav Verdhan, November 2020, Limerick

xviii
Foreword
Computer Vision, not too long ago the exclusive purview of science fiction,
is quickly becoming commonplace across industries, if not in society at
large. The progress in the field to emulate human vision, that most prized
of human senses, is nothing but astonishing. It was only 1957 when Russell
Kirsch scanned the world’s first photograph, a black and white image of
his boy1. By the late 1980s, the work of Sirovich and Kirby2 helped establish
face recognition as a viable technology for biometric applications.
Facebook made the technology ubiquitous, notwithstanding privacy
concerns and legal challenges action3, when in 2010, it incorporated face
recognition in its social media platform.
The capabilities of Deep Learning vision systems to interpret and
extract information from images permeates all aspects of society. Only the
most skeptical among us doubt a not too distant future with self-driving
cars outnumbering those driven by their human counterparts or computer-­
aided diagnosis (CADx) of medical images becoming an ordinary
service supplied by medical providers. Computer vision applications

1
 he Associated Press “Computer scientist, pixel inventor Russell Kirsch dead
T
at 91”, The Associated Press, AP News, August 13, 2020, https://apnews.com/
article/technology-oregon-science-portland-us-news-db92e0b593f5156da9
70c0a1e9f90944. Accessed 9 December 2020.
2
Low-Dimensional Procedure for the Characterization of Human Faces, L. Sirovich
and M Kirby, March 1987, Journal of the Optical Society of America. A, Optics and
image science 4(3):519-24
3
Natasha Singer and Mike Isaac “Facebook to Pay $550 Million to Settle Facial
Recognition Suit” The New York Times, Jan 29, 2020 , https://www.nytimes.
com/2020/01/29/technology/facebook-privacy-lawsuit-earnings.html.
Accessed 9 December 2020.

xix
Foreword

already control access to our mobile devices and can outperform human
inspectors in the tedious but critical task of inspecting for defects in all
types of manufacturing processes. That is how I met Vaibhav, or V, as he is
known to his friends and colleagues. Collaborating on methods to improve
existing computer vision systems to ensure defect-free products critical for
human vision. Not lost is an appreciation of the circular history. We teach
computers how to see; they help manufacture products vital to improve
and care for human vision.
In this book, V takes a practical and convenient approach to the
subject. The abundant use of case studies is facilitated by ready-to-use
Python code and links to datasets and other tools. The practitioner’s
learning experience is enhanced by access to the resources needed to work
in a step-by-step fashion through each case study. The book organizes the
subject into three parts. In chapters 1 through 4, V describes the nature
of Neural Networks and demystifies how they learn. Along the way, he
points out different architectures and their historical significance. The
practitioner gets to experience, with all required resources in hand, the
elegant simplicity of LeNet, the improved efficiency of AlexNet, and
the popular VGG Net. In chapters 5 through 7, the practitioner applies
simple yet powerful computer vision applications such as training
systems to detect objects and recognize human faces. When progressing
into performing video analytics, we encounter the nagging problem of
vanishing and exploding gradients and how to overcome it using skip
connections in the ResNet architecture. Finally, in chapter 8, we review
the complete model development process, starting with a correctly
defined business problem and systematically advancing until the model is
deployed and maintain in a production environment.

xx
Foreword

We are now just starting to see the dramatic increase in complexity


and impact of tasks performed by computer systems that match and often
exceed what until recently, would be considered exclusively human vision
capabilities. Those aspiring to make this technology their ally, grow more
adept at incorporating vision systems into their practice, and become a
more skillful practitioner will greatly gain from the tools, techniques, and
methods presented in this book.
David O. Ramos
Jacksonville, FL
16 December 2020

xxi
CHAPTER 1

Introduction to
Computer Vision and
Deep Learning
Vision is the best gift to mankind by God.

Right from our birth, vision allows us to develop a conscious mind. Colors,
shapes, objects, and faces are all building blocks for our world. This gift of
nature is quite central to our senses.
Computer vision is one of the capabilities which allows the machines
to replicate this power. And using Deep Learning, we are enhancing our
command and making advancements in this field.
This book will examine the concepts of computer vision in the light
of Deep Learning. We will study the basic building blocks of Neural
Networks, develop pragmatic use cases by taking a case study–based
approach, and compare and contrast the performance of various solutions.
We will discuss the best practices, share the tips and insights followed
in the industry, make you aware of the common pitfalls, and develop a
thought process to design Neural Networks.

© Vaibhav Verdhan 2021 1


V. Verdhan, Computer Vision Using Deep Learning,
https://doi.org/10.1007/978-1-4842-6616-8_1
Chapter 1 Introduction to Computer Vision and Deep Learning

Throughout the book, we introduce a concept, explore it in detail, and


then develop a use case in Python around it. Since a chapter first builds the
foundations of Deep Learning and then its pragmatic usage, the complete
knowledge enables you to design a solution and then develop Neural
Networks for better decision making.
Some knowledge of Python and object-oriented programming
concepts is expected for a good understanding. A basic to intermediate
understanding of data science is advisable though not a necessary
requirement.
In this introductory chapter, we will develop the concepts of Image
Processing using OpenCV and Deep Learning. OpenCV is a great library
and is widely used in robotics, face recognition, gesture recognition, AR,
and so on. Deep Learning, in addition, offers a higher level of complexity
and flexibility to develop Image Processing use cases. We will cover the
following topics in this chapter:

(1) Image Processing using OpenCV

(2) Fundamentals of Deep Learning

(3) How Deep Learning works

(4) Popular Deep Learning libraries

1.1 Technical requirements


We are developing all the solutions in Python throughout the book; hence,
installation of the latest version of Python is required.
All the code, datasets, and respective results are checked into a code
repository at https://github.com/Apress/computer-vision-using-
deep-learning/tree/main/Chapter1. You are advised to run all the code
with us and replicate the results. This will strengthen your grasp on the
concepts.

2
Chapter 1 Introduction to Computer Vision and Deep Learning

1.2 Image Processing using OpenCV


An image is also like any other data point. On our computers and mobile
phones, it appears as an object or icon in .jpeg, .bmp, and .png formats.
Hence, it becomes difficult for humans to visualize it in a row-column
structure, like we visualize any other database. Hence, it is often referred to
as unstructured data.
For our computers and algorithms to analyze an image and work on it,
we have to represent an image in the form of integers. Hence, we work on
an image pixel by pixel. Mathematically, one of the ways to represent each
pixel is the RGB (Red, Green, Blue) value. We use this information to do
the Image Processing.

Info The easiest way to get RGB for any color is to open it in Paint
in Windows operating system. Hover over any color and get the
respective RGB value. In Mac OS, you can use Digital Colour Meter.

Deep Learning allows us to develop use cases which are much more
complex to be resolved using traditional Image Processing techniques. For
example, detecting a face can be done using OpenCV too, but to be able to
recognize one will require Deep Learning.
During the process of developing computer vision solutions using
Deep Learning, we prepare our image dataset first. During preparation, we
might have to perform grayscaling of the images, detect contours, crop the
images, and then feed them to the Neural Network.
OpenCV is the most famous library for such tasks. As a first step, let’s
develop some building blocks of these Image Processing methods. We will
create three solutions using OpenCV.

Note Go to www.opencv.org and follow the instructions over


there to get OpenCV installed on your system.

3
Chapter 1 Introduction to Computer Vision and Deep Learning

The images used for the solutions are the commonly available
ones. You are advised to examine the code and follow the step-by-step
implementation done. We will detect shape, colors, and a face in an image.
Let’s dive into the exciting world of images!

1.2.1 Color detection using OpenCV


When we think of an image, it is made up of shapes, sizes, and colors.
Having the capability to detect shape, size, or color in an image can
automate a lot of processes and save huge efforts. In the very first example,
we are going to develop a “color detection system.”
Color detection is having a wide range of utility across domains and
industries like manufacturing, automotive, electricity, utilities, and so
on. Color detection can be used to look for abruptions, failures, and
disruptions to normal behavior. We can train sensors to take a particular
decision based on the color and raise an alarm if required.
An image is represented using pixels, and each pixel is made up of RGB
values ranging from 0 to 255. We will be using this property to identify the
blue color in the image (Figure 1-1). You can change the respective values
for the blue color and detect any color of choice.
Follow these steps:
1. Open the Python Jupyter Notebook.

2. First load the necessary libraries, numpy and OpenCV.

import numpy as np
import cv2

3. Load the image file.

image = cv2.imread('Color.png')

4
Chapter 1 Introduction to Computer Vision and Deep Learning

Figure 1-1. Raw image to be used for color detection. The image
shown has four different colors, and the OpenCV solution will detect
them individually

4. Now let us convert our raw image to HSV (Hue Saturation


Value) format. It enables us to separate from saturation and
pseudo-illumination. cv2.cvtColor allows us to do that.

hsv_convert = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

5. Define the upper and lower ranges of the color here. We


are detecting the blue color. From the numpy library, we
have given the respective range for the blue color.

lower_range = np.array([110,50,50])
upper_range = np.array([130,255,255])

6. Now, let’s detect the blue color and separate it from


the rest of the image.

mask_toput = cv2.inRange(hsv_convert, lower_range,


upper_range)
cv2.imshow('image', image)
cv2.imshow('mask', mask_toput)
while(True):
k = cv2.waitKey(5)& 0xFF if k== 27: break

5
Chapter 1 Introduction to Computer Vision and Deep Learning

The output of this code will be as shown in Figure 1-2.

Figure 1-2. Output of the colour detection system. We want to detect


blue colour which is detected and separated from rest of the image

As visible, the blue color is highlighted in white, while the rest of the
image is in black. By changing the ranges in step 5, you can detect different
colors of your choice.
With color done, it is time to detect a shape in an image; let’s do it!

1.3 Shape detection using OpenCV


Like we detected the blue color in the last section, we will detect triangle,
square, rectangle, and circle in an image. Shape detection allows you to
separate portions in the image and check for patterns. Color and shape
detections make a solution very concrete. The usability lies in safety
monitoring, manufacturing lines, automobile centers, and so on.
For Shape detection, we get the contours of each shape, check the
number of elements, and then classify accordingly. For example, if this
number is three, it is a triangle. In this solution, you will also observe how
to grayscale an image and detect contours.

6
Chapter 1 Introduction to Computer Vision and Deep Learning

Follow these steps to detect shapes:

1. Import the libraries first.

import numpy as np
import cv2

2. Load the raw image now shown in Figure 1-3.

shape_image = cv2.imread('shape.png')

Figure 1-3. Raw input image for detecting the three shapes of circle,
triangle, and rectangle

3. Convert the image to grayscale next. Grayscaling is


done for simplicity since RGB is three-dimensional
while grayscale is two-dimensional, and converting
to grayscale simplifies the solution. It also makes the
code efficient.

gray_image = cv2.cvtColor(shape_image, cv2.COLOR_BGR2GRAY)


ret,thresh = cv2.threshold(gray_image,127,255,1)

4. Find the contours in the image.

contours,h = cv2.findContours(thresh,1,2)

7
Chapter 1 Introduction to Computer Vision and Deep Learning

5. Try to approximate each of the contours using


approxPolyDP. This method returns the number of
elements in the contours detected. Then we decide the
shape based on the number of elements in the contours.
If the value is three, it is a triangle; if it’s four, it is a
square; and so on.

for cnt in contours:


    approx =
    
cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
print (len(approx))
if len(approx)==3:
    print ("triangle")
    cv2.drawContours(shape_image,[cnt],0,(0,255,0),-1)
elif len(approx)==4:
    print ("square")
    cv2.drawContours(shape_image,[cnt],0,(0,0,255),-1)
elif len(approx) > 15:
    print ("circle")
    cv2.drawContours(shape_image,[cnt],0,(0,255,255),-1)
cv2.imshow('shape_image',shape_image)   cv2.waitKey(0)
cv2.destroyAllWindows()

6. The output of the preceding code is shown in Figure 1-4.

Figure 1-4. The output of the color detection system. A circle is shown
in yellow, square is shown in red, and triangle is shown in green

8
Chapter 1 Introduction to Computer Vision and Deep Learning

You can now detect shapes in any image. We have detected a circle,
a triangle, and a square. A good challenge will be to detect a pentagon or
hexagon; are you game?
Let’s do something more fun now!

1.3.1 Face detection using OpenCV


Face detection is not a new capability. Whenever we look at a picture, we
can recognize a face quite easily. Our mobile phone camera draws square
boxes around a face. Or on social media, a square box is created around a
face. It is called face detection.
Face detection refers to locating human faces in digital images. Face
detection is different from face recognition. In the former, we are only
detecting a face in an image, whereas in the latter we are putting a name to
the face too, that is, who is the person in the photo.
Most of the modern cameras and mobiles have a built-in capability
to detect faces. A similar solution can be developed using OpenCV. It is
simpler to understand and implement and is built using the Haar-cascade
algorithm. We will highlight faces and eyes in a photograph while using
this algorithm in Python.
Haar-cascade classifier is used to detect faces and other facial
attributes like eyes in an image. It is a Machine Learning solution wherein
training is done on a lot of images which have a face and which do not have
a face in them. The classifier learns the respective features. Then we use the
same classifier to detect faces for us. We need not do any training here as the
classifier is already trained and ready to be used. Saves time and effort, too!

Info Object detection using Haar-based cascade classifiers was


proposed by Paul Viola and Michael Jones in their paper “Rapid
Object Detection using a Boosted Cascade of Simple Features” in
2001. You are advised to go through this path-breaking paper.

9
Chapter 1 Introduction to Computer Vision and Deep Learning

Follow these steps to detect a face:

1. Import the libraries first.

import numpy as np
import cv2

2. Load the classifier xml file. The .xml file is designed


by OpenCV and is created by training cascade of
negative faces superimposed on positive images and
hence can detect the facial features.

face_cascade = cv2.CascadeClassifier(
'haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier(
'haarcascade_eye.xml')

3. Next, load the image (Figure 1-5).

img = cv2.imread('Vaibhav.jpg')

Figure 1-5. Raw input image of a face which is used for the face
detection using the Haar-cascade solution

4. Convert the image to grayscale.

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

10
Another random document with
no related content on Scribd:
tariff act have been proclaimed with Portugal, with Italy and
with Germany. Commercial conventions under the general
limitations of the fourth section of the same act have been
concluded with Nicaragua, with Ecuador, with the Dominican
Republic, with Great Britain on behalf of the island of
Trinidad and with Denmark on behalf of the island of St.
Croix. These will be early communicated to the Senate.
Negotiations with other governments are in progress for the
improvement and security of our commercial relations."

The question of the ratification of all these treaties was


pending in the Senate when the term of the 56th Congress
expired, March 4, 1901. Opposing interests in the United
States seemed likely then to defeat their ratification. On the
last day of the special session of the Senate, March 5-9, an
agreement extending the time for the ratification of the
French reciprocity treaty was received and referred to the
committee on foreign relations. On the 15th of March,
Secretary Hay and Lord Pauncefote signed protocols extending
for one year the time of ratification for four of the British
West Indian reciprocity treaties, namely, Jamaica, Bermuda,
Guiana and Turks and Caicos islands.

UNITED STATES OF AMERICA: A. D. 1900.


Naval strength.

See (in this volume)


NAVIES OF THE SEA POWERS.

UNITED STATES OF AMERICA: A. D. 1900.


State of Indian schools.
Recent Indian policy.
Indian population.

See (in this volume)


INDIANS, AMERICAN: A. D. 1899-1900.
UNITED STATES OF AMERICA: A. D. 1900 (January).
Report of the First Philippine Commission.

See (in this volume)


PHILIPPINE ISLANDS: A. D. 1900 (JANUARY).

UNITED STATES OF AMERICA: A. D. 1900 (January-March).


The outbreak of the "Boxers" in northern China.

See (in this volume)


CHINA: A. D. 1900 (JANUARY-MARCH).

UNITED STATES OF AMERICA: A. D. 1900 (February).


Negotiation of the Hay-Pauncefote Convention relative
to the Nicaragua Canal.

See (in this volume)


CANAL, INTEROCEANIC: A. D. 1900 (DECEMBER).

UNITED STATES OF AMERICA: A. D. 1900 (March-December).


Passage of the Financial Bill.
Settlement of the question of the monetary standard.
The working of the act.

Legislation in the direction sought by the advocates of the


gold standard and of a reformed monetary system for the
country, whose agitations are referred to above, was attained
in the spring of 1900, by the passage of an important
"Financial Bill" which became law on the 14th of March.

See (in this volume)


UNITED STATES OF AMERICA: A. D. 1896-1898

The provisions and the effect of the Act were summarized at


the time by the Secretary of the Treasury, Mr. Gage, in a
published statement, as follows:
"The financial bill has for its first object what its title
indicates, the fixing of the standard of value and the
maintaining at a parity with that standard of all forms of
money issued or coined by the United States. It reaffirms that
the unit of value is the dollar, consisting of 25.8 grains of
gold, nine tenths fine, but from that point it goes on to make
it the duty of the Secretary of the Treasury to maintain all
forms of money issued or coined at a parity with this
standard. It puts into the hands of the Secretary ample power
to do that.
{640}
For that purpose, the bill provides in the Treasury bureaus of
issue and redemption and transfers from the general fund of
the Treasury's cash $150,000,000 in gold coin and bullion to
redemption fund, that gold to be used for the redemption of
United States notes and Treasury notes. That fund is
henceforth absolutely cut out of and separated from the cash
balance in the Treasury and the available cash balance will
hereafter show a reduction of $150,000,000 from the figures
that have heretofore prevailed. This $150,000,000 redemption
fund is to be used for no other purpose than the redemption of
United States notes and Treasury notes and those notes so
redeemed may be exchanged for the gold in the general fund or
with the public, so that the reserve fund is kept full with
gold to the $150,000,000 limit. If redemptions go on so that
the gold in this reserve fund is reduced below $100,000,000,
and the Secretary is unable to build it up to the $150,000,000
mark by exchange for gold in the general fund or otherwise, he
is given power to sell bonds and it is made his duty to
replenish the gold to the $150,000,000 mark by such means.

"The 'endless chain' is broken by a provision which prohibits


the use of notes so redeemed to meet deficiencies in the
current revenues. The act provides for the ultimate retirement
of all the Treasury notes issued in payment for silver bullion
under the Sherman act. As fast as that bullion is coined into
silver dollars Treasury notes are to be retired and replaced
with an equal amount of silver certificates. The measure
authorizes the issue of gold certificates in exchange for
deposits of gold coin, the same as at present, but suspends
that authority whenever and so long as the gold in the
redemption fund is below $100,000,000 and gives to the
Secretary the option to suspend the issue of such certificates
whenever the silver certificates and United States notes in
the general fund of the Treasury exceed $40,000,000. The bill
provides for a larger issue of silver certificates, by
declaring that hereafter silver certificates shall be issued
only in denominations of $10 and under except as to 10 per
cent. of the total volume. Room is made for this larger use of
silver certificates in the way of small bills by another
provision which makes it necessary as fast as the present
silver certificates of high denominations are broken up into
small bills to cancel a similar volume of United States notes
of small denominations and replace them with notes of
denominations of $10 and upward. Further room is made for the
circulation of small silver certificates by a clause which
permits national banks to have only one third of their capital
in denomination under $10.

"One clause of the bill which the public will greatly


appreciate is the right it gives to the Secretary to coin any
of the 1890 bullion into subsidiary silver coins up to a limit
of $100,000,000. There has for years been a scarcity of
subsidiary silver during periods of active retail trade, but
this provision will give the Treasury ample opportunity to
supply all the subsidiary silver that is needed. Another
provision that the public will greatly appreciate is the
authority given to the Secretary to recoin worn and uncurrent
subsidiary silver now in the Treasury or hereafter received.

"A distinct feature of the bill is in reference to refunding


the 3 per cent. Spanish war loan, the 2 per cent. bonds
maturing in 1907 and the 5 per cent. bonds maturing in 1904, a
total of $839,000,000, into new 2 per cent. bonds. These new 2
per cent. bonds will not be offered for sale, but will only be
issued in exchange for an equal amount, face value, of old
bonds. This exchange will save the Government, after deducting
the premium paid, nearly $23,000,000, if all the holders of the
old bonds exchange them for the new ones. National banks that
take out circulation based on the new bonds are to be taxed
only one half of 1 per cent. on the average amount of
circulation outstanding, while those who have circulation
based on a deposit of old bonds will be taxed, as at present,
1 per cent. There are some other changes in the national
banking act. The law permits national banks with $25,000
capital to be organized in places of 3,000 inhabitants or
less, whereas heretofore the minimum capital has been $50,000.
It also permits banks to issue circulation on all classes of
bonds deposited up to the par value of the bonds, instead of
90 per cent. of their face, as heretofore. This ought to make
an immediate increase in national bank circulation of
something like $24,000,000, as the amount of bonds now
deposited to secure circulation is about $242,000,000. If the
price of the new 2s is not forced so high in the market that
there is no profit left to national banks in taking out
circulation, we may also look for a material increase in
national bank circulation based on additional deposits of
bonds. National banks are permitted under the law to issue
circulation up to an amount equal to their capital. The total
capital of all national banks is $616,000,000. The total
circulation outstanding is $253,000,000. There is, therefore,
a possibility of an increase in circulation of $363,000,000,
although the price of the 2 per cent. bonds, as already
foreshadowed by market quotations in advance of their issue,
promises to be so high that the profit to the banks in taking
out circulation will not be enough to make the increase
anything like such a possible total."

Upon the working of the Act, during the first nine months of
its operation, Secretary Gage remarked as follows, in his
annual report dated December 14, 1900:
"The operation of the act of March 14 last with respect to
these two important matters of our finances has well
exemplified its wisdom. Confidence in the purpose and power of
the Government to maintain the gold standard has been greatly
strengthened. The result is that gold flows toward the
Treasury instead of away from it. At the date of this report
the free gold in the Treasury is larger in amount than at any
former period in our history. Including the $150,000,000
reserve, the gold in the Treasury belonging to the Government
amounts to over $242,000,000, while the Treasury holds,
besides, more than $230,000,000, against which certificates
have been issued. That provision of the act which liberalized
the conditions of bank-note issue was also wise and timely.
Under it, … there has been an increase of some $77,000,000 in
bank-note issues. To this fact may be chiefly attributed the
freedom from stress for currency to handle the large harvests
of cotton, wheat, and corn. In this respect the year has been
an exception to the general rule of stringency which for
several years has so plainly marked the autumn season.

{641}

"Nevertheless, the measures referred to, prolific as they have


been in good results, will yet need re-enforcement in some
important particulars. Thus, as to the redemption fund
provided for in said act, while the powers conferred upon the
Secretary are probably ample to enable a zealous and watchful
officer to protect fully the gold reserve, there appears to be
lacking sufficient mandatory requirement to furnish complete
confidence in the continued parity, under all conditions,
between our two forms of metallic money, silver and gold. Upon
this point further legislation may become desirable. As to the
currency, while the liberalizing of conditions has, as previously
noted, found response in a necessary increase of bank-note
issues, there is under our present system no assurance
whatever that the volume of bank currency will be continuously
responsive to the country's needs, either by expanding as such
needs require or by contracting when superfluous in amount.
The truth is that safe and desirable as is our currency system
in many respects, it is not properly related. The supply of
currency is but remotely, if at all, influenced by the ever
changing requirements of trade and industry. It is related
most largely, if not entirely, to the price of Government
bonds in the market."

Annual Report of the Secretary of the Treasury, 1900,


pages 72-73.

UNITED STATES OF AMERICA: A. D. 1900 (April).


Speech of Senator Hoar in denial of the right of the
government of the United States, under the Constitution, to
hold the Philippine Islands as a subject state.

On the 17th of April, the following joint resolution was under


consideration in the Senate: "Be it resolved by the Senate and
House of Representatives of the United States of America in
Congress assembled, that the Philippine Islands are territory
belonging to the United States; that it is the intention of
the United States to retain them as such and to establish and
maintain such governmental control throughout the archipelago
as the situation may demand." Senator Hoar, of Massachusetts,
spoke in opposition to the resolution, and some passages from
his speech are quoted here, because they are notably
representative of the ground and spirit of an opposition which
existed within the party controlling the government to the war
of subjugation in the Philippine Islands, to which the party
and the government were finally committed by the adoption of
this Congressional declaration. The attitude and argument of
parties on the question are set forth below, in the platforms
and manifestos of the presidential election. This speech
exhibits a feeling on that question which was overridden in
the winning party, and it has an importance both forensic and
historical:
"The American people, so far as I know, were all agreed that
their victory [in the Spanish American war] brought with it
the responsibility of protecting the liberated peoples from
the cupidity of any other power until they could establish
their own independence in freedom and in honor. I stand here
to-day to plead with you not to abandon the principles that
have brought these things to pass. I implore you to keep to
the policy that has made the country great, that has made the
Republican party great, that has made the President great. I
have nothing new to say. But I ask you to keep in the old
channels, and to keep off the old rocks laid down in the old
charts, and to follow the old sailing orders that all the old
captains of other days have obeyed, to take your bearings, as
of old, from the north star,

Of whose true fixed and resting quality


There is no fellow in the firmament,

and not from this meteoric light of empire.

"Especially, if I could, would I persuade the great Republican


party to come back again to its old faith, to its old
religion, before it is too late. There is yet time. The
President has said again and again that his is only an ad
interim policy until Congress shall act. It is not yet too
late. Congress has rejected, unwisely, as I think, some
declarations for freedom. But the two Houses have not as yet
committed themselves to despotism. The old, safe path, the
path alike of justice and of freedom, is still easy. It is a
path familiar, of old, to the Republican party. If we have
diverged from it for the first time, everything in our
history, everything in our own nature calls us back. The great
preacher of the English church tells you how easy is the
return of a great and noble nature from the first departure
from rectitude:—
'For so a taper, when its crown of flame is newly blown off,
retains a nature so symbolical to light,
that it will with greediness reenkindle and snatch a ray
from the neighbor fire.'

"I, for one, believed, and still believe that the pathway to
prosperity and glory for the country was also the pathway to
success and glory for the Republican party. I thought the two
things inseparable. If, when we made the treaty of peace, we
had adhered to the purpose we declared when we declared war;
if we had dealt with the Philippine Islands as we promised to
deal, have dealt, and expect to deal with Cuba, the country
would have escaped the loss of 6,000 brave soldiers, other
thousands of wrecked and shattered lives, the sickness of many
more, the expenditure of hundreds of millions, and, what is
far worse than all, the trampling under foot of its cherished
ideals. There would have been to-day a noble republic in the
East, sitting docile at our feet, receiving from us
civilization, laws, manners, and giving in turn everything the
gratitude of a free people could give-love, obedience, trade.
The Philippine youth would throng our universities; our
Constitution, our Declaration, the lives of Washington and
Lincoln, the sayings of Jefferson and Franklin would have been
the textbooks of their schools. How our orators and poets
would have delighted to contrast America liberating and
raising up the republic of Asia, with England subduing and
trampling under foot the republic of Africa. Nothing at home
could have withstood the great party and the great President
who had done these things. We should have come from the next
election with a solid North and have carried half the South.
You would at least have been spared the spectacle of great
Republican States rising in revolt against Republican
policies. I do not expect to accomplish anything for liberty
in the Philippine Islands but through the Republican party.
Upon it the fate of these Islands for years to come is to
depend. If that party can not be persuaded, the case is in my
judgment for the present hopeless. …
{642}

"The practical question which divided the American people last


year, and which divides them to-day, is this: Whether in
protecting the people of the Philippine Islands from the
ambition and cupidity of other nations we are bound to protect
them from our own. … In dealing with this question, Mr.
President, I do not mean to enter upon any doubtful ground. I
shall advance no proposition ever seriously disputed in this
country till within twelve months. … If to think as I do in
regard to the interpretation of the Constitution; in regard to
the mandates of the moral law or the law of nations, to which
all men and all nations must render obedience; in regard to
the policies which are wisest for the conduct of the State, or
in regard to those facts of recent history in the light of
which we have acted or are to act hereafter, be treason, then
Washington was a traitor; then Jefferson was a traitor; then
Jackson was a traitor; then Franklin was a traitor; then
Sumner was a traitor; then Lincoln was a traitor; then Webster
was a traitor; then Clay was a traitor; then Corwin was a
traitor; then Kent was a traitor; then Seward was a traitor;
then McKinley, within two years, was a traitor; then the
Supreme Court of the United States has been in the past a nest
and hotbed of treason; then the people of the United States,
for more than a century, have been traitors to their own flag
and their own Constitution.

"'We are presented with an issue that can be clearly and


sharply stated as a question of constitutional power, a
question of international law, a question of justice and
righteousness, or a question of public expediency. This can be
stated clearly and sharply in the abstract, and it can be put
clearly and sharply by an illustration growing out of existing
facts.

"The constitutional question is: Has Congress the power, under


our Constitution, to hold in subjection unwilling vassal
States?

"The question of international law is: Can any nation


rightfully convey to another sovereignty over an unwilling
people who have thrown off its dominion, asserted their
independence, established a government of their own, over whom
it has at the time no practical control, from whose territory
it has been disseized, and which it is beyond its power to
deliver?

"The question of justice and righteousness is: Have we the


right to crush and hold under our feet an unwilling and
subject people whom we had treated as allies, whose
independence we are bound in good faith to respect, who had
established their own free government, and who had trusted us?

"The question of public expediency is: Is it for our advantage


to promote our trade at the cannon's mouth and at the point of
the bayonet?

"All these questions can be put in a way of practical


illustration by inquiring whether we ought to do what we have
done, are doing, and mean to do in the case of Cuba; or what
we have done, are doing, and some of you mean to do in the
case of the Philippine Islands.

"It does not seem to me to be worth while to state again at


length the constitutional argument which I have addressed to
the Senate heretofore. It has been encountered with eloquence,
with clearness and beauty of statement, and, I have no doubt,
with absolute sincerity by Senators who have spoken upon the
other side. But the issue between them and me can be summed up
in a sentence or two, and if, so stated, it can not be made
clear to any man's apprehension, I despair of making it clear
by any elaboration or amplification. I admit that the United
States may acquire and hold property, and may make rules and
regulations for its disposition. I admit that, like other
property, the United States may acquire and hold land. It may
acquire it by purchase. It may acquire it by treaty. It may
acquire it by conquest. And it may make rules and regulations
for its disposition and government, however it be acquired.
When there are inhabitants upon the land so acquired it may
make laws for their government. But the question between me
and the gentlemen on the other side is this: Is this
acquisition of territory, of land or other property, whether
gained by purchase, conquest, or treaty, a constitutional end
or only a means to a constitutional end? May you acquire,
hold, and govern territory or other property as an end for
which our Constitution was framed, or is it only a means
toward some other and further end? May you acquire, hold, and
govern property by conquest, treaty, or purchase for the sole
object of so holding and governing it, without the
consideration of any further constitutional purpose? Or must
you hold it for a constitutional purpose only, such as the
making of new States, the national defense and security, the
establishment of a seat of government, or the construction of
forts, harbors, and like works, which, of course, are
themselves for the national defense and security?

"I hold that this acquisition, holding and governing, can be


only a means for a constitutional end—the creation of new
States or some other of the constitutional purposes to which I
have adverted. And I maintain that you can no more hold and
govern territory than you can hold and manage cannon or fleets
for any other than a constitutional end; and I maintain that
the holding in subjection an alien people, governing them
against their will for any fancied advantage to them, is not
only not an end provided for by the Constitution, but is an
end prohibited therein. … It is an end which the generation
which framed the Constitution and the Declaration of
Independence declared was unrighteous and abhorrent. So, in my
opinion, we have no constitutional power to acquire territory
for the purpose of holding it in subjugation, in a state of
vassalage or serfdom, against the will of its people.

"It is to be noted just here that we have acquired no


territory or other property in the Philippine Islands, save at
few public buildings. By every other acquisition of territory
the United States became a great land owner. She owned the
public lands as she had owned the public lands in the
Northwest ceded to her by the old States. But you own nothing
in the Philippines. The people own their farms and dwellings
and cities. The religious orders own the rest. The Filipinos
desire to do what our English ancestors did in the old days
when England was Catholic. The laity feared that the Church
would engross all the land. So they passed their statute of
mortmain. You have either got to let the people of the
Philippine Islands settle this matter for themselves, or you
must take upon you the delicate duty of settling it for them.
{643}
Your purchase or conquest is a purchase or conquest of nothing
but sovereignty. It is a sovereignty over a people who are
never to be admitted to exercise it or to share it. In the
present case we have not, I repeat, bought any property. We
have undertaken to buy mere sovereignty. There were no public
lands in the Philippine Islands, the property of Spain, which
we have bought and paid for. The mountains of iron and the
nuggets of gold and the hemp-bearing fields—do you purpose to
strip the owners of their rightful title? We have undertaken
to buy allegiance, pure and simple. And allegiance is just
what the law of nations declares you can not buy. …

"I have been unable to find a single reputable authority more


than twelve months old for the power now claimed for Congress
to govern dependent nations or territories not expected to
become States. The contrary, until this war broke out, has
been taken as too clear for reasonable question. I content
myself with a few authorities. Among them are Daniel Webster,
William H. Seward, the Supreme Court of the United States,
James Madison.
"Daniel Webster said in the Senate, March 23, 1848: 'Arbitrary
governments may have territories and distant possessions,
because arbitrary governments may rule them by different laws
and different systems. We can do no such thing. They must be
of us, part of us, or else strangers. I think I see a course
adopted which is likely to turn the Constitution of the land
into a deformed monster, into a curse rather than a blessing;
in fact, a frame of an unequal government, not founded on
popular representation, not founded on equality, but on the
grossest inequality; and I think that this process will go on,
or that there is danger that it will go on, until this Union
shall fall to pieces. I resist it to-day and always! Whoever
falters or whoever flies, I continue the contest!'

"James Madison said in the Federalist: 'The object of the


Federal Constitution is to secure the union of the thirteen
primitive States, which we know to be practicable; and to add
to them such other States as may arise in their own bosoms, or
in their neighborhood, which we can not doubt will be
practicable.'

James Madison,
Federalist, Number 14.

"William H. Seward said: 'It is a remarkable feature of the


Constitution of the United States that its framers never
contemplated colonies, or provinces, or territories at all. On
the other hand, they contemplated States only, nothing less
than States, perfect States, equal States, as they are called
here, sovereign States. … There is reason—there is sound
political wisdom in this provision of the Constitution
excluding colonies, which are always subject to oppression,
and excluding provinces, which always tend to corrupt and
ultimately to break down the parent State.'

Seward's Works
Volume 1, page 122.

'By the Constitution of the United States, there are no


subjects, Every citizen of any one State is a free and equal
citizen of the United States. Again, by the Constitution of
the United States there are no permanent provinces or
dependencies.'

Seward's Works
Volume 4, page 167.

"The Supreme Court of the United States, in the case of


Fleming vs. Page, said: 'The genius and character of our
institutions are peaceful; and the power to declare war was
not conferred upon Congress for the purposes of aggression or
aggrandizement, but to enable the Government to vindicate by
arms, if it should become necessary, its own rights and the
rights of its citizens. A war, therefore, declared by
Congress, can never be presumed to be waged for the purpose of
conquest or the acquisition of territory; nor does the law
declaring the war imply an authority to the President to
enlarge the limits of the United States by subjugating the
enemy's country.'

"Our territories, so far, have all been places where Americans


would go to dwell as citizens, to establish American homes, to
obtain honorable employment, and to build a State. Will any
man go to the Philippine Islands to dwell, except to help
govern the people, or to make money by a temporary residence?

"When hostilities broke out, February 5, 1899, we had no


occupancy of and no title of any kind to any portion of the
Philippine territory, except the town and bay of Manila.
Everything else was in the peaceful possession of the
inhabitants. In such a condition of things, Mr. President,
international law speaks to us with its awful mandate. It
pronounces your proposed action sheer usurpation and robbery.
You have no better title, according to the law of nations, to
reduce this people to subjection than you have to subjugate
Mexico or Haiti or Belgium or Switzerland. This is the settled
doctrine, as declared by our own great masters of
jurisprudence. You have no right, according to tho law of
nations, to obtain by purchase or acquisition sovereignty over
a people which is not actually exercised by the country which
undertakes to convey it or yield it. We have not yet completed
the acquisition. But at the time we entered upon it, and at
the time of this alleged purchase, the people of the
Philippine Islands, as appears by General Otis's report, by
Admiral Dewey's report, and the reports of officers for whom
they vouched, held their entire territory, with the exception
of the single town of Manila. They had, as appears from these
reports, a full organized government. They had an army
fighting for independence, admirably disciplined, according to
the statement of zealous advocates of expansion.

"Why, Mr. President, is it credible that any American


statesman, that any American Senator, that any intelligent
American citizen anywhere, two years ago could have been found
to affirm that a proceeding like that of the Paris treaty
could give a just and valid title to sovereignty over a people
situated as were the people of those islands? A title of
Spain, originally by conquest, never submitted to nor admitted
by the people of the islands, with frequent insurrections at
different times for centuries, and then the yoke all thrown
off, a constitutional government, schools, colleges, churches,
universities, hospitals, town governments, a legislature, a
cabinet, courts, a code of laws, and the whole island occupied
and controlled by its people, with the single exception of one
city; with taxes lawfully levied and collected, with an army
and the beginning of a navy.

{644}
"And yet the Senate, the Congress enacted less than two years
ago that the people of Cuba—controlling peaceably no part of
their island, levying no taxes in any orderly or peaceable
way, with no administration of justice, no cabinet—not only of
right ought to be, but were, in fact, a free and independent
State. I did not give my assent to that declaration of fact. I
assented to the doctrine that they of right ought to be. But I
thought the statement of fact much calculated to embarrass the
Government of the United States, if it were bound by that
declaration; and it has been practically disregarded by the
Administration ever since. But the question now is a very
different one. You not only deny that the Filipinos are, but
you deny that they of right ought to be free and independent;
and you recognize Spain as entitled to sell to you the
sovereignty of an island where she was not at the time
occupying a foot of territory, where her soldiers were held
captives by the government of the island, a government to
which you had delivered over a large number of Spanish
prisoners to be held as captives. And yet you come here to-day
and say that they not only are not, but they of right ought
not to be free and independent; and when you are pressed you
answer us by talking about mountains of iron and nuggets of
gold, and trade with China.

"I affirm that you can not get by conquest, and you can not
get by purchase, according to the modern law of nations,
according to the law of nations as accepted and expounded by
the United States, sovereignty over a people, or title to a
territory, of which the power that undertakes to sell it or
the power from whom you undertake to wrest it has not the
actual possession and dominion. … You cannot buy a war. More
than this, you cannot buy a tyrant's claim to subject again an
oppressed people who have achieved their freedom. …

"Gentlemen tell us that the bill of the Senator from Wisconsin


is copied from that introduced in Jefferson's time for the
purchase of Louisiana. Do you claim that you propose to deal
with these people as Jefferson meant to deal with Louisiana?
You talk of Alaska, of Florida, of California; do you mean to
deal with the Philippines as we mean to deal with Alaska and
dealt with Florida or California?

"I have spoken of the Declaration of Independence as a solemn


affirmation of public law, but it is far more than that. It is
a solemn pledge of national faith and honor. It is a baptismal
vow. It is the bedrock of our republican institutions. It is,
as the Supreme Court declared, the soul and spirit of which
the Constitution is but the body and letter. It is the light
by which the Constitution must be read. … There is expansion
enough in it, but it is the expansion of freedom and not of
despotism; of life, not of death. Never was such growth in all
human history as that from the seed Thomas Jefferson planted.
The parable of the mustard seed, than which, as Edward Everett
said, 'the burning pen of inspiration, ranging heaven and
earth for a similitude, can find nothing more appropriate or
expressive to which to liken the Kingdom of God,' is repeated
again. 'Whereunto shall we liken it, or with what comparison
shall we compare it? It is like a grain of mustard seed,
which, when it is sown in the earth, is less than all the
seeds that be in the earth. But when it is sown, it groweth
up, and becometh greater than all herbs, and shooteth out
great branches, so that the fowls of the air may lodge under
the shadow of it.' This is the expansion of Thomas Jefferson.
It has covered the continent. It is on both the seas. It has
saved South America. It is revolutionizing Europe. It is the
expansion of freedom. It differs from your tinsel, pinchbeck,
pewter expansion as the growth of a healthy youth into a
strong man differs from the expansion of an anaconda when he
swallows his victim. Ours is the expansion of Thomas
Jefferson. Yours is the expansion of Aaron Burr. It is
destined to as short a life and to a like fate. …

"There are 1,200 islands in the Philippine group. They extend


as far as from Maine to Florida. They have a population
variously estimated at from 8,000,000 to 12,000,000. There are
wild tribes who never heard of Christ, and islands that never
heard of Spain. But among them are the people of the island of
Luzon, numbering 3,500,000, and the people of the Visayan
Islands, numbering 2,500,000 more. They are a Christian and
civilized people. They wrested their independence from Spain
and established a republic. Their rights are no more to be
affected by the few wild tribes in their own mountains or by
the dwellers in the other islands than the rights of our old
thirteen States were affected by the French in Canada, or the
Six Nations of New York, or the Cherokees of Georgia, or the
Indians west of the Mississippi. Twice our commanding
generals, by their own confession, assured these people of
their independence. Clearly and beyond all cavil we formed an
alliance with them. We expressly asked them to co-operate with
us. We handed over our prisoners to their keeping; we sought
their help in caring for our sick and wounded. We were told by
them again and again and again that they were fighting for
independence. Their purpose was as well known to our generals,
to the War Department, and to the President, as the fact that
they were in arms. We never undeceived them until the time
when hostilities were declared in 1899. The President declared
again and again that we had no title and claimed no right to
anything beyond the town of Manila. Hostilities were begun by
us at a place where we had no right to be, and were continued
by us in spite of Aguinaldo's disavowal and regret and offer
to withdraw to a line we should prescribe. If we crush that
republic, despoil that people of their freedom and
independence, and subject them to our rule, it will be a story
of shame and dishonor. …

"But we are told if we oppose the policy of our imperialistic


and expanding friends we are bound to suggest some policy of
our own as a substitute for theirs. We are asked what we would
do in this difficult emergency. It is a question not difficult
to answer. I for one am ready to answer it.
"1. I would declare now that we will not take these islands to
govern them against their will.

"2. I would reject a cession of sovereignty which implies that


sovereignty may be bought and sold and delivered without the
consent of the people. Spain had no rightful sovereignty over
the Philippine Islands. She could not rightfully sell it to
us. We could not rightfully buy it from her.

"3. I would require all foreign governments to keep out of


these islands.

"4. I would offer to the people of the Philippines our help in


maintaining order until they have a reasonable opportunity to
establish a government of their own.

{645}

"5. I would aid them by advice, if they desire it, to set up a


free and independent government.

"6. I would invite all the great powers of Europe to unite in


an agreement that that independence shall not be interfered
with by us, by themselves, or by any one of them with the
consent of the others. As to this I am not so sure. I should
like quite as well to tell them it is not to be done whether
they consent or not.

"7. I would declare that the United States will enforce the
same doctrine as applicable to the Philippines that we
declared as to Mexico and Haiti and the South American
Republics. It is true that the Monroe Doctrine, a doctrine
based largely on our regard for our own interests, is not
applicable either in terms or in principle to a distant
Asiatic territory. But, undoubtedly, having driven out Spain,
we are bound, and have the right, to secure to the people we
have liberated an opportunity, undisturbed and in peace, to

You might also like