Professional Documents
Culture Documents
Perry Xiao
Copyright © 2022 by John Wiley & Sons, Inc. All rights reserved.
ISBN: 978-1-119-82086-4
ISBN: 978-1-119-82094-9 (ebk)
ISBN: 978-1-119-82096-3 (ebk)
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or
by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted
under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission
of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clear-
ance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web
at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions
Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008,
or online at http://www.wiley.com/go/permission.
Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or war-
ranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all
warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be
created or extended by sales or promotional materials. The advice and strategies contained herein may not
be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in
rendering legal, accounting, or other professional services. If professional assistance is required, the services
of a competent professional person should be sought. Neither the publisher nor the author shall be liable for
damages arising herefrom. The fact that an organization or Website is referred to in this work as a citation
and/or a potential source of further information does not mean that the author or the publisher endorses
the information the organization or Website may provide or recommendations it may make. Further, readers
should be aware the Internet Websites listed in this work may have changed or disappeared between when
this work was written and when it is read.
For general information on our other products and services or for technical support, please contact our Cus-
tomer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993
or fax (317) 572-4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic formats. For more information about Wiley products, visit our web site at
www.wiley.com.
Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc.
and/or its affiliates, in the United States and other countries, and may not be used without written permis-
sion. Python is a registered trademark of Python Software Foundation. All other trademarks are the property
of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned
in this book.
I would also like to dedicate this book to Grace Qing Wang, who sadly
passed away during the course of writing this book. Grace Qing Wang was a
young, energetic professional woman who was passionate about innovation
and artificial intelligence in education. She was also a collaborator and a
good friend. Through Grace I have made many professional connections
that were very beneficial to my career.
About the Author
Dr. Perry Xiao is a professor and course director at the School of Engineering,
London South Bank University in London, United Kingdom. He got his BEng
degree in opto-electronics, MSc degree in solid-state physics, and PhD degree
in photophysics. He is a charted engineering (CEng), a Fellow (FIET) from
the Institution of Engineering and Technology (IET), and a Senior Fellow
(SFHEA) from the Higher Education Academy (HEA). He has been teaching
electronics, software, computer networks, and telecommunication subjects
at both the undergraduate level and the postgraduate level for nearly two
decades. He also supervises BEng final project students and MSc project
students every year. His main research interest is to develop novel infrared
and electronic sensing technologies for skin bioengineering applications and
industrial nondestructive testing (NDT). To date, he has finished more than
12 PhD student supervisions, obtained two UK patent applications, published
more than 100 scientific papers, been editorial reviewer for nine journals, and
generated nearly £1 million in research grants.
He is also a director and cofounder of Biox Systems Ltd., UK, a university
spin-o ff company that designs and manufactures state-o f-t he-a rt skin
measurement instruments, AquaFlux and Epsilon, which have been used in
more than 200 organizations worldwide, including leading cosmetic com-
panies, universities, research institutes, and hospitals.
vii
About the Technical Editors
Dr. Weiheng Liao, DPhil (Oxon), is a computer scientist and technology entre-
preneur in AI. He has authored and co-authored a number of influential papers
in top journals and conferences and is the visiting scholar of several research
universities. His interests include machine learning, AutoML, deep learning,
explainable AI, natural language processing, and their applications in finance
and investment. He cofounded YouShore, one of the world’s first teams to
employ deep NLP to analyze social media data, to extract alternative data, and
to construct alpha signals.
If you want to know more about his recent work, please visit w w w
.madebydata.com.
ix
Acknowledgments
xi
Contents at a Glance
Prefacexxiii
Part I Introduction
Chapter 1 Introduction to AI 3
Chapter 2 AI Development Tools 23
Part II Machine Learning and Deep Learning
Chapter 3 Machine Learning 53
Chapter 4 Deep Learning 117
Part III AI Applications
Chapter 5 Image Classification 201
Chapter 6 Face Detection and Face Recognition 265
Chapter 7 Object Detections and Image Segmentations 337
Chapter 8 Pose Detection 433
Chapter 9 GAN and Neural-Style Transfer 465
Chapter 10 Natural Language Processing 491
Chapter 11 Data Analysis 543
Chapter 12 Advanced AI Computing 613
Index659
xiii
xiv Contents at a Glance
Prefacexxiii
Part I Introduction
Chapter 1 Introduction to AI 3
1.1 What Is AI? 3
1.2 The History of AI 5
1.3 AI Hypes and AI Winters 9
1.4 The Types of AI 11
1.5 Edge AI and Cloud AI 12
1.6 Key Moments of AI 14
1.7 The State of AI 17
1.8 AI Resources 19
1.9 Summary 21
1.10 Chapter Review Questions 22
Chapter 2 AI Development Tools 23
2.1 AI Hardware Tools 23
2.2 AI Software Tools 24
2.3 Introduction to Python 27
2.4 Python Development Environments 30
2.4 Getting Started with Python 34
2.5 AI Datasets 45
2.6 Python AI Frameworks 47
2.7 Summary 49
2.8 Chapter Review Questions 50
Part II Machine Learning and Deep Learning
Chapter 3 Machine Learning 53
3.1 Introduction 53
3.2 Supervised Learning: Classifications 55
xv
xvi Contents
Scikit-Learn Datasets 56
Support Vector Machines 56
Naive Bayes 67
Linear Discriminant Analysis 69
Principal Component Analysis 70
Decision Tree 73
Random Forest 76
K-Nearest Neighbors 77
Neural Networks 78
3.3 Supervised Learning: Regressions 80
3.4 Unsupervised Learning 89
K-means Clustering 89
3.5 Semi-supervised Learning 91
3.6 Reinforcement Learning 93
Q-Learning 95
3.7 Ensemble Learning 102
3.8 AutoML 106
3.9 PyCaret 109
3.10 LazyPredict 111
3.11 Summary 115
3.12 Chapter Review Questions 116
Chapter 4 Deep Learning 117
4.1 Introduction 117
4.2 Artificial Neural Networks 120
4.3 Convolutional Neural Networks 125
4.3.1 LeNet, AlexNet, GoogLeNet 129
4.3.2 VGG, ResNet, DenseNet, MobileNet,
EffecientNet, and YOLO 140
4.3.3 U-Net 152
4.3.4 AutoEncoder 157
4.3.5 Siamese Neural Networks 161
4.3.6 Capsule Networks 163
4.3.7 CNN Layers Visualization 165
4.4 Recurrent Neural Networks 173
4.4.1 Vanilla RNNs 175
4.4.2 Long-Short Term Memory 176
4.4.3 Natural Language Processing and Python
Natural Language Toolkit 183
4.5 Transformers 187
4.5.1 BERT and ALBERT 187
4.5.2 GPT-3 189
4.5.3 Switch Transformers 190
4.6 Graph Neural Networks 191
4.6.1 SuperGLUE 192
4.7 Bayesian Neural Networks 192
Contents xvii
The year 2020 was a year of turmoil, conflicts, and division. The most significant
event was no doubt the COVID-19 pandemic, which was, and still is, raging
in more than 200 countries and affecting the lives of hundreds of millions of
people. I spent a good part of the year working from home. There are many
disadvantages of remote working; however, it does have at least one advantage:
it saved me at least two hours a day traveling to and from work. This gave me
more time to think about, to plan, and to propose this book.
I am absolutely fascinated with artificial intelligence, and I have read many
artificial intelligence books. But most of the books are heavily focused on the
mathematics of artificial intelligence, which makes them difficult to understand
for people without mathematics or computer science backgrounds. I have
always wanted to write a book that could make it easier to get into the artificial
intelligence field for beginners—people from all different disciplines. Thanks
to the countless researchers and developers around the world and their open
source code, particularly Python-based open source code, it is much easier to
use artificial intelligence now than 10 years ago. Through this book, you will
find that you can do amazing things with just a few lines of code, and in some
cases, you don’t need to code at all.
I am a big fan of open source, and for a research field as controversial as
artificial intelligence, it is better for everyone to work together. So, I want to
express my ultimate gratitude to those who made their work available for
the benefit of others.
We are living in an era of digital revolutions and digital technologies such
as artificial intelligence, the Internet of Things, Industry 4.0, 5G technologies,
digital twin, cybersecurity, big data, cloud computing, blockchains, and, on the
horizon, quantum computing. They are all being developed at a breathtaking
xxiii
xxiv Preface
speed. In the future, the Internet of Things will provide a means to connect all
things around us and to use sensors to collect data. The industry version of the
Internet of Things is called Industry 4.0, which will connect all sorts of things for
manufacturers. Digital twin is a digital representation of a process, product, or
service updated from real-time data. With digital twin, we can predict problems
before they even occur, prevent downtime, develop new opportunities for the
future through simulations. 5G technologies will provide a means for fast and
low-latency communications for the data. Cybersecurity will provide a means
to protect the data. Big data will provide a means to analyze the data in large
quantity. Cloud computing will provide the storage, display, and analysis of
the data remotely, in the cloud. Blockchains will provide traceability to the data
through distributed ledgers. Quantum computing will make some of the com-
putation faster, in fact, many orders of magnitude faster. Artificial intelligence
will be right at the heart of all the technologies, which allows us to analyze the
data intelligently. As you can see, all these digital technologies are going to
become intertwined to make us work better and live smarter.
That is why I have always said to my students, you can change your future.
Your future is in your hands. The key is learning, even after graduation.
Learning is a lifelong mission. In today’s ever-evolving world, with all the
quickly developing digital technologies, you need to constantly reinvent your-
self; you will need to learn everything and learn anything. The disadvantage
of fast-changing technologies is that you will need to learn all the time, but
the advantage is no one has any more advantages than you; you are on the
same starting line as everyone else. The rest is up to you!
I believe artificial intelligence will be just a tool for everyone in the future,
just like software coding is today. Artificial intelligence will no doubt affect
every aspect of our lives and will fundamentally change the way we live, how
we work, and how we socialize. The more you know about artificial intelli-
gence and the more involved you are in artificial intelligence, the better you
can transform your life.
Many successful people are lifelong learners. American entrepreneur and
business magnate Elon Musk is a classic example. As the world’s richest man,
he learned many things by himself, from computer programming, Internet,
finance, to building cars and rockets. British comedian Lee Evans once said
that by the end of the day, if you have learned something new, then it is a good
day. I hope you will have a good day every day and enjoy reading this book!
Professor Perry Xiao
July 2021, London
Preface xxv
Part II
Chapter 3: Machine Learning
Chapter 4: Deep Learning
Part III
Chapter 5: Image Classifications
Chapter 6: Face Detection and Recognition
Chapter 7: Object Detections and Image Segmentations
Chapter 8: Pose Detection
Chapter 9: GAN and Neural-Style Transfer
Chapter 10: Natural Language Processing
Chapter 11: Data Analysis
Chapter 12: Advanced AI Computing
Example Code
All the example source code is available on the website that accompanies this
book.
—Nou wa!.… bromde de andere rooier, ik seg moar dá’ die ouë
rakkert van ’n Hassel main al drie keer ’n graof noà’ main harses
mept hep!.… toe’k ’n kwetiertje te loat maffe hep!.… en sain naif.…
uit de polder.… ranselt puur s’n rooiers mi de staikvork in d’r pinse.…
aas se effe wá’ segge.… onder hoarlie.… werrik!—
Bij elk woord, rukten z’n dof-groene klauwen den grond in, kluiterden
aardbonken voor ’m uit, sprak ie afgehakt en hijgend.
—Nou, die ken veur main, sain buik verliese.… giftte Hesse, z’n
zandhand in den mond uitzuigend, waar de nagel onder ’t vleesch
losgescheurd was.
—Jesis.… wá’ jai main ’n roer-menietje bin!.… lolde de ander.…
f’rleje joar he’k main duim afhakt.… hier kaik.. mi de moker.… op de
schee.… scheeringpoaltjes.… ik hep sain d’r soo moar anplak.…
hep effe moar bloed.…
Weer was ie opgesprongen naar z’n kruik, die blinkte bij nog niet
gerooid bed-end, slobberde gretig z’n lauwe slappe koffie.—
Zweetvet drupte ’m langs z’n rooiige gezicht, den bronzigen hals in.
Willem Hassel had geloerd, maar durfde toch niets zeggen, nou ie
zag dat de kerel ’m wel wat mans leek.—
Kees was van z’n boonen aan ’t wortelen wieden gezet. Zweetzwaar
en paf-heet ploeterde ie hurkend, met stekende lendepijn [302]van
ongewoonte. Driftiger grabbelde z’n hand tusschen ’t gewas, en
telkens brandde ie zich aan netels. Boordevol werkdrift zat ie. En
giftig, heftig, was ie op z’n neef Willem, nou ie gehoord had, hoe
bazig ie ’n stakker, die daar te druipen stond van zweet, had
afgebluft om ’n slok drinken in de hitte. Wat ’n branie! Nou dat ze’r ’n
beetje bovenop gekomme binne! Eerst toch net zoo arme skooiers
aa’s sai.… en mi gelukkies en gapperaie bai de bolle.… en
bloedsuigerwerk!.. nou leeke ’t heere.… Je sou se ’n skoffel op d’r
test sloàn.… ’n mestvork in d’r pins steke!.… die skarre!.…
—Seg, sou je je aige nie rais afprottetaire.… je stoan net soo mooi te
kaike hier.… Hee! foader!.… hai staikt t’ met gain poot uit!.…
Kees groeide van lol. Hij had Jaap wel de hand willen drukken, al
sprak ie uit gewoonte, ook al jaren niet met ’m.
—Lá’ sain s’n gangetje, zei norsch oom Hassel, wien bestig ’t
stramme van Willem beviel.
’n Paar dagen later kwam ouë Gerrit op ’t land, achter zijn huis.
—Nog ’n vent? en Kees dan!.… d’r blaift t’met niks of’r.… An Peters
van de Baanwaik hé’k femurge twee hoek oarbeie f’kocht.… en drie
hoek an De Gauw.… veur Duitsloànd!.… sloffies, mi en sonder
doppies.… half raip an de pluk!.…
—Gaif main ’n bonk uit je koartesak, riep Piet naar Dirk, die juist ’n
stuk brood, dat ie te veel had, weer in z’n vuilen stukkenzak wou
wegduwen.
Kees, in wije kakensperring, hapte door, gretig. Ant had weer wat op
den reutel gekregen, nou ze zagen, dat Kees aan ’t werk was.
Goedig en rustig zat de kerel, lekker zichvoelend in de
zonkoestering, blij dat ie wat kon meenemen voor Wimpie. Zelf wist
ie niet goed waarom, maar hij voelde zich zóó bestig en tevree, nu ie
zwaar-ploeteren kon en wat verdiende, dat ie den Ouë wel wat wou
zeggen, al loerde die nog zoo stug-angstig naar ’m op.
—F’rdroaid aa’s t’met ’t gewas opskiete wil.… d’r waa’s te veul kou..
tussche.… nog gain oarbaie.… t’met half Juni!
Vrouw Hassel zat op de punt van ’n stoel voor potdicht raampje, met
’r neus achter ’n groote blompot geranium, die vuurde tegen ’r
grauwe kop, en vochtige aardlucht uitwasemde. Geel-grauw stond ’r
rimpelig gezicht. Ze had zich weer driemaal [306]bevuild. Nou was ’t
uit. Guurt wou niet langer blijven, als de Ouë er niet Dokter Troost
bijhaalde. Nou, in ’n half jaar was ’t zóó verergerd, dat ze de heele
boel bedierf.
Dat zou ie nou moar erais doen … Nou moar weer es op stap noar
dokter. Dat die vervloekte kerel sain niet in ’t ziekefonds wou
naime.… kostte sain zoo de hoaren van z’n hoofd. En noar een aer
loopen kon ie ook niet. Want de notoaris waa’s dokters zwager, die
sain veurskoten had.… En van ’t winter most ie miskien weer bai
sain ankloppe veur mest!.… mi vaif persint? Neenet! hai sou moar
stiekum morge er op afstappe.… ’t waif vroage gaf tug niks!.…
—U.… ung.… a.. a.. aci.. aci.. ci.… Bor.. Bor.. Bori.. ci!..
—’t Skoap hep sekers koors.… se kwailt puur van d’r pain in d’r
saitje.…
Assistentje stond ernstiger van z’n stamper naar ’t kind te kijken, dat
met blond krullebolletje net even boven apotheekbank uitkeek, ’t
klein bang gezichtje sip òploerend naar de glanskopering en ’t rood
vonkwerk van al kleinere stampertjes, die voor d’r neusje uitgestald
stonden, naast weegschaal.—Wasem van kruiden en
benauwingslucht van warme smerige lijven, zoog door ’t winkeltje. ’n
Jongetje achter de toonbank, op ’n trapje, verrinkelde tusschen
medicijnfleschjes ’n licht schoonmaakspel. Zoete valeriaan en
jodiumstank drong benauwend door, uit den kelder.… Stil, versuft en
beangstigd van eerbied, stond ’t fondsvolkje, glanspotjes, fleschjes
en schaaltjes te bestaren, luisterend in ontzag, naar de woord-
afhakkende stem van assistentje.
Wijd lapte dikke vrouw ’r tong uit den mond, waarop wijs-
geheimzinnig, grijs assistentje staarde, zich telkens den bril
kwaadaardigjes op den neus vaststootend.
Niet goed wist ie wat anders te vragen, dan juist, dàt, wat ie dokter
zoo dikwijls had hooren zeggen. Dikke vrouw stond weer klaar, met
gretigen uitleg:
—Moar.… main kindere.… dan kaa’k nie weg.… twai legge d’r
siek.…
—Heb je geen oore, schreeuwde ie, bleekig van drift, dat de dikke
vrouw bedremmeld wegbukte door de achterstaanders, de deur
uitschonkerde.
—De klaine hoest d’r aige deur twaie.… hai leg te rille aa’s ’n oal.…
en aile datie doen.… aile!!
’n Donker behaarde kerel was ’t, hoog boven wijs brilgestaar van
assistentje uitkijkend.
—Joa … klonk dreig-zwaar z’n stem, ik.… ikke kom veur main waif..
an ’t end van Duinkaik.…
—Nou.… seg.. wacht erais effetjes!.. hep jai d’r puur effe geduld
mee.… loa me rais denke.… kaike?.… wá’ hep se tog weer said.. ’t
waif?.. je ken ’t moar nie in je kop skraive!.… Ojoà! se hep puur al.…
vaif nachte pain hier.… doar.… bai d’r bui-j’k.… se hep ’n
deureloop.… saa’k moàr klesseneere hee? en uitskaie ken se nie.…
vat je?.…
—Dá’ ken nie.… Dokter is t’r nie.… Of ze mot maar zelf hier
komme.…
—Dá’ ka’ je begraipe! Se ken puur gain kippestap doen.. nee Ouë.…
se haigt in malkoar van swakkighait.… Neenet.. ik mo dokter
hebbe.…
—Soo.. soo.. nou als je d’r op staat, zal dokter late wete wanneer ie
komme kan.…
—Daa’s net.… moar.. moar gaif jai d’r nou vast ’n drankie mee.…
hée smakwammes!.… daa’s aêr krek of ’k nie weust bin.… Sai.… sai
ken kwoalik stappe.… enne.… sai loopt puur laig.… soo slap aa’s ’n
tieme is ’t waif.
—Wa’ mot ie van main tong!.… aa’s skoap thuis te krepaire lait.…
soo sout hé’k nooit-nie.…
—Nou aa’s ’n lairelap.… mehair.… ’t slaim stoan d’r ’n duim dik op.
Achter Gerrit was ’n slank blond meisje, dochter van ’n notabele uit
stedeke ingestapt, luchtig. Luid-lachend groette ze vriendinnetje op
straat, die ’r gebracht had.
—Soo.. soo!.. dus je wil nie probeere met ’t kind? Strak drukte ie z’n
bril in neusgleufje, montuur tegen z’n ooren.
—Soo.… Soo!
—Soo, nou vrouw, jij bent lang van stof hoor! Maar ik heb je vooruit
gewaarschuwd.—
—Soo.… soo!.… maar ik zeg dat ’t nou genoeg is, barstte uit, z’n
nijdige stem; woede-doorschokt dreigde z’n gezicht.… Kort en
goed.… of ’t wel of nie in ’t reglement staat.… daar heb jij niks mee
noodig.… Wou je mij hier de wette [314]stelle?.… Al die buite
Wiereland woont, betaalt.. Dat weet je, dat wil dokter zoo. Dan wou
jij zeker, dat als jij naar Lemmer ging wone, dat dokter dan ook maar
van omstreke sprak hee?.… kort, en goed, ik heb je ’t gezegd.… En
nou marsch!.…
—Is ’t nou uit, donderde ie giftig, als jij nie dokke ken, kom jij met je
kind hier.… kort en goed!.…
—Vort wijf! daar is ’t gat van de deur, ben jij belatafeld! En jij Hassel?
—
Gerrit had verlegen staan luisteren, bang al, dat ie straks dokter
Troost zou zien.
—Nou Weimer, daa’s nie best mi’t waif.… mi de doàg ergert sai.…
dâ laikt puur toof’rai.… hoho!.… hoho!.. wá’ da’ mins skeelt.… hoho..
hoho.. ik wou juustemint dokter effe spreke.…
Stijfjes en onrustig was ouë Gerrit, met z’n pet in de hand, onnoodig
afslaand z’n kleeren, en lacherig-verlegen, duf-klein wachtkamertje
ingestapt. Stijf, in een hoekje bleef ie staan, strak op den
prachtmoestuin van dokter uitkijkend. ’n Paar meiden, wit-geschort,
stonden daar warm-blank, tusschen het goudgroen boomgewas te
lachen.—
Niks op z’n gemak voelde ie zich, beverig, bang dat dokter iets aan
’m zien zou van eigen gedachten aan z’n spullen en steelgenot.—Na
vijf minuutjes strak tuin-ingekijk, rinkelde ’n schelletje langs den
wand, kwam grijs hoofd van assistentje deuropening in, dat ’m
wenkte te volgen.
Ouë Gerrit was door vriendschaptoon van dokter altijd overstuur. Hij
wist nooit goed, wat ie met hem voor had, en hoe ie z’n antwoord
moest aanleggen. Was ie vriendelijk terug, dan keek dokter norsch.
Bleef ie stijf, dan lachte dokter ’m uit. En nou, zoo heelemaal diep in
dat prachtige huis. Van verlegenheid trok ie zich aan de baard,
haakte ie z’n handen door z’n zilveren lokken.
—Ikke hep soo.… bai main aige docht.… aa’s da’.… hoho.… aa’s
dá’ ’t waif miskien bestiges in ’n gesticht sel konne.… sai bevuilt d’r
aige.… Guurt,.… Guurt.… hoho.…