MATLAB Machine Learning Recipes: A

Problem-Solution Approach 3rd Edition

Michael Paluszek
Visit to download the full and correct content document:
https://ebookmass.com/product/matlab-machine-learning-recipes-a-problem-solution-
approach-3rd-edition-michael-paluszek/
 MATLAB Machine
Learning Recipes
A Problem-Solution Approach

Third Edition

Michael Paluszek
Stephanie Thomas
MATLAB Machine Learning Recipes: A Problem-Solution Approach

Michael Paluszek Stephanie Thomas

Plainsboro, NJ Plainsboro, NJ
USA USA

ISBN-13 (pbk): 978-1-4842-9845-9 ISBN-13 (electronic): 978-1-4842-9846-6

https://doi.org/10.1007/978-1-4842-9846-6

Copyright © 2024 by Michael Paluszek and Stephanie Thomas

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material
is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, repro-
duction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic
adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.
Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every
occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion
and to the benefit of the trademark owner, with no intention of infringement of the trademark.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary
rights.
While the advice and information in this book are believed to be true and accurate at the date of publication, neither
the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may
be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.
Managing Director, Apress Media LLC: Welmoed Spahr
Acquisitions Editor: Celestin Suresh John
Development Editor: Laura Berendson
Coordinating Editor: Mark Powers

Cover designed by eStudioCalamar

Cover image by Mohamed Nohassi@Unsplash.com
Distributed to the book trade worldwide by Apress Media, LLC, 1 New York Plaza, New York, NY 10004, U.S.A.
Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit
www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science
+ Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.
For information on translations, please e-mail booktranslations@springernature.com; for reprint, paperback, or
audio rights, please e-mail bookpermissions@springernature.com.
Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are
also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://
www.apress.com/bulk-sales.
Any source code or other supplementary material referenced by the author in this book is available to readers
on GitHub (https://github.com/Apress). For more detailed information, please visit https://www.apress.com/gp/
services/source-code.

Paper in this product is recyclable

To our families.
Contents

About the Authors XVII

About the Technical Reviewer XIX

Introduction XXI

1 An Overview of Machine Learning 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Elements of Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.2 Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.3 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 The Learning Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Taxonomy of Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5.1 Kalman Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5.2 Adaptive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.6 Autonomous Learning Methods . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6.1 Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6.2 Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.6.3 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.6.4 Support Vector Machines (SVMs) . . . . . . . . . . . . . . . . . . 16
1.7 Artificial Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.7.1 What Is Artificial Intelligence? . . . . . . . . . . . . . . . . . . . . 17
1.7.2 Intelligent Cars . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.7.3 Expert Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2 Data for Machine Learning in MATLAB 21

2.1 Introduction to MATLAB Data Types . . . . . . . . . . . . . . . . . . . . . 21
2.1.1 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.1.2 Cell Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.1.3 Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.1.4 Numerics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.1.5 Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

V
CONTENTS

2.1.6 Datastore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.1.7 Tall Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.1.8 Sparse Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.1.9 Tables and Categoricals . . . . . . . . . . . . . . . . . . . . . . . . 30
2.1.10 Large MAT-Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2 Initializing a Data Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.3 mapreduce on an Image Datastore . . . . . . . . . . . . . . . . . . . . . . . 36
2.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.4 Processing Table Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.5 String Concatenation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.6 Arrays of Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
2.7 Substrings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.7.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.7.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.7.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.8 Reading an Excel Spreadsheet into a Table . . . . . . . . . . . . . . . . . . . 44
2.8.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.8.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.8.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.9 Accessing ChatGPT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.9.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.9.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.9.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.10 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

VI
 CONTENTS

3 MATLAB Graphics 49
3.1 2D Line Plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
3.2 General 2D Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.3 Custom Two-Dimensional Diagrams . . . . . . . . . . . . . . . . . . . . . . 54
3.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.4 Three-Dimensional Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.5 Draw a 3D Object with a Texture . . . . . . . . . . . . . . . . . . . . . . . . 59
3.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.6 General 3D Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.7 Building a GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.7.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.7.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.7.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.8 Animating a Bar Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.8.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.8.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.8.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.9 Drawing a Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.9.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.9.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.9.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
3.10 Importing a Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.10.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.10.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.10.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

VII
CONTENTS

4 Kalman Filters 85
4.1 Gaussian Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.2 A State Estimator Using a Linear Kalman Filter . . . . . . . . . . . . . . . . 87
4.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
4.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.3 Using the Extended Kalman Filter for State Estimation . . . . . . . . . . . . 106
4.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.4 Using the UKF for State Estimation . . . . . . . . . . . . . . . . . . . . . . 111
4.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
4.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
4.5 Using the UKF for Parameter Estimation . . . . . . . . . . . . . . . . . . . . 117
4.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
4.6 Range to a Car . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

5 Adaptive Control 127

5.1 Self-Tuning: Tuning an Oscillator . . . . . . . . . . . . . . . . . . . . . . . 128
5.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
5.2 Implement MRAC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.3 Generating a Square Wave Input . . . . . . . . . . . . . . . . . . . . . . . . 140
5.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.4 Demonstrate MRAC for a Rotor . . . . . . . . . . . . . . . . . . . . . . . . 142
5.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
5.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

VIII
 CONTENTS

5.5 Ship Steering: Implement Gain Scheduling for Steering Control of a Ship . . 145
5.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
5.6 Spacecraft Pointing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
5.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
5.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
5.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
5.7 Direct Adaptive Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5.7.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5.7.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5.7.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

6 Fuzzy Logic 157

6.1 Building Fuzzy Logic Systems . . . . . . . . . . . . . . . . . . . . . . . . . 158
6.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
6.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
6.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
6.2 Implement Fuzzy Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.3 Window Wiper Fuzzy Controller . . . . . . . . . . . . . . . . . . . . . . . . 169
6.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.4 Simple Discrete HVAC Fuzzy Controller . . . . . . . . . . . . . . . . . . . . 174
6.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.5 Variable HVAC Fuzzy Controller . . . . . . . . . . . . . . . . . . . . . . . . 180
6.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
6.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

7 Neural Aircraft Control 191

7.1 Longitudinal Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
7.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
7.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
7.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

IX
CONTENTS

7.2 Numerically Finding Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . 198

7.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
7.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.3 Numerical Simulation of the Aircraft . . . . . . . . . . . . . . . . . . . . . . 200
7.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
7.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
7.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
7.4 Activation Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
7.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
7.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
7.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
7.5 Neural Net for Learning Control . . . . . . . . . . . . . . . . . . . . . . . . 204
7.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
7.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
7.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
7.6 Enumeration of All Sets of Inputs . . . . . . . . . . . . . . . . . . . . . . . 208
7.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.7 Write a Sigma-Pi Neural Net Function . . . . . . . . . . . . . . . . . . . . . 210
7.7.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
7.7.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
7.7.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
7.8 Implement PID Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.8.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.8.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.8.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.9 PID Control of Pitch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
7.9.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
7.9.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
7.9.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
7.10 Neural Net for Pitch Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . 223
7.10.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
7.10.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
7.10.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
7.11 Nonlinear Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.11.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.11.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.11.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.12 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

X
 CONTENTS

8 Introduction to Neural Nets 229

8.1 Daylight Detector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
8.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
8.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
8.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
8.2 Modeling a Pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
8.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
8.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
8.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
8.3 Single Neuron Angle Estimator . . . . . . . . . . . . . . . . . . . . . . . . . 235
8.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
8.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
8.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
8.4 Designing a Neural Net for the Pendulum . . . . . . . . . . . . . . . . . . . 240
8.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
8.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
8.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
8.5 XOR Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
8.6 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
8.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

9 Classification of Numbers Using Neural Networks 257

9.1 Generate Test Images with Defects . . . . . . . . . . . . . . . . . . . . . . . 257
9.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
9.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
9.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
9.2 Create the Neural Net Functions . . . . . . . . . . . . . . . . . . . . . . . . 262
9.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
9.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
9.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
9.3 Train a Network with One Output Node . . . . . . . . . . . . . . . . . . . . 267
9.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
9.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
9.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
9.4 Testing the Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
9.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
9.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
9.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
9.5 Train a Network with Many Outputs . . . . . . . . . . . . . . . . . . . . . . 273
9.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
9.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
9.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
9.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
XI
CONTENTS

10 Data Classification with Decision Trees 279

10.1 Generate Test Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
10.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
10.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
10.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
10.2 Drawing Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
10.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
10.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
10.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
10.3 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
10.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
10.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
10.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
10.4 Creating a Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
10.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
10.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
10.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
10.5 Handmade Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
10.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
10.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
10.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
10.6 Training and Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
10.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
10.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
10.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
10.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

11 Pattern Recognition with Deep Learning 303

11.1 Obtain Data Online for Training a Neural Net . . . . . . . . . . . . . . . . . 305
11.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
11.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
11.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
11.2 Generating Training Images of Cats . . . . . . . . . . . . . . . . . . . . . . 305
11.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
11.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
11.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
11.3 Matrix Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
11.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
11.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
11.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
11.4 Convolution Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
11.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311

XII
 CONTENTS

11.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311

11.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
11.5 Pooling to Outputs of a Layer . . . . . . . . . . . . . . . . . . . . . . . . . . 312
11.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
11.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312
11.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
11.6 Fully Connected Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
11.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
11.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
11.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
11.7 Determining the Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
11.7.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
11.7.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
11.7.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
11.8 Test the Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
11.8.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
11.8.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
11.8.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
11.9 Recognizing an Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
11.9.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
11.9.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
11.9.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
11.10 Using AlexNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
11.10.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
11.10.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
11.10.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

12 Multiple Hypothesis Testing 327

12.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
12.2 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
12.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
12.2.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
12.2.3 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
12.2.4 Measurement Assignment and Tracks . . . . . . . . . . . . . . . . 333
12.2.5 Hypothesis Formation . . . . . . . . . . . . . . . . . . . . . . . . . 334
12.2.6 Track Pruning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
12.3 Billiard Ball Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
12.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
12.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
12.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
12.4 Billiard Ball MHT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342

XIII
CONTENTS

12.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342

12.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
12.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
12.5 One-Dimensional Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
12.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345
12.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
12.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347
12.6 One-Dimensional MHT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
12.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
12.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
12.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
12.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

13 Autonomous Driving with MHT 355

13.1 Automobile Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
13.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
13.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
13.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
13.2 Automobile Radar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
13.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
13.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
13.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
13.3 Passing Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
13.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
13.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
13.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
13.4 Automobile Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
13.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363
13.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
13.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
13.4.4 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
13.5 Automobile Simulation and the Kalman Filter . . . . . . . . . . . . . . . . . 367
13.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
13.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
13.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
13.6 Automobile Target Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 371
13.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
13.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
13.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
13.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

XIV
 CONTENTS

14 Spacecraft Attitude Determination 377

14.1 Star Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
14.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
14.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
14.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
14.2 Camera Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
14.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
14.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
14.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
14.3 Celestial Sphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
14.3.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
14.3.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
14.3.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383
14.4 Attitude Simulation of Camera Views . . . . . . . . . . . . . . . . . . . . . 384
14.4.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
14.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
14.4.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
14.5 Yaw Angle Rotation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
14.5.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
14.5.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
14.5.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
14.6 Yaw Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
14.6.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
14.6.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
14.6.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 388
14.7 Attitude Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
14.7.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
14.7.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
14.7.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
14.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

15 Case-Based Expert Systems 401

15.1 Building Expert Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
15.1.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
15.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
15.1.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
15.2 Running an Expert System . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
15.2.1 Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
15.2.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
15.2.3 How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
15.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410

XV
CONTENTS

A A Brief History 411

B Software for Machine Learning 419

Bibliography 431

Index 435

XVI
About the Authors

Michael Paluszek is President of Princeton Satellite Systems,

Inc. (PSS) in Plainsboro, New Jersey. Michael founded PSS
in 1992 to provide aerospace consulting services. He used
MATLAB to develop the control system and simulations for the
Indostar-1 geosynchronous communications satellite. This led to
the launch of Princeton Satellite Systems’ first commercial MAT-
LAB toolbox, the Spacecraft Control Toolbox, in 1995. Since
then, he has developed toolboxes and software packages for air-
craft, submarines, robotics, and nuclear fusion propulsion, result-
ing in Princeton Satellite Systems’ current extensive product line.
He is working with the Princeton Plasma Physics Laboratory on a compact nuclear fusion re-
actor for energy generation and space propulsion. He is also leading the development of new
power electronics for fusion power systems and working on heat engine–based auxiliary power
systems for spacecraft. Michael is a lecturer at the Massachusetts Institute of Technology.
Prior to founding PSS, Michael was an engineer at GE Astro Space in East Windsor, NJ.
At GE, he designed the Global Geospace Science Polar despun platform control system and led
the design of the GPS IIR attitude control system, the Inmarsat-3 attitude control systems, and
the Mars Observer delta-V control system, leveraging MATLAB for control design. Michael
also worked on the attitude determination system for the DMSP meteorological satellites. He
flew communication satellites on over 12 satellite launches, including the GSTAR III recovery,
the first transfer of a satellite to an operational orbit using electric thrusters.
At Draper Laboratory, Michael worked on the Space Shuttle, Space Station, and submarine
navigation. His Space Station work included designing Control Moment Gyro–based control
systems for attitude control.
Michael received his bachelor’s degree in Electrical Engineering and master’s and engi-
neer’s degrees in Aeronautics and Astronautics from the Massachusetts Institute of Technology.
He is the author of numerous papers and has over a dozen US patents. Michael is the au-
thor of MATLAB Recipes, MATLAB Machine Learning, Practical MATLAB Deep Learning:
A Projects-Based Approach, Second Edition, all published by Apress, and ADCS: Spacecraft
Attitude Determination and Control by Elsevier.

XVII
ABOUT THE AUTHORS

Stephanie Thomas is Vice President of Princeton Satellite Sys-

tems, Inc. in Plainsboro, New Jersey. She received her bache-
lor’s and master’s degrees in Aeronautics and Astronautics from
the Massachusetts Institute of Technology in 1999 and 2001.
Stephanie was introduced to the PSS Spacecraft Control Tool-
box for MATLAB during a summer internship in 1996 and has
been using MATLAB for aerospace analysis ever since. In her
over 20 years of MATLAB experience, she has developed many
software tools, including the Solar Sail Module for the Spacecraft
Control Toolbox, a proximity satellite operations toolbox for the
Air Force, collision monitoring Simulink blocks for the Prisma
satellite mission, and launch vehicle analysis tools in MATLAB and Java. She has developed
novel methods for space situation assessment, such as a numeric approach to assessing the
general rendezvous problem between any two satellites implemented in both MATLAB and
C++. Stephanie has contributed to PSS’ Spacecraft Attitude and Orbit Control textbook, featur-
ing examples using the Spacecraft Control Toolbox, and written many software User’s Guides.
She has conducted SCT training for engineers from diverse locales such as Australia, Canada,
Brazil, and Thailand and has performed MATLAB consulting for NASA, the Air Force, and
the European Space Agency. Stephanie is the author of MATLAB Recipes, MATLAB Machine
Learning, and Practical MATLAB Deep Learning published by Apress. In 2016, she was named
a NASA NIAC Fellow for the project “Fusion-Enabled Pluto Orbiter and Lander.” Stephanie
is an Associate Fellow of the American Institute of Aeronautics and Astronautics (AIAA) and
Vice Chair of the AIAA Nuclear and Future Flight Propulsion committee. Her ResearchGate
profile can be found at https://www.researchgate.net/profile/Stephanie-Thomas-2.

XVIII
About the Technical Reviewer

Joseph Mueller took a new position as Principal Astrodynamics

Engineer at Millennium Space Systems in 2023, where ground-
breaking capabilities in space are being built, one small satellite
at a time, and he is honored to be a part of it.
From 2014 to 2023, he was a senior researcher at Smart In-
formation Flow Technologies, better known as SIFT. At SIFT, he
worked alongside amazing people, playing in the sandbox of in-
credibly interesting technical problems. His research projects at
SIFT included navigation and control for autonomous vehicles,
satellite formation flying, space situational awareness, and robotic
swarms.
Joseph is married and is a father of three, living in Champlin, MN.
His Google Scholar profile can be found at https://scholar.google.com/citations?hl=en&
user=breRtVUAAAAJ and his ResearchGate profile at www.researchgate.net/profile/Joseph-
Mueller-2.

XIX
Introduction

Machine Learning is becoming important in every engineering discipline. For example:

1. Autonomous cars: Machine learning is used in almost every aspect of car control systems.

2. Plasma physicists use machine learning to help guide experiments on fusion reactors.
TAE Technologies has used it with great success in guiding fusion experiments. The
Princeton Plasma Physics Laboratory (PPPL) has used it for the National Spherical Torus
Experiment to study a promising candidate for a nuclear fusion power plant.

3. It is used in finance for predicting the stock market.

4. Medical professionals use it for diagnoses.

5. Law enforcement and others use it for facial recognition. Several crimes have been solved
using facial recognition!

6. An expert system was used on NASA’s Deep Space 1 spacecraft.

7. Adaptive control systems steer oil tankers.

There are many, many other examples.

While many excellent packages are available from commercial sources and open source
repositories, it is valuable to understand how these algorithms work. Writing your own al-
gorithms is valuable both because it gives you insight into the commercial and open source
packages and also because it gives you the background to write your custom Machine Learning
software specialized for your application.
MATLAB had its origins for that very reason. Scientists who needed to do operations
on matrices used numerical software written in FORTRAN. At the time, using computer lan-
guages required the user to go through the write-compile-link-execute process which was time-
consuming and error-prone. MATLAB presented the user with a scripting language that allowed
the user to solve many problems with a few lines of a script that executed instantaneously. MAT-
LAB has built-in visualization tools that helped the user better understand the results. Writing
MATLAB was a lot more productive and fun than writing FORTRAN.
The goal of MATLAB Machine Learning Recipes: A Problem-Solution Approach is to help
all users harness the power of MATLAB to solve a wide range of learning problems. The book
has something for everyone interested in Machine Learning. It also has material that will allow
people with an interest in other technology areas to see how Machine Learning, and MATLAB,
can help them solve problems in their areas of expertise.

XXI
INTRODUCTION

Using the Included Software

This textbook includes a MATLAB toolbox that implements the examples. The toolbox
consists of

1. MATLAB functions

2. MATLAB scripts

3. HTML help

The MATLAB scripts implement all of the examples in this book. The functions encapsulate the
algorithms. Many functions have built-in demos. Just type the function name in the command
window, and it will execute the demo. The demo is usually encapsulated in a subfunction. You
can copy out this code for your demos and paste it into a script. For example, type the function
name PlotSet into the command window, and the plot in Figure 1 will appear.
1 >> PlotSet

cos
1
A
B
0.5

0
y

-0.5

-1
0 100 200 300 400 500 600 700 800 900 1000
x

sin
1

0.5

0
y

-0.5

-1
0 100 200 300 400 500 600 700 800 900 1000
x

Figure 1: Example plot from the function PlotSet.m

XXII
 INTRODUCTION

If you open the function, you will see the demo:

1 %%% PlotSet>Demo
2 function Demo
3
4 x = linspace(1,1000);
5 y = [sin(0.01*x);cos(0.01*x);cos(0.03*x)];
6 disp('PlotSet: One x and two y rows')
7 PlotSet( x, y, 'figure title', 'PlotSet Demo',...
8 'plot set',{[2 3], 1},'legend',{{'A' 'B'},{}},'plot title',{'cos','
sin'});

You can use these demos to start your scripts. Some functions, like right-hand-side functions
for numerical integration, don’t have demos. If you type a function name at the command line
that doesn’t have a built-in demo, you will get an error as in the code snippet below.
1 >> RHSAutomobileXY
2 Error using RHSAutomobileXY (line 17)
3 A built-in demo is not available.

The toolbox is organized according to the chapters in this book. The folder names are
Chapter_01, Chapter_02, etc. In addition, there is a General folder with functions that support
the rest of the toolbox. In addition, you will need the open source package GLPK (GNU Linear
Programming Kit) to run some of the code. Nicolo Giorgetti has written a MATLAB mex
interface to GLPK that is available on SourceForge and included with this toolbox. The interface
consists of

1. glpk.m

2. glpkcc.mexmaci64, or glpkcc.mexw64, etc.

3. GLPKTest.m

which are available from https://sourceforge.net/projects/glpkmex/. The second item is the

mex file of glpkcc.cpp compiled for your machine, such as Mac or Windows. Go to www.gnu.
org/software/glpk/ to get the GLPK library and install it on your system. If needed, download
the GLPKMEX source code as well and compile it for your machine, or else try another of the
available compiled builds.

XXIII
CHAPTER 1

An Overview of Machine Learning

1.1 Introduction
Machine Learning is a field in computer science where data is used to predict, or respond to,
future data. It is closely related to the fields of pattern recognition, computational statistics, and
artificial intelligence. The data may be historical or updated in real time. Machine learning is
important in areas like facial recognition, spam filtering, content generation, and other areas
where it is not feasible, or even possible, to write algorithms to perform a task.
For example, early attempts at filtering junk emails had the user write rules to determine
what was junk or spam. Your success depended on your ability to correctly identify the attributes
of the message that would categorize an email as junk, such as a sender address or words in the
subject, and the time you were willing to spend to tweak your rules. This was only moderately
successful as junk mail generators had little difficulty anticipating people’s handmade rules.
Modern systems use machine learning techniques with much greater success. Most of us are
now familiar with the concept of simply marking a given message as “junk” or “not junk” and
take for granted that the email system can quickly learn which features of these emails identify
them as junk and prevent them from appearing in our inbox. This could now be any combination
of IP or email addresses and words and phrases in the subject or body of the email, with a variety
of matching criteria. Note how the machine learning in this example is data driven, autonomous,
and continuously updating itself as you receive emails and flag them. However, even today, these
systems are not completely successful since they do not yet understand the “meaning” of the
text that they are processing.
Content generation is an evolving area. By training engines over massive data sets, the
engines can generate content such as music scores, computer code, and news articles. This has
the potential to revolutionize many areas that have been exclusively handled by people.
In a more general sense, what does machine learning mean? Machine learning can mean
using machines (computers and software) to gain meaning from data. It can also mean giving
machines the ability to learn from their environment. Machines have been used to assist humans
for thousands of years. Consider a simple lever, which can be fashioned using a rock and a length
of wood, or an inclined plane. Both of these machines perform useful work and assist people,
but neither can learn. Both are limited by how they are built. Once built, they cannot adapt to
changing needs without human interaction.

© The Author(s), under exclusive license to APress Media, LLC, part of Springer Nature 2024 1
M. Paluszek, S. Thomas, MATLAB Machine Learning Recipes,
https://doi.org/10.1007/978-1-4842-9846-6 1
CHAPTER 1 OVERVIEW

Machine learning involves using data to create a model that can be used to solve a problem.
The model can be explicit, in which case the machine learning algorithm adjusts the model’s
parameters, or the data can form the model. The data can be collected once and used to train a
machine learning algorithm, which can then be applied. For example, ChatGPT scrapes textual
data from the Internet to allow it to generate text based on queries. An adaptive control system
measures inputs and command responses to those inputs to update parameters for the control
algorithm.
In the context of the software we will be writing in this book, machine learning refers to
the process by which an algorithm converts the input data into parameters it can use when
interpreting future data. Many of the processes used to mechanize this learning derive from
optimization techniques and, in turn, are related to the classic field of automatic control. In
the remainder of this chapter, we will introduce the nomenclature and taxonomy of machine
learning systems.

1.2 Elements of Machine Learning

This section introduces key nomenclature for the field of machine learning.

1.2.1 Data
All learning methods are data driven. Sets of data are used to train the system. These sets may
be collected and edited by humans or gathered autonomously by other software tools. Control
systems may collect data from sensors as the systems operate and use that data to identify pa-
rameters or train the system. Content generation systems scour the Internet for information. The
data sets may be very large, and it is the explosion of data storage infrastructure and available
databases that is largely driving the growth in machine learning software today. It is still true that
a machine learning tool is only as good as the data used to create it, and the selection of training
data is practically a field in itself. Selection of data for many systems is highly automated.

NOTE When collecting data for training, one must be careful to ensure that the time
variation of the system is understood. If the structure of a system changes with time, it may be
necessary to discard old data before training the system. In automatic control, this is sometimes
called a forgetting factor in an estimator.

1.2.2 Models
Models are often used in learning systems. A model provides a mathematical framework for
learning. A model is human-derived and based on human observations and experiences. For
example, a model of a car, seen from above, might be that it is rectangular with dimensions that
fit within a standard parking spot. Models are usually thought of as human-derived and provide
a framework for machine learning. However, some forms of machine learning develop their
models without a human-derived structure.

2
 CHAPTER 1 OVERVIEW

1.2.3 Training
A system which maps an input to an output needs training to do this in a useful way. Just as
people need to be trained to perform tasks, machine learning systems need to be trained. Train-
ing is accomplished by giving the system an input and the corresponding output and modifying
the structure (models or data) in the learning machine so that mapping is learned. In some ways,
this is like curve fitting or regression. If we have enough training pairs, then the system should
be able to produce correct outputs when new inputs are introduced. For example, if we give
a face recognition system thousands of cat images and tell it that those are cats, we hope that
when it is given new cat images it will also recognize them as cats. Problems can arise when you
don’t give it enough training sets, or the training data is not sufficiently diverse, for instance,
identifying a long-haired cat or hairless cat when the training data is only of short-haired cats.
A diversity of training data is required for a functioning algorithm.
Supervised Learning
Supervised learning means that specific training sets of data are applied to the system. The
learning is supervised in that the “training sets” are human-derived. It does not necessarily
mean that humans are actively validating the results. The process of classifying the systems’
outputs for a given set of inputs is called “labeling.” That is, you explicitly say which results are
correct or which outputs are expected for each set of inputs.
The process of generating training sets can be time-consuming. Great care must be taken
to ensure that the training sets will provide sufficient training so that when real-world data is
collected, the system will produce correct results. They must cover the full range of expected
inputs and desired outputs. The training is followed by test sets to validate the results. If the
results aren’t good, then the test sets are cycled into the training sets, and the process is repeated.
A human example would be a ballet dancer trained exclusively in classical ballet technique.
If they were then asked to dance a modern dance, the results might not be as good as required
because the dancer did not have the appropriate training sets; their training sets were not suffi-
ciently diverse.
Unsupervised Learning
Unsupervised learning does not utilize training sets. It is often used to discover patterns in data
for which there is no “right” answer. For example, if you used unsupervised learning to train
a face identification system, the system might cluster the data in sets, some of which might be
faces. Clustering algorithms are generally examples of unsupervised learning. The advantage
of unsupervised learning is that you can learn things about the data that you might not know in
advance. It is a way of finding hidden structures in data.

3
CHAPTER 1 OVERVIEW

Semi-supervised Learning
With this approach, some of the data are in the form of labeled training sets, and other data are
not [12]. Typically, only a small amount of the input data is labeled, while most are not, as the
labeling may be an intensive process requiring a skilled human. The small set of labeled data is
leveraged to interpret the unlabeled data.
Online Learning
The system is continually updated with new data [12]. This is called “online” because many of
the learning systems use data collected while the system is operating. It could also be called
recursive learning. It can be beneficial to periodically “batch” process data used up to a given
time and then return to the online learning mode. The spam filtering systems collect data from
emails and update their spam filter. Generative deep learning systems like ChatGPT use massive
online learning.

1.3 The Learning Machine

Figure 1.1 shows the concept of a learning machine. The machine absorbs information from
the environment and adapts. The inputs may be separated into those that produce an immediate
response and those that lead to learning. In some cases, they are completely separate. For ex-
ample, in an aircraft, a measurement of altitude is not usually used directly for control. Instead,
it is used to help select parameters for the actual control laws. The data required for learning
and regular operation may be the same, but in some cases, separate measurements or data will
be needed for learning to take place. Measurements do not necessarily mean data collected by
a sensor such as radar or a camera. It could be data collected by polls, stock market prices, data
in accounting ledgers, or any other means. Machine learning is then the process by which the
measurements are transformed into parameters for future operation.

Measurements (Learning)

Learning

Parameters Actions

Machine
Environment
Actions

Measurements (Immediate Use)

Figure 1.1: A learning machine that senses the environment and stores data in memory

4
 CHAPTER 1 OVERVIEW

Note that the machine produces output in the form of actions. A copy of the actions may
be passed to the learning system so that it can separate the effects of the machine’s actions
from those of the environment. This is akin to a feedforward control system, which can result
in improved performance.
A few examples will clarify the diagram. We will discuss a medical example, a security
system, and spacecraft maneuvering.
A doctor might want to diagnose diseases more quickly. They would collect data on tests
on patients and then collate the results. Patient data might include age, height, weight, historical
data like blood pressure readings and medications prescribed, and exhibited symptoms. The
machine learning algorithm would detect patterns so that when new tests were performed on a
patient, the machine learning algorithm would be able to suggest diagnoses or additional tests to
narrow down the possibilities. As the machine learning algorithm was used, it would, hopefully,
get better with each success or failure. Of course, the definition of success or failure is fuzzy. In
this case, the environment would be the patients themselves. The machine would use the data
to generate actions, which would be new diagnoses. This system could be built in two ways.
In the supervised learning process, test data and known correct diagnoses are used to train the
machine. In an unsupervised learning process, the data would be used to generate patterns that
might not have been known before, and these could lead to diagnosing conditions that would
normally not be associated with those symptoms.
A security system might be put into place to identify faces. The measurements are camera
images of people. The system would be trained with a wide range of face images taken from
multiple angles. The system would then be tested with these known persons and its success rate
validated. Those that are in the database memory should be readily identified, and those that are
not should be flagged as unknown. If the success rate was not acceptable, more training might
be needed, or the algorithm itself might need to be tuned. This type of face recognition is now
common, used in Mac OS X’s “Faces” feature in Photos, face identification on the new iPhone
X, and Facebook when “tagging” friends in photos.
For precision maneuvering of a spacecraft, the inertia of the spacecraft needs to be known.
If the spacecraft has an inertial measurement unit that can measure angular rates, the inertia
matrix can be identified. This is where machine learning is tricky. The torque applied to the
spacecraft, whether by thrusters or momentum exchange devices, is only known to a certain
degree of accuracy. Thus, the system identification must sort out, if it can, the torque scaling
factor from the inertia. The inertia can only be identified if torques are applied. This leads to
the issue of stimulation. A learning system cannot learn if the system to be studied does not
have known inputs, and those inputs must be sufficiently diverse to stimulate the system so that
the learning can be accomplished. Training a face recognition system with one picture will not
work.

5
CHAPTER 1 OVERVIEW

1.4 Taxonomy of Machine Learning

In this book, we take a larger view of machine learning than is normal. Machine learning as de-
scribed earlier is the collecting of data, finding patterns, and doing useful things based on those
patterns. We expand machine learning to include adaptive and learning control. These fields
started independently but now are adapting technology and methods from machine learning.
Figure 1.2 shows how we organize the technology of machine learning into a consistent
taxonomy. You will notice that we created a title that encompasses three branches of learning;
we call the whole subject area “Autonomous Learning.” That means learning without human
intervention during the learning process. This book is not solely about “traditional” machine
learning. Other, more specialized books focus on any one of the machine learning topics. Op-
timization is part of the taxonomy because the results of optimization can be discoveries, such
as a new type of spacecraft or aircraft trajectory. Optimization is also often a part of learning
systems.

Autonomous
Learning

Control Machine Learning

State Inductive
Estimation Learning

Pattern
Adaptive Recognition Expert
Control Systems

Data Mining
System

Fuzzy Logic

Optimal
Control Optimization

Figure 1.2: Taxonomy of machine learning. The dotted lines show connections between branches

6
 CHAPTER 1 OVERVIEW

There are three categories under Autonomous Learning. The first is Control. Feedback con-
trol is used to compensate for uncertainty in a system or to make a system behave differently
than it would normally behave. If there was no uncertainty, you wouldn’t need feedback. For
example, if you are a quarterback throwing a football at a running player, assume for a moment
that you know everything about the upcoming play. You know exactly where the player should
be at a given time, so you can close your eyes, count, and just throw the ball to that spot. As-
suming the player has good hands, you would have a 100% reception rate! More realistically,
you watch the player, estimate the player’s speed, and throw the ball. You are applying feedback
to the problem. As stated, this is not a learning system. However, if now you practice the same
play repeatedly, look at your success rate, and modify the mechanics and timing of your throw
using that information, you would have an adaptive control system, the second box from the top
of the control list. Learning in control takes place in adaptive control systems and also in the
general area of system identification.
System identification is learning about a system. By system, we mean the data that rep-
resents anything and the relationships between elements of that data. For example, a particle
moving in a straight line is a system defined by its mass, the force on that mass, its velocity,
and its position. The position is related to the velocity times time, and the velocity is related and
determined by the acceleration, which is the force divided by the mass.
Optimal control may not involve any learning. For example, what is known as full-state
feedback produces an optimal control signal but does not involve learning. In full-state feed-
back, the combination of model and data tells us everything we need to know about the system.
However, in more complex systems, we can’t measure all the states and don’t know the param-
eters perfectly, so some form of learning is needed to produce “optimal” or the best possible re-
sults. In a learning system, optimal control would need to be redefined as the system learns. For
example, an optimal space trajectory assumes thruster characteristics. As a mission progresses,
the thruster performance may change, requiring recomputation of the “optimal” trajectory.
System identification is the process of identifying the characteristics of a system. A sys-
tem can, to a first approximation, be defined by a set of dynamical states and parameters. For
example, in a linear time-invariant system, the dynamical equation is

ẋ = Ax + Bu
. (1.1)

where A and B are matrices of parameters, u is an input vector, and x is the state vector. System
identification would find A and B. In a real system, A and B are not necessarily time invariant,
and most systems are only linear to a first approximation.
The second category is what many people consider true Machine Learning. This is mak-
ing use of data to produce behavior that solves problems. Much of its background comes from
statistics and optimization. The learning process may be done once in a batch process or con-
tinually in a recursive process. For example, in a stock buying package, a developer might have
processed stock data for several years, say before 2008, and used that to decide which stocks
to buy. That software might not have worked well during the financial crash. A recursive pro-
gram would continuously incorporate new data. Pattern recognition and data mining fall into
this category. Pattern recognition is looking for patterns in images. For example, the early AI

7
Another random document with
no related content on Scribd:
them well between cloths; and make a pickle of the following
ingredients:
Six eschalots, minced
White peppercorns 2 oz.
Mace, bruised 1½ oz.
Nutmeg, sliced 1½ oz.
Common table salt 6 oz.
White-wine vinegar 5 pints
Skim this well, boiling it fifteen minutes, and, filling jars with the fruit,
pour the liquor and spices equally upon them, when about new milk
warm, and tie bladder over the jars.

beet-roots.
Pickled beet-roots which have both fine colour and flavour to
recommend them are seldom to be met with, particularly in the
provinces. If this method is tried, it will most certainly recommend
them. Take half a dozen roots of the deepest blood-red colour, put
them into a pail of cold water, and with a soft brush scour and wash
them well, and without breaking the skin in the least. Put them into a
saucepan of boiling water, and let them boil gently until tender, and
no longer, then take them up, wipe dry, and leave them until the next
day. Now peel them nicely, and cut them across in slices a quarter of
an inch thick, not using the extremities. You may cut the slices into
various ornamental and grotesque figures, and lay them in open-
mouthed jars, and make the following pickle:
Mace 1 oz.
Cloves, bruised 2 oz.
Peppercorns 2 oz.
Bay salt, pounded 4 oz.
Ginger, sliced 2 oz.
Horseradish, sliced 1 oz.
Best vinegar ½ gallon
Boil these ten or fifteen minutes, skimming well, and, when cold,
pour over the roots. Replenish the next day what pickle may have
been absorbed, and cover the jars with bladder and leather. This
pickle is ready in a month, and is very good. It makes a beautiful
garnish with fish at dinner, &c. &c.

button mushrooms, for pies and sauces.

Pick out expressly for this purpose a couple of quarts of fresh
gathered button mushrooms, cut the stalks out closely, and wipe
them singly with a piece of soft flannel dipped in moistened bay salt,
place them apart on dishes and scatter a little finely beaten salt
amongst them. Put them into a roomy saucepan along with
Mace, slightly bruised ½ oz.
White peppercorns, slightly
bruised ¾ oz.
Bay leaves, shred ½ oz.
Cloves, bruised ¼ oz.
and let them simmer gently and until all their juice is taken up. Take
the pan from the fire, and when cooled a little add to them four or five
glasses of good white wine, agitate the mushrooms in the pan,
replace it on the fire and bring it to the boil for one minute, then add
three pints of the best pale vinegar and boil for ten minutes slowly.
Now put the mushrooms into glass or stone jars that are clean and
perfectly dry, and when cold make secure with corks or bladder, and
keep them in a dry cool room. This is an estimable pickle, and will be
appreciated duly by lady-cooks, who best know its usefulness and
the various ways in which both the mushrooms and their pickle may
be made available. The wine should be good old Madeira, and the
quantity may be increased with great advantage.

green parsley.
Take fresh green curled parsley just at maturity, pick out the most
handsome sprigs and put them into salt and water strong enough to
float an egg, and let remain so for five or six days; set them to drain
on a sieve, and then immerse them in another fresh pickle of the
same strength for ten days longer, changing the brine twice. Then
drain them again, and put them into pure cold spring water for two
days, changing the water daily, and when again drained scald them
in boiling water until they are of a nice green, and dry them between
soft cloths. Make, then, the following pickle of
Mace ½ oz.
Nutmeg, sliced 1 oz.
Eschalots, minced 1 oz.
Horseradish, sliced 2 oz.
White-wine vinegar 3 pints
which must be boiled ten or twelve minutes and well skimmed. Put
the parsley branches lightly into jars and pour the pickle over,
covering well. Fill up again with pickle the next day, and cover that
again with pure olive oil to the thickness of an inch or thereabouts.
Cover close with wetted bladder, and over that, when dried, with soft
leather, and keep in a dry airy room.

walnut catsup.
When walnuts have attained maturity, and are being deprived of
the outside green shells by the fruiterers, take half a peck of these
husks, put them into a jar, and pour on them as much cold strong
pickling vinegar as will quite cover them; bung up the jar, and so let
them remain three months. Then press out the liquor upon a sieve,
and to every gallon of it take
Cloves 1 oz.
Mace ¾ oz.
Ginger 1½ oz.
Jamaica pepper 1½ oz.
Black pepper 1 oz.
Garlic 1 oz.
 Port wine lees 1½ quart
Anchovies 8 oz.
With all these boil up the liquor of the walnuts, and let them simmer
twenty minutes, skimming well the whole time, then put it aside for
two days and boil it again until reduced one-third part. When cold,
you may put it in bottles, which cork well and seal with wax. It will be
an excellent catsup, and will be greatly improved by long keeping.

mushroom catsup.
Throw large black flap mushrooms into a vessel, and crush them
with the hands well, throwing in a large handful of common salt to
each peck, and let them so lie for two days. Then put them into a
crock of earthenware, and let them be macerated in a cool baker’s
oven for six hours or so, and, when cold, press out the juice, which
boil with the following, to each gallon of the liquor:
Mace ½ oz.
Jamaica pepper 1 oz.
Black pepper 1 oz.
Cloves 1½ oz.
Ginger 1 oz.
Garlic 1 oz.
Bay salt 9 oz.
The simmering and skimming must be continued as long as any filth
rises, and let it then be put away for a day or two, and boiled up
again, being kept well up to the boiling point until reduced to half its
original quantity. When cold it may be put into bottles and firmly
corked and waxed.

tomato catsup.
When tomatoes are fully ripe take two dozen of fine, large, sound
ones, put them into jars and bake until they are tender; strain off the
water from them, and pass the pulp through a sieve, then add to
every pound of the pulp,
Eschalots, shred 1 oz.
Garlic, shred ½ oz.
Bay salt ¼ oz.
White pepper, finely
powdered ¼ oz.
Chili vinegar 1 pint
Boil them together until the whole is quite soft, and pass it again
through a sieve. Now, to every pound of the pulp add the juice of two
lemons, and one large Seville orange, boil it again until it has
attained the consistence of thick cream, and when cold bottle it; cork
and seal well.

celery, crab salad.

Open and wash thoroughly clean a fine head of celery, wipe dry,
and cut it across into a basin, add to it two ounces of some good rich
old cheese sliced thinly, a teaspoonful of mustard mixed as for the
table, a tablespoonful of best olive oil, the same quantity of vinegar,
with pepper and salt to your taste. Mix all well together.

elder-flower vinegar.
Pick out all the stalks from a peck of fresh elder flowers and put
them into a vessel with two gallons of white-wine vinegar, set them
under the influence of bright sunbeams for fourteen days and
upwards, or at a short distance from a continuous fire, and then filter
the vinegar through a new flannel bag; fill bottles, which must be well
corked and sealed.

tarragon vinegar.
Take the leaves of tarragon just before it blossoms, put a pound of
them to three quarts of the best white-wine vinegar in a stone jar,
and let them infuse sixteen days. Then drain it and strain through a
flannel bag; add for every two gallons a quarter of an ounce of
isinglass dissolved in sherry wine, and let it be agitated briskly in a
large stone bottle two days. Leave it a month to get fine, then draw it
off into clean dry glass bottles, which cork well and seal.

white-gooseberry vinegar.
Vinegars should be made at home if you wish to rely upon their
quality. This will be superior to any white-wine vinegar, “so called at
the shops,” and as such will be extremely serviceable in all large
establishments and families. Choose fruit of the lightest colour you
can get when fully ripe, mash it with a wooden mallet or potato
beetle. To every peck of the fruit put two gallons of water, stir them
well for an hour and let them ferment three weeks, repeating the
stirring daily. Then strain off the liquor and add for every gallon:
Loaf sugar 1 lb.
Yeast, thick and fresh 1 tablespoonful
Treacle 1 tablespoonful
Let it work for three or four days, then put it into a sweet barrel of
convenient size, and stop it down for twelve months.

syrup d’orgeat, a paris receipt.

This elegant syrup is thus made:
Sweet almonds 20 oz.
Bitter almonds 8 oz.
Refined sugar 9 lb.
Water 4 pints
Blanch the almonds, dry them perfectly and pound them in a mortar
with the sugar, adding gradually two thirds of the water; strain
through linen, and wash the almonds on the strainer with the
remainder of the water, and dissolve the sugar in the strained liquor
by a gentle heat. Pour the syrup into an earthenware vessel, remove
the scum, and, when nearly cold, add two ounces of orange-flower
water. Put it into clean clear glass bottles, cork well, and seal
effectually.

an excellent curry-powder.
Turmeric 2 oz.
Coriander seeds 6 oz.
Ginger ½ oz.
Cinnamon 2 drachms
Cayenne pepper 6 drachms
Black pepper ½ oz.
Mace 1 drachm
Fenugreek 1½ oz.
Pimento 2 drachms
Cloves 1 drachm
Nutmeg ½ oz.
Pound all the above separately in a mortar, mix thoroughly for twenty
minutes, then sift and again pound the returns, which, when in finest
powder, mix with bulk; put into dry bottles, cork them well and seal.
Some persons prefer more turmeric and less coriander. Others add
two ounces of the best Durham mustard (scorched). Others, half an
ounce of cardamoms or two ounces of cummin. The colour should
be light yellow—brown, not bright yellow.

notes.
It has been incontestably proved by Baron Liebig and other
Professors of Chemistry, that the albumen and gelatine constitute
the leading nutritive ingredients in the different kinds of flesh and fish
used as food; and I have arrived at the conclusion, that any mode of
curing which deprives them of these valuable properties, is opposed
to facts in science and to common-sense, and cannot therefore be
tolerated.
On the nutritive properties of animal food, Professor Brande
writes: “When the muscular parts of animals are washed repeatedly
in cold water, the fibrinous matter which remains, consists chiefly of
albumen, and is, in its chemical properties, analogous to the clot of
blood.”
In mutton, the albumen or fibrin amounts to as much as twenty-
two per cent., and of gelatine to seven per cent., giving a total of
twenty-nine per cent. of nutritive matter. In beef, the albumen is
twenty, and the gelatine six per cent., yielding a total of twenty-six
per cent. of nutritive matter.
When a piece of meat is covered with salt, or immersed in brine,
the salt penetrates the whole fibre of the flesh, and the juices
contained within are drawn out, and mix with the brine; the salts of
potass contained in it, are exchanged and superseded by those of
soda, derived from the salt with which it has been cured; now, as a
constant supply of potass is required in the system to renew the
muscular fibre, it is quite clear that the want of it must be attended
with some derangement of the health; and hence the benefit derived
from the taking of vegetables, which by supplying potass, make up
for the want of this alkali in the meat.
Albumen is coagulated by heat, and is drawn out by cold water;
this fact is referred to in Note, No. 11.

No. 1. The coating of small articles, of the better sorts, excludes

the air. It is a mixture of gelatine with treacle, applied when hot, and
which when exposed to the air becomes hardened, yet elastic as
india-rubber. See Note, No. 12.
No. 2. The smoking with different sorts of fuel exerts a
considerable influence on the flavour and preservation of the articles
so treated; for example, the mutton of the Ardennes forest, Belgium,
which owes its superiority to the juniper bushes with which it is dried
and smoked. And again, kippered salmon smoked with cedar-wood,
at the request of some of our wealthy Jew families, is excellent,
though rather expensive.
No. 3. The spicing of bacon was adopted some years since, and
chiefly in Ireland, to hide the inferior quality of the meat.
No. 4. Bay salt is far preferable to common salt for curing meats
and fish, but the expense deters many persons from using it, except
in small quantities. The rock, or mineral salt of Cheshire, is equal in
all respects to the bay salt of commerce, and I have long used it with
the greatest success. Common salt leaves a bitter smatch on all food
cured with it after being long kept. See Note, No. 12.
No. 5. There is no remedy for over-salted provisions. You may,
indeed, cut them into slices and lay them in water, but this only
affects the outsides of large pieces, as hams, &c. If I had a ham that
I suspected of being over salted, I should put it in an old bag, and
bury it in my garden for a week or more, according to size.
No. 6. Rubbing large joints of meat over with a profusion of
common salt, and letting them lie, to “draw out the blood,” as it is
termed, is contrary to all reason, for away goes the chief part of the
flavour and nutriment.
No. 7. Sprats are so different in their animal construction as to be
easily detected from genuine fish—Gorgona anchovies. The
Armenian bole, often made use of to colour the sauce, has very
properly been exposed and condemned.
No. 8. Pure olive oil will preserve meat and fish, after it has been
cured, for a long time; but oils drawn from lard and other spurious
imitations, will not fail to hasten their decay. The oil should never be
heated, when used for this purpose.
No. 9. Subjecting meats to a water-bath is not to be resorted to. I
was shown a specification previous to the taking out of a patent (in
France, by a French gentleman) to cure the more expensive sorts of
fish. The first part of the process proposed, was subjecting the fish to
a water-bath, first for three hours, and, changing the water, then to
two hours further immersion in warm water. I, of course, entered my
protest against such unreasonable treatment. I am sure I could not
conjecture what became of both flavour and nutriment after so long
immersion.
No. 10. Meats to be boiled, and particularly fish, must be put into
boiling water, and after being kept up to the boiling point fifteen
minutes or so, let them only simmer until done.
No. 11. The albumen is drawn out from both meat, fish, and
vegetables by cold water; how, then, can we reconcile ourselves with
the foolish old practice of laying the heads, for instance, and other
parts, in pails of water, and leaving them for hours in that state, to
lose all their goodness. Vegetables for pickling, too, are often treated
in this way. Lying in water cannot possibly clean anything. Wash well,
and hang up to dry, is more reasonable.
No. 12. A very effective coating for small cured articles is made
thus: To four pounds of hard, compact gelatine, add as much soft or
rain-water as will just cover it, and stir it about occasionally for six
hours. When it has stood twenty-four hours, and all the water is
absorbed, submit it to the action of heat in a water-bath, and the
gelatine will be quickly dissolved. Take it off the fire as soon as the
froth is perceived to rise, and mix with it three and a half pounds of
molasses, which has previously been made thoroughly hot. Stir the
composition well together while in the water-bath over the fire, not
suffering it to boil. After it has been thus subjected to heat for half an
hour, and well stirred all the time, it should be taken off the fire and
allowed to cool a little; it is now ready for use, and to be applied to
the article of food with a soft brush. Set it in a current of air to harden
quickly. A second paying-over with the composition may be done
with advantage sometimes. For larger articles, as hams, &c. &c., the
best transparent glue may be used instead of gelatine, adding to the
composition, when a little cooled, a few drops of essence of nutmegs
or pimento. If when cold the coating is found to be not firm enough,
the proportion of gelatine or glue must be slightly increased, and
when, on the contrary, it is too brittle, the quantity of molasses may
be increased.
No. 13. For the accommodation of parties residing at a distance,
peat or bog-earth, rock salt from the Cheshire mines, charcoal, &c.
&c., may be had, ready for immediate, use, on application to Mr.
Robinson, provision curer, Runcorn, Cheshire, and on very moderate
terms.
 INDEX.
Aberdeen red herrings, 55
Anchovies, British, 59
— Gorgona, to feed, 59
— Gorgona, smoked, 63
— essence of, 129
Apparatus for drying, smoking, &c., 1
Asparagus, pickled, 155

Bacon, choice breakfast, 7

— Leicestershire spiced, 23
Barberries pickled, 154
Bath chaps, 24
Beef as hare, potted, 120
Beef’s heart, potted, 123
Beef, hung, Shropshire sirloin, 6
— Melton hunt, 8
Beef’s heart, smoked, 10
Beef, Ulverston red flank, 11
— hams, 13
— Hambro rough, 13
— Breslau, 14
— Whitehaven corned, 15
— Dutch, 25
Beetroots, pickled, 167
Bloaters, 50
Black puddings, Jersey, 90
Birmingham and Oxford tripe, 75
Boar’s head smoked, 19
Brawn, calf’s head, 77
Bucaning meats, described, 1
Bucaned beef kidneys, 80
— udder, 81
— calf’s liver, 82
— beef skirts, 83

Cabbage, red, pickled, 145

Catsup, tomato, 130
— mushroom, 171
— walnut, 170
Cauliflowers, pickled, 146
Caveach herrings, 74
Caviare brown, 70
— white, 71
Cavis of mackerel, 72
Celery, pickled, 151
— crab salad, 173
Charcoal, preservative quality, 4
Chetna, Bengal, 131
Coating composition, to make, 180
— to apply, 180
Codlins, pickled, 154
Coated turbot fins, 60
— river eels, 62
— conger eels, 68
— young pig, 114
Collared salmon, 43
 — side of venison, 109
— young pig, 112
Crabs potted, 107
Crab salad, 173
Conger eels, 69
Currants red, pickled, 151
— preserved for tarts, 135
Curry powder, excellent, 175

Dried Mutton, as in the Ardennes, 29

— breast of mutton as venison, 33
Dried Ulverston red flank of beef, 11
— Conger eels, high flavoured, 69
Digby herrings, 55

Eels, conger, smoked, 66

— collared, 68
— dried, 69
— potted, 118
Elder-flower vinegar, 173
Essences to make, 1
Essence of lobsters, 127
— shrimps, 128
— anchovies, 129

Fish, nutriment in (Dr. Davy), 36

— sauce, excellent, 131
Foots of sugar, to be preferred, 4
Fuel for smoking and drying with, 3

Geese, smoked, 79
German saveloys, 89
Gherkins, pickled, 156
Grapes, „ 152
Goose, a perpetual (beef’s heart), 34
Green West India ginger, preserved, 134

Hambro’ Pickle, for beef and pork, 31

Hams, Westphalia, 19
— eclipsed, 20
Hare, potted, 114
Haunch of mutton as venison, 26
Herrings, marinated, 103

Italian Cincerelli, 65

Jersey Black Puddings, 90

Kippered Herrings, 52
— superior, spiced, 53
— salmon, superior, 40

Lemon mangoes, pickled, 159

Lemon pickle, 160
Lemons, preserved, 139
Lobsters, essence of, 127
— pickled, 102
— potted, 106

Mackarel Kippered, 45
— (May-fish), 46
— superior pressed, 47
Maltcooms, to keep cured goods in, 4
Mangoes, pickled, 161
Marinated herrings, 103
— eels, 99
— high flavour, 100
— salmon, 92
— sprats, 104
— shrimps, 96
— salmon roes, 127
— tench and carp, 93
— trout and grayling, 97
— veal, 125
— another method, 126
Marmalade, raspberry, 144
Moor-game, potted, 115
Morello cherries, jam of, 144
Mushroom catsup, 171
— buttons, pickled, for pies and sauces, 168
Mutton, dried as in the Ardennes, 29
— breast of, collar as venison, 33
— haunch as venison, 26
— thigh of l’Diable, 27
— Welsh hams, 28

Nasturtiums, pickled, 166

Neats’ tongues, potted, 121
— pickled, 16
— high flavoured, 17
— to pickle, 30

Oversalted Meat, to rectify, 178

Pickled Vegetables,
— asparagus, 155
— barberries, 154
— beetroots, 167
— cauliflowers, 146
 — currants, red, 151
— celery, 151
— codlins, 154
— gherkins, 156
— golden pippins, 165
— grapes, 152
— mushrooms, white, 147
— mangoes (lemon), 159
— lemon pickle, 160
— mangoes (cucumber), 161
— nasturtiums, 166
— mushroom buttons, 168
— peaches and nectarines, 165
— piccalilli, 157
— parsley (green), 169
— onions, silver, 148
— walnuts, green, 163
— „ white, 164
— samphire, 146
Pickled Meats and Fish,
— herrings, 73
— smelts, 101
— lobsters, 102
Pickle for pork, 31
— superior, 32
— a preservative (excellent), 32
— the Hambro’, for beef and pork, 31
Pig, a young one collared, 112
Polony, Russian, 87
Provocative, a, 132
Portable soup, 78
— much richer, 78
Porker’s head, smoked, 23
Preservatives, 4
Potted beef’s heart, 122
— crabs, 107
— hare, 114
— eels, 118
— lobsters, 106
— Moor game, 115
— ox cheek, 84
— neat’s tongue, 121
— beef as hare, 120
— pigeons, 86
— snipes and woodcocks, 116
— shrimps, 119
— „ l’Diable, 85
— trout, 117
— venison, 124
Preserved
— apricots, 140
— barberries, 142
— cucumbers, 137
— golden pippins, 143
— greengage plums, 138
— damsons, 140
— Hambro’ grapes, 142
— lemons, 139
— Morello cherries, 141
— peaches and nectarines, 138
— tomatoes, 136

Smoked Meats,
— beef’s heart, 10
— beef hams, 13
— „ Breslau, 14
— boar’s head, 19
— calf’s head brawn, 76
— Dutch beef, 25
— geese, smoked, 78
— goose, a perpetual, 34
 — Hambro beef, 13
— hung beef, 6
— Leicestershire spiced bacon, 23
— Melton hunt beef, 9
— mutton, as in the Ardennes, 29
— neats’ tongues, high flavour,17
— Norfolk chine, 21
— porker’s head, 23
— polony, Russian, 87
— German saveloys, 89
— venison, side of, 111
— Whitehaven corned beef, 15
— Westphalia hams, 19
— „ eclipsed,20
Smoked Fish,
— eels, river, 62
— „ conger, 66
— Gorgona anchovies, 63
— herrings, bloaters, 50
— „ kippered, 51
— Mackerel, kippered, 45
— „ May-fish, 46
— „ superior, 47
— salmon, Welsh, 37
— „ Dutch, 39
— „ superior kipper, 40
— „ American, 48
— „ collared, 43
— herrings, Digby, 55
— „ Aberdeen reds,55
— speldings, 56
— sprats, 56
Smelts, pickled, 101
— potted, 105
Snipes and woodcocks, potted, 116
Sprats, marinated, 104
Shrimps, essence of, 128
Sausage spice (French), 132
Syrup for preserving fruit, to prepare, 132
Samphire, green, pickled, 146
Silver onions, pickled, 148
Syrup d’Orgeat (French), 174

Tench and Carp, marinated, 93

Tomatoes paste, 129
— catsup, 130
— „ 172
Tripe, Birmingham and Oxford, 75
Trout and grayling, marinated, 97
— potted, 117
Turbot fins, 60

Veal Marinated, 125

— „ 126
Vinegar, elder flower, 173
— tarragon, 173
— white gooseberry, 174

Walnuts, pickled, 164

— preserved, 149
— green, pickled, 163
— catsup, 170

Yorkshire Pressed Pork, 74