Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.

org/terms-privacy

Linear

Second Edition
and

Algebra
Applied
Matrix
Analysis
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

— Paul Halmos
The only way to learn mathematics is to do mathematics.
 Matrix
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Analysis
and
Applied
Linear
Algebra
Second Edition

Carl D. Meyer
North Carolina State University
Raleigh, NC

Society for Industrial and Applied Mathematics

Philadelphia
 Copyright © 2023 by the Society for Industrial and Applied Mathematics
10 9 8 7 6 5 4 3 2 1
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

All rights reserved. Printed in the United States of America. No part of this book may be reproduced, stored, or transmitted in any
manner without the written permission of the publisher. For information, write to the Society for Industrial and Applied Mathematics,
3600 Market Street, 6th Floor, Philadelphia, PA 19104-2688 USA.
No warranties, express or implied, are made by the publisher, authors, and their employers that the programs contained in this volume
are free of error. They should not be relied on as the sole basis to solve a problem whose incorrect solution could result in injury to
person or property. If the programs are employed in such a manner, it is at the user’s own risk and the publisher, authors, and their
employers disclaim all liability for such misuse.
Trademarked names may be used in this book without the inclusion of a trademark symbol. These names are used in an editorial
context only; no infringement of trademark is intended.

Publications Director Kivmars H. Bowling

Executive Editor Elizabeth Greenspan
Acquisitions Editor Elizabeth Greenspan
Developmental Editor Rose Kolassiba
Managing Editor Kelly Thomas
Production Editor David Riegelhaupt
Copy Editor Susan Fleshman
Production Manager Donna Witzleben
Production Coordinator Cally A. Shrader
Compositor Carl D. Meyer
Graphic Designer Doug Smock

Library of Congress Cataloging-in-Publication Data

Names: Meyer, C. D. (Carl Dean), author.
Title: Matrix analysis and applied linear algebra / Carl D. Meyer, North
Carolina State University, Raleigh, NC.
Description: Second edition. | Philadelphia : Society for Industrial and
Applied Mathematics, [2023] | Series: Other titles in applied
mathematics ; 188 | Includes bibliographical references and index. |
Summary: “Matrix Analysis and Applied Linear Algebra, Second Edition
circumvents the traditional definition-theorem-proof format, and
includes topics not normally found in undergraduate textbooks. Taking
readers from elementary to advanced aspects of the subject, the author
covers both theory and applications. The theoretical development is
rigorous and linear, obviating the need for circular or non-sequential
references. An abundance of examples and a rich variety of applications
will help students gain further insight into the subject. A study and
solutions guide is also available”-- Provided by publisher.
Identifiers: LCCN 2022048558 (print) | LCCN 2022048559 (ebook) | ISBN
9781611977431 (hardback) | ISBN 9781611977455 (paperback) | ISBN
9781611977448 (ebook) | ISBN 9781611977462 (ebook other)
Subjects: LCSH: Algebras, Linear. | Matrices. | Algebras, Linear. |
Matrices. | AMS: Linear and multilinear algebra; matrix theory --
Instructional exposition (textbooks, tutorial papers, etc.). | Linear
and multilinear algebra; matrix theory -- Basic linear algebra -- Norms
of matrices, numerical range, applications of functional analysis to
matrix theory.
Classification: LCC QA188 .M495 2023 (print) | LCC QA188 (ebook) | DDC
512/.5--dc23/eng20230112
LC record available at https://lccn.loc.gov/2022048558
LC ebook record available at https://lccn.loc.gov/2022048559

is a registered trademark.
 Contents
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Preface . . . . . . . . . . . . . . . . . . . . . . . ix

Photo Credits
. . . . . . . . . . . . . . . . . . . . . . xiii

1. The Language of Linear Algebra . . . . . . . 1

1.1 Introduction . . . . . . . . . . . . . . . . . . 1
1.2 The Language . . . . . . . . . . . . . . . . . 2
1.3 Elementary Geometry . . . . . . . . . . . . . . 15
1.4 Euclidean Norm and Standard Inner Product . . . . 22
1.5 Non-Euclidean Norms and Convergence . . . . . . 32
1.6 Orthogonality . . . . . . . . . . . . . . . . . 39
1.7 Linearity and Matrix Multiplication . . . . . . . . 52
1.8 Elementary Properties of Matrix Multiplication . . . 67
1.9 Matrix Inner Product and Norms . . . . . . . . . 82
1.10 Unitary and Orthogonal Matrices . . . . . . . . . 92
1.11 Short History of Matrix Theory . . . . . . . . 115

2. Systems, Elimination, and Echelon Forms . . . 119

2.1 Gaussian Elimination . . . . . . . . . . . . . 119
2.2 Elementary Matrices and Echelon Forms . . . . . 136
2.3 Nonsingular Matrices and Inverses . . . . . . . 159
2.4 Rank of a Matrix . . . . . . . . . . . . . . . 180
2.5 General Linear Systems . . . . . . . . . . . . 201
2.6 Two Typical Applications . . . . . . . . . . . 220
2.7 The Kronecker Product . . . . . . . . . . . . 227
2.8 Making Gaussian Elimination Work . . . . . . . 233
2.9 Ill-Conditioned Systems . . . . . . . . . . . . 249
2.10 Triangular Factorizations . . . . . . . . . . . 259
2.11 Short History of Triangular Factorizations . . . . 283

3. Eigensystem Basics . . . . . . . . . . . . 287

3.1 Introduction to Eigensystems . . . . . . . . . . 287
3.2 Similarity and Diagonalization . . . . . . . . . 304
3.3 Functions of Diagonalizable Matrices . . . . . . 323
3.4 Normal Matrices . . . . . . . . . . . . . . . 339
 vi Contents

3.5 Singular Value Decomposition (or SVD) . . . . . 358

Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

3.6 Positive Definite Matrices and Quadratic Forms . 382

4. Vector Spaces . . . . . . . . . . . . . . . 409

4.1 Spaces and Subspaces . . . . . . . . . . . . . 409
4.2 The Fundamental Subspaces . . . . . . . . . . 428
4.3 Orthogonal Complements and Projections . . . . 456
4.4 Least Squares . . . . . . . . . . . . . . . . 481
4.5 Coordinates . . . . . . . . . . . . . . . . . 505
4.6 Change of Bases . . . . . . . . . . . . . . . 528
4.7 Complementary Subspaces . . . . . . . . . . . 544
4.8 Jordan Form for Nilpotent Matrices . . . . . . . 571
4.9 Jordan Form for General Matrices . . . . . . . 584
4.10 Functions of Nondiagonalizable Matrices . . . . . 595
4.11 Difference Equations, Limits, and Summability . . 616

5. Inner Product Spaces and Fourier Expansions . 645

5.1 Inner Product Spaces . . . . . . . . . . . . . 645
5.2 Gram–Schmidt Process . . . . . . . . . . . . 658
5.3 QR Factorization . . . . . . . . . . . . . . . 666
5.4 Fourier Expansions . . . . . . . . . . . . . . 687
5.5 Discrete Fourier Transform . . . . . . . . . . . 716

6. Eigenvalue Continuity and Computations . . . 757

6.1 Eigenvalue Continuity . . . . . . . . . . . . . 757
6.2 Eigenvalue Localization . . . . . . . . . . . . 765
6.3 Power Methods . . . . . . . . . . . . . . . . 772
6.4 QR Iterations . . . . . . . . . . . . . . . . 779

7. Perron–Frobenius and Nonnegative Matrices . 797

7.1 Introduction . . . . . . . . . . . . . . . . . 797
7.2 Nonnegative Matrices . . . . . . . . . . . . . 799
7.3 Positive Matrices . . . . . . . . . . . . . . . 804
7.4 Perron–Frobenius Theorem . . . . . . . . . . 808
7.5 Primitive Matrices . . . . . . . . . . . . . . 820
7.6 Periodic Matrices . . . . . . . . . . . . . . . 825
7.7 Differentiation of the Perron Root and Vector . . 840
 Contents vii

7.8 Perron Complementation . . . . . . . . . . . 851

Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

7.9 Markov Chains . . . . . . . . . . . . . . . . 859

7.10 Stochastic Complementation . . . . . . . . . . 882
7.11 Simon–Ando Theory for Weakly Coupled Systems 890

8. Matrix Analysis via Resolvent Calculus . . . . 899

8.1 A Short Review . . . . . . . . . . . . . . . 899
8.2 Eigenvalue Continuity . . . . . . . . . . . . . 911
8.3 Eigenvalue Differentiation . . . . . . . . . . . 922

9. Appendix: An Overview of Determinants . . . 939

9.1 Introduction . . . . . . . . . . . . . . . . . 939
9.2 Fundamentals . . . . . . . . . . . . . . . . 940
9.3 Additional Properties of Determinants . . . . . . 956

Index . . . . . . . . . . . . . . . . . . . . . . 971
 viii von Neumann Essay
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

As a mathematical discipline travels far from its empirical source, or still

more, if it is a second and third generation only indirectly inspired by ideas
coming from “reality” it is beset with very grave dangers. It becomes more and
more purely aestheticizing, more and more purely l’art pour l’art. This need
not be bad, if the field is surrounded by correlated subjects, which still have
closer empirical connections, or if the discipline is under the influence of men
with an exceptionally well-developed taste. But there is a grave danger that
the subject will develop along the line of least resistance, that the stream, so
far from its source, will separate into a multitude of insignificant branches, and
that the discipline will become a disorganized mass of details and complexities.
In other words, at a great distance from its empirical source, or after much
“abstract” inbreeding, a mathematical subject is in danger of degeneration. At
the inception the style is usually classical; when it shows signs of becoming
baroque, then the danger signal is up. It would be easy to give examples, to
trace specific evolutions into the baroque and the very high baroque, but this,
again, would be too technical. In any event, whenever this stage is reached, the
only remedy seems to me to be the rejuvenating return to the source: the re-
injection of more or less directly empirical ideas. I am convinced that this was a
necessary condition to conserve the freshness and the vitality of the subject and
that this will remain equally true in the future.

— John von Neumann

From his essay The Mathematician
Works of the Mind, University of Chicago Press, 1947.
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Preface: Second Edition

This second edition of Matrix Analysis and Applied Linear Algebra differs
substantially from the first edition in that this edition has been completely rewrit-
ten to include reformulations, extensions, and pedagogical enhancements. The
goal in preparing this edition was to create an easily readable and flexible text-
book that is adaptable for a single semester course or a more complete two-
semester course. The following features are some of the characteristics of this
edition.

• Scaffolding. Reacting to criticism concerning the lack of motivation in his

writings, Gauss remarked that architects of great cathedrals do not obscure
the beauty of their work by leaving the scaffolding in place after the con-
struction has been completed. His philosophy epitomized the formal presen-
tation and teaching of mathematics throughout the nineteenth and twentieth
centuries, and it is still commonly found in mid-to-upper-level mathematics
textbooks. The inherent efficiency and natural beauty of mathematics are
compromised by straying too far from Gauss’s viewpoint. But, as with most
things in life, appreciation is generally preceded by some understanding sea-
soned with a bit of maturity, and in mathematics this comes from seeing
some of the scaffolding. This book’s front cover portrays this philosophy.

• Purpose. The purpose of this text is to present the contemporary theory and
applications of linear algebra to university students studying mathematics,
engineering, or applied science at the postcalculus level. Because linear al-
gebra is usually encountered between basic problem solving courses such as
calculus or differential equations and more advanced courses that require
students to cope with mathematical rigors, the challenge in teaching applied
linear algebra is to expose some of the scaffolding while conditioning students
to appreciate the utility and beauty of the subject. Effectively meeting this
challenge and bridging the inherent gaps between basic and more advanced
mathematics are primary goals of this book.

• Linear Development. An effort was made to ensure that the development of

the theory is completely linear. With the exception of a few references to
determinants in the appendix, material at any point is not dependent on
subsequent developments. This allows instructors to move linearly through
their course without the need to substantially refer to later developments.
 x Preface: Second Edition
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

• Graduated Sophistication. In addition to being linear, the development is

also graduated by level of sophistication. The text starts from traditional
first principles in the earlier chapters and progresses through a deeper un-
derstanding of both theory and applications in the later chapters. This allows
for a traditional single-term course based on roughly half of the text without
having to refer to more advanced topics, and it provides for a complete two-
term course covering the range of theory and applications generally reserved
for discussions beyond the basics.

• Carefully Constructed Exercises. There are ample exercises at all levels of

difficulty throughout the text. Each section is complemented by problems
that range from straightforwardly easy to moderately challenging to being
difficult for all but the best students. Many exercises are carefully constructed
so as to condition students for topics that soon follow.

• Supplemental Solutions and Study Guide. An accompanying Study and So-

lutions Guide containing complete solutions and discussions of each exercise
is available as a standalone volume. The supplement is designed to help stu-
dents understand the point of each exercise and to guide their development
as they move through the subject. It is possible to use the text in conjunction
with the Study and Solutions Guide for self-directed study.

• Flexible Rigor. The development is rigorous at all levels, but accompanying

narratives allow formality to be tempered or bypassed. Theorems and proofs
are clearly highlighted, but the text is designed so that the degree of rigor
that an instructor chooses to employ is flexible enough to accommodate a
strict treatment or a casual overview. Along with each major theorem there
are associated discussions and illustrative examples designed to convince stu-
dents of the validity of the theorem without a deep dive into the proof.

• Applications. As in the first edition, there is an emphasis on applications, but

in this second edition both the depth and breadth of the applications have
been expanded. The material on numerical aspects as well as the development
of the theory and applications of the discrete Fourier transform are particular
examples.

• History. This second edition also contains enhancements of the historical

remarks over those in the first edition. There is a sharper focus on the per-
sonalities of the individuals who created and contributed to the subject’s
development.

• Acknowledgments. This edition evolved from the first edition due to the help
of countless people from around the world who suggested improvements to
nearly all facets of the book, and I am deeply indebted to them. Special
 Preface: Second Edition xi
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

thanks go to the hundreds of students at NC State University who took both

undergraduate and graduate courses in the last twenty-two years using the
first edition of this text. Experiences derived from teaching these students
were invaluable in preparing this second edition, and my interactions with
them are responsible for many improvements. Finally, I wish to acknowledge
Elizabeth Greenspan, Executive Editor at SIAM, for her encouragement and
patience over the years since the first edition appeared. This edition would
not have been possible if it were not for her belief in me and my book.

• Dedication. I dedicate this book to Bethany, my friend, partner, and wife, and
to our children Martin D. Meyer and Holly Meyer Kenney, and to our grand-
children, Margaret Elizabeth Meyer Harding, Allison Catherine Kenney, and
Ryan Patrick Kenney.

Carl D. Meyer
March 27, 2023
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

xii

— Leopold Kronecker
God made the integers; all else is the work of man.
 Photo Credits: Second Edition
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Page 167 (Schur): Archives of the Mathematisches Forschungsinstitut Oberwolfach (CC BY-SA 2.0
DE). https://creativecommons.org/licenses/by-sa/2.0/de/deed.en

Page 167 (Haynsworth): Courtesy of the Department of Mathematics and Statistics, College of
Sciences and Mathematics, Auburn University.

Page 284 (Doolittle): Courtesy of Antiochiana, Antioch College.

Page 285 (Crout): Courtesy of the MIT Museum.

Page 286 (Goldstine): Courtesy of American Philosophical Society.

Page 424 (Courant): Used with permission of The National Academies Press, from Biographical
Memoirs: Volume 82, Page 78, 2003; permission conveyed through Copyright Clearance Center, Inc.

Page 658 (Schmidt): Archives of the Mathematisches Forschungsinstitut Oberwolfach (CC BY-SA
2.0 DE). https://creativecommons.org/licenses/by-sa/2.0/de/deed.en

Page 765 (Galois duel): Bowen, Jonathan & Giannini, Tula. (2015). Galois Connections: Mathematics,
Art, and Archives. DOI: 10.14236/ewic/eva2015.18.

Page 766 (Gerschgorin): Courtesy of the University of St Andrews.

Page 766 (Taussky): Archives of the Mathematisches Forschungsinstitut Oberwolfach (CC BY-SA
2.0 DE). https://creativecommons.org/licenses/by-sa/2.0/de/deed.en

Page 766 (Brauer): Reprinted with permission from North Carolina Academy of Science. “Alfred
T. Brauer, Mathematician, and Developer of Libraries,” by Richard D. Charmichael, Journal of the
Elisha Mitchell Scientific Society, Vol. 102, No.3, 1986, pp. 88-106.

Page 772 (von Mises): Courtesy of the University of St Andrews.

Page 774 (Wielandt): Archives of the Mathematisches Forschungsinstitut Oberwolfach (CC BY-SA
2.0 DE). https://creativecommons.org/licenses/by-sa/2.0/de/deed.en

Page 779 (Rutishauer): Courtesy of the University of St Andrews.

Page 779 (Francis): Reprinted from Gene Golub, Frank Uhlig, “The QR algorithm: 50 years later its
genesis by John Francis and Vera Kublanovskaya and subsequent developments,” IMA Journal of
Numerical Analysis, 2009, Volume 29, Issue 3, Page 468, by permission of Oxford University Press.
 xiv Photo Credits: Second Edition
Downloaded 02/21/24 to 2.220.83.7 . Redistribution subject to SIAM license or copyright; see https://epubs.siam.org/terms-privacy

Page 779 (Kublanovskaya): Reprinted from Gene Golub, Frank Uhlig, “The QR algorithm: 50 years
later its genesis by John Francis and Vera Kublanovskaya and subsequent developments,” IMA Jour-
nal of Numerical Analysis, 2009, Volume 29, Issue 3, Page 474, by permission of Oxford University
Press.

Page 791 (Golub): Courtesy of Stanford News Service.

Page 791 (Golub license plate): Gene Golub’s license plate, photographed by Professor P. M. Kroo-
nenberg of Leiden University.
Page 791 (Kahan): Photographed by George M. Bergman, Berkeley. Archives of the Mathematisches
Forschungsinstitut Oberwolfach.
Page 798 (Perron): Photographed by Karl-August Keil, Augsburg. Archives of the Mathematisches
Forschungsinstitut Oberwolfach.

Page 798 (Frobenius): Courtesy of the University of St Andrews.

Page 834 (Leslie): Reprinted by permission from Springer Nature: Springer The Leslie matrix (1945)
by Nicolas Bacar Copyright 2011.

Page 849 (Neumann): Courtesy of Maxine Lerman.

Page 867 (Brin and Page): Photographed by Joi Ito (CC BY 2.0).
https://creativecommons.org/licenses/by/2.0/legalcode

Page 890 (Ando): Photograph by Stefano Siviero.

Page 899 (Kato): Photographed by George M. Bergman, Berkeley. Archives of the Mathematisches
Forschungsinstitut Oberwolfach.