mt2118 ch1-3

Undergraduate study in
Economics, Management,
Finance and the Social Sciences
A d va n ce d li n e a r a lg e b ra
M . Anthony, M . Harvey
M T2118, 2790118
2 0 1 1
This is an extract from a subject guide for an undergraduate course offered as part of the
University of London International Programmes in Economics, M anagement, Finance and
the Social Sciences. M aterials for these programmes are developed by academics at the
London School of Economics and Political Science ( LSE) .
For more information, see: www.londoninternational.ac.uk
This guide was prepared for the University of London International Programmes by:
Professor M. Anthony, BSc, MA, PhD and Dr M. Harvey, BSc, MSc, PhD, Department of
Mathematics, The London School of Economics and Political Science.
This is one of a series of subject guides published by the University. We regret that due to
pressure of work the authors are unable to enter into any correspondence relating to, or aris-
ing from, the guide. If you have any comments on this subject guide, favourable or unfavour-
able, please use the form at the back of this guide.
University of London International Programmes
Publications Office
Stewart House
32 Russell Square
London WC1B 5DN
United Kingdom
Website: www.londoninternational.ac.uk
Published by: University of London
University of London 2006
Reprinted with minor revisions 2011
The University of London asserts copyright over all material in this subject guide except where
otherwise indicated. All rights reserved. No part of this work may be reproduced in any form,
or by any means, without permission in writing from the publisher.
We make every effort to contact copyright holders. If you think we have inadvertently used
your copyright material, please let us know.
Contents
i
Contents
Preface ..........................................................................................................v
Introduction ................................................................................................. 1
1.1 This course ........................................................................................................ 1
1.2 Reading............................................................................................................. 2
1.3 Online study resources ....................................................................................... 3
1.4 Using the guide ................................................................................................. 4
1.5 Examination ...................................................................................................... 4
1.6 The use of calculators ........................................................................................ 5
2 Matrices, vectors, and systems of linear equations ................................. 7
2.1 Introduction ...................................................................................................... 7
2.2 Vectors and matrices .......................................................................................... 7
2.2.1 Row operations. ...................................................................................... 10
2.2.2 Determinants. ......................................................................................... 11
2.2.3 Invertible matrices .................................................................................. 13
2.3 Rank, range, null space, and linear equations ................................................... 15
2.3.1 The rank of a matrix ................................................................................ 15
2.3.2 Rank and systems of linear equations ...................................................... 16
2.3.3 General solution of a linear system in vector notation .............................. 19
2.3.4 Null space .............................................................................................. 20
2.3.5 Range .................................................................................................... 21
2.4 Learning outcomes ......................................................................................... .22
2.5 Sample examination questions ......................................................................... 23
2.6 Answers to or comments on selected activities ................................................. 24
2.7 Sketch answers to or comments on selected questions ..................................... 26
3 Vector spaces, basis, dimension .............................................................. 29
3.1 Introduction .................................................................................................... 29
3.2 Vector spaces .................................................................................................. 29
3.2.1 Defnition of a vector space...................................................................... 29
3.2.2 Examples ................................................................................................ 31
3.3 Subspaces........................................................................................................ 32
3.3.1 An alternative characterisation of a subspace .......................................... 35
3.4 Subspaces connected with matrices ................................................................. 35
3.4.1 Null space .............................................................................................. 35
3.4.2 Range .................................................................................................... 36
3.5 Linear span...................................................................................................... 36
3.5.1 Lines and Planes in
3
............................................................................ 37
3.5.2 Row space and column space ................................................................. 38
3.6 Linear independence. ....................................................................................... 38
3.7 Testing linear independence in
n
.................................................................... 40
3.8 Bases and dimension ....................................................................................... 43
3.8.1 Coordinates ............................................................................................ 45
3.8.2 Dimensions ............................................................................................. 46
3.8.3 Dimension and bases of subspaces ......................................................... 47
3.9 Finding a basis for a linear span in
n
.............................................................. 48
3.10 Basis and dimension of range and null space ................................................. 49
3.11 Learning outcomes ........................................................................................ 52
3.12 Sample examination questions ....................................................................... 52
3.13 Answers to or comments on selected activities ............................................... 54
3.14 Sketch answers to or comments on selected questions ................................... 58
118 Advanced linear algebra
ii
4 Linear transformations, change of basis ................................................ 65
4.1 Introduction .................................................................................................... 65
4.2 Linear transformations ..................................................................................... 65
4.3 Examples ......................................................................................................... 66
4.4 Linear transformations and matrices ................................................................. 67
4.4.1 Rotation in
2
........................................................................................ 68
4.4.2 Identity and zero linear transformations .................................................. 70
4.4.3 Composition and combinations of linear transformations ......................... 70
4.4.4 Inverse linear transformations ................................................................. 70
4.4.5 Linear transformations from V to W......................................................... 71
4.5 Range and null space ...................................................................................... 72
4.6 Rank and nullity .............................................................................................. 73
4.7 Coordinate change .......................................................................................... 75
4.8 Change of basis and similarity ......................................................................... 77
4.9 Learning outcomes .......................................................................................... 79
4.11 Sketch answers to or comments on selected activities ..................................... 80
5 Diagonalisation ...................................................................................... 85
5.1 Introduction .................................................................................................... 85
5.2 Eigenvalues and eigenvectors .......................................................................... 85
5.2.1 Defnitions ............................................................................................... 85
5.2.2 Finding eigenvalues and eigenvectors...................................................... 86
5.2.3 Eigenspaces ............................................................................................ 89
5.3 Diagonalisation of a square matrix ................................................................... 90
5.3.1 Diagonalisation, when can it fail? ........................................................... 93
5.4 Learning outcomes .......................................................................................... 97
5.5 Sample examination questions ......................................................................... 97
5.6 Sketch answers to or comments on selected activities ....................................... 98
5.7 Sketch answers to or comments on selected questions ..................................... 99
6 Inner products, orthogonality, orthogonal diagonalisation ................. 101
6.1 Introduction .................................................................................................. 101
6.2 The inner product of two real n-vectors .......................................................... 101
6.2.1 Geometric Interpretation in
2
and
3
.................................................. 102
6.3 Inner products more generally ........................................................................ 105
6.3.1 Norms in a vector space ........................................................................ 106
6.3.2 The Cauchy-Schwarz inequality ............................................................. 107
6.3.3 Generalised geometry ........................................................................... 108
6.3.4 Orthogonal vectors ............................................................................... 109
6.3.5 Orthogonality and linear independence ................................................. 109
6.4 Orthogonal matrices and orthonormal sets ..................................................... 110
6.5 Gram-Schmidt orthonormalisation process ..................................................... 111
6.6 Orthogonal diagonalisation of symmetric matrices .......................................... 113
6.10 Sketch answers to or comments on selected questions ................................. 122
7 Applications of diagonalisation ............................................................ 127
7.1 Introduction .................................................................................................. 127
7.2 Powers of matrices ........................................................................................ 128
7.3 Systems of difference equations ..................................................................... 129
7.3.1 Difference equations: revision ............................................................... 129
7.3.2 Systems of difference equations ............................................................ 130
7.3.3 Solving by change of variable ................................................................ 130
Contents
iii
7.3.4 Solving using matrix powers ................................................................. 132
7.3.5 Markov Chains ..................................................................................... 134
7.4 Linear systems of differential equations .......................................................... 138
7.5 Quadratic forms Applications of orthogonal diagonalisation ........................ 140
7.5.1 Information on quadratic forms ............................................................. 141
7.5.2 Quadratic forms in
2
conic sections .................................................. 145
8 Direct sums and projections ................................................................. 155
8.1 Introduction .................................................................................................. 155
8.2 The direct sum of two subspaces .................................................................... 155
8.2.1 The sum of two subspaces .................................................................... 155
8.2.2 Direct sums .......................................................................................... 156
8.3 Orthogonal complements ............................................................................... 158
8.3.1 The orthogonal complement of a subspace ............................................ 158
8.3.2 Orthogonal complements of null spaces and ranges .............................. 160
8.4 Projections .................................................................................................... 162
8.4.1 The definition of a projection ................................................................. 162
8.4.2 An example .......................................................................................... 163
8.4.3 Orthogonal projections ......................................................................... 163
8.5 Characterising projections and orthogonal projections .................................... 164
8.5.1 Projections are idempotents .................................................................. 164
8.6 Orthogonal projection on to the range of a matrix .......................................... 166
8.7 Minimising the distance to a subspace ........................................................... 167
8.8 Fitting functions to data: least squares approximation .................................... 168
8.8.1 A linear algebra view. ............................................................................ 169
8.8.2 An example .......................................................................................... 169
8.10 Sample examination questions ..................................................................... 172
8.11 Sketch answers to or comments on selected activities ................................... 172
9 Complex matrices, vector spaces ......................................................... 175
9.1 Introduction .................................................................................................. 175
9.2 Complex numbers .......................................................................................... 175
9.2.1 Complex numbers ................................................................................. 176
9.2.2 Roots of polynomials ............................................................................ 178
9.2.3 The complex plane ................................................................................ 179
9.2.4 Polar form of z ...................................................................................... 179
9.2.5 Exponential form of z ............................................................................ 181
9.2.6 Complex solutions of differential and difference equations .................... 183
9.3 Complex vector spaces .................................................................................. 187
9.4 Complex matrices .......................................................................................... 188
9.5 Complex inner product spaces ....................................................................... 190
9.5.1 The inner producton
n
. ........................................................................ 190
9.5.2 Complex inner product in general.......................................................... 191
9.5.3 Orthogonal vectors ............................................................................... 193
9.6 The adjoint. Hermitian and unitary matrices .................................................... 195
9.6.1 The adjoint ................................................................................................. 195
9.6.2 Hermitian matrices ................................................................................ 197
9.6.3 Unitary matrices ................................................................................... 198
9.7 Unitary diagonalisation. Normal matrices ....................................................... 200
9.8 Spectral decomposition .................................................................................. 203
iv
9.10 Sample examination questions ..................................................................... 208
9.11 Sketch answers to or comments on selected activities ................................... 210
10 Sample examination paper ................................................................. 219
10.1 The format ................................................................................................... 219
10.2 A Sample examination paper ....................................................................... 219
v
Preface
This subject guide is not a course text. It sets out the logical sequence in
which to study the topics in the course. Where coverage in the main texts
is weak, it provides some additional background material. Further reading
is essential.
We are grateful to Keith Martin and James Ward for their careful reading
of a draft of the guide, and for their many helpful comments.
Notes
vi
Chapter 1: Introduction
1
Chapt er 1
Int roduct ion
In this very brief introduction, we aim to give you an idea of the nature
of this course and to advise on how best to approach it. We give general
information about the contents and use of this subject guide, and on
recommended reading and how to use the textbooks.
1.1 This course
Relat ionship t o previous mat hemat ics courses
If you are taking this half course as part of a BSc degree, courses which
must be passed before this half course may be attempted are 05A
Mathematics 1 and 05B Mathematics 2 or 174 Calculus. Any
references in the text to these courses for prerequisite material will apply
equally to whatever prerequisite you have taken.
In 05A Mathematics 1 and 05B Mathematics 2 you will have learned
about matrices, systems of equations, eigenvalues and eigenvectors,
diagonalisation of 2 2 matrices and solving systems of linear equations.
In 118 Advanced linear algebra we explore these topics more
deeply: we will see new techniques and methods, and we will explore
the theoretical ideas behind these. So, for this course, you will not simply
have to solve problems: you will have to be able to reason more abstractly,
and be able to prove things. In this course, we need to work with precise
defnitions, and you will need to know these. Not only will you need to
know these, but you will have to understand them, and be able ( through
the use of them) to demonstrate that you understand them. Simply learning
the defnitions without understanding what they mean is not going to be
adequate. I hope that these words of warning dont discourage you, but I
think its important to make it clear that this is a course at a higher level
than those prerequisite courses.
Please note: 118 Advanced linear algebra will not be offered under
the New Regulations. It will be replaced by 175 Further linear algebra.
This new course will be available from September 2012.
Aims
This half course is designed to:
enable you to acquire further skills in the techniques of linear algebra, as
well as understanding of the principles underlying the subject
prepare you for further courses in mathematics and/or related
disciplines (e.g. economics, actuarial science).
Learning out comes
At the end of this course, and having completed the Essential reading and
activities, you should have:
used the concepts, terminology, methods and conventions covered in this
course to solve the mathematical problems in this subject
the ability to demonstrate an understanding of the underlying principles
of the subject
2
the ability to solve unseen mathematical problems involving an
understanding of the concepts and applications of these methods.
Topics covered
Descriptions of topics to be covered appear in the relevant chapters.
However, it is useful to give a brief overview at this stage.
Diagonalisation of 2 by 2 matrices was covered in 05B Mathematics
2. Here, the techniques are extended to larger matrices, but more
importantly the theoretical underpinning is given.This involves the
important ideas of vector spaces, linear independence, basis, dimension
and linear transformations. Wethen explore eigen values, diagonalisation,
and the applications of these techniques to solving systems of difference
and differential equations (extending methods from 05B Mathematics
2),and tofinding powers of matrices.Then,we study inner products,
orthogonality, and quadratic forms. Next, we turn our attention to the
ideas of direct sums, projections and least squares, topics that have useful
applications in many areass uch as statistics and econometrics. Finally,
we look at complex matrices andc omplex vector spaces, extending and
generalising the theory developed in the previous chapters.
Throughout, the emphasis is on the theory as much as on the methods.
That is to say, the aim in this course is not only to provide you with useful
technique sand methods of linear algebra, but to enable you to understand
why these techniques (and those you encountered in your earlier
mathematical study) work.
Not all chapters of the guide are the same length. It should not be thought
that you should spend the same amount of time on each chapter. We will
not try to specify how much relative time should be spent on each: that will
vary from person to person and we do not want to be prescriptive.
As a very rough guide (bearing in mind that this must vary from individual
to individual), we would suggest that the percentages of time spent on each
chapter are something along the lines suggested in the following table.
(This should not be taken as any indication about the composition of the
examination.)
Chapter %
2 10
3 17
4 10
5 10
6 10
7 13
8 13
9 17
1.2 Reading
There are many books that would be useful for this course, and the course
does not follow any particular one.
Most topics in this course are covered in great detail by a large number of
books. For that reason, we have resisted the temptation to specify Essential
reading in each chapter of the guide. What is true, however, is that
textbook reading is essential. Textbooks will provide more in-depth
explanations than you will find in this guide (which is explicitly not meant
3
to be a textbook), and they will also provide many more examples to
study, and many more exercises to work through.
The following books are the ones we have referred to in this guide:
Anton, H. Elementary Linear Algebra, (John Wiley, 2005)
[ISBN 9780471449037].
1
Lay, David C. Linear Algebra and its Applications, (Pearson Education, 2011)
fourth edition [ISBN 9780321385178].
Ostaszewski, A. Advanced Mathematical Methods. (Cambridge: Cambridge
University Press, 1991) [ISBN 9780521289641].
Simon, C.P. and Blume, L. Mathematics for Economists, (W.W Norton and
Company, 1994) [ISBN 9780393957334].
(for revision of some basic topics) Anthony, M. and Biggs, N. Mathematics for
Economics and Finance: Methods and Modelling. (Cambridge: Cambridge
University Press, 1996) [ISBN 9780521551137].
You should be able to find relevant reading on a topic by finding it in the
index. In several places we direct you to one of the above textbooks for
a proof of as statement or theorem, as corroboration of the result or to
satisfy your desire to know why the statement or theorem is true. Such
proofs will not be examined. As the editions of the textbooks vary, we have
not given explicit section references in the margin.
Detailed reading references in this subject guide refer to the editions of the
set textbooks listed above. New editions of one or more of these textbooks
may have been published by the time you study this course. You can use
a more recent edition of any of the books; use the detailed chapter and
section headings and the index to identify relevant readings. Also check
the virtual learning environment (VLE) regularly for updated guidance on
readings.
1.3 Online study resources
In addition to the subject guide and the reading, it is crucial that you take
advantage of the study resources that are available online for this course,
including the VLE and the Online Library.
You can access the VLE, the Online Library and your University of London
email account via the Student Portal at:
http://my.londoninternational.ac.uk
You should have received your login details for the Student Portal with
your official offer, which was emailed to the address that you gave
on your application form. You have probably already logged in to the
Student Portal in order to register! As soon as you registered, you will
automatically have been granted access to the VLE, Online Library and
your fully functional University of London email account.
If you forget your login details at any point, please email uolia.support@
london.ac.uk quoting your student number.
The VLE
The VLE, which complements this subject guide, has been designed to
enhance your learning experience, providing additional support and a
sense of community. It forms an important part of your study experience
with the University of London and you should access it regularly.
The VLE provides a range of resources for EMFSS courses:
Self-testing activities: Doing these allows you to test your own
understanding of subject material.
1
There are many
editions and variants of
this book, such as the
`Applications version.
Any one is equally
useful, you will not need
more than one of them.
You can find the relevant
sections in any edition
by using the index.
4
Electronic study materials: The printed materials that you receive from
the University of London are available to download, including updated
reading lists and references.
Past examination papers and Examiners commentaries: These provide
advice on how each examination question might best be answered.
A student discussion forum: This is an open space for you to discuss
interests and experiences, seek support from your peers, work
collaboratively to solve problems and discuss subject material.
Videos: There are recorded academic introductions to the subject,
interviews and debates and, for some courses, audio-visual tutorials
and conclusions.
Recorded lectures: For some courses, where appropriate, the sessions
from previous years Study Weekends have been recorded and made
available.
Study skills: Expert advice on preparing for examinations and
developing your digital literacy skills.
Feedback forms.
Some of these resources are available for certain courses only, but we
are expanding our provision all the time and you should check the VLE
regularly for updates.
Making use of the Online Library
The Online Library contains a huge array of journal articles and other
resources to help you read widely and extensively.
To access the majority of resources via the Online Library you will either
need to use your University of London Student Portal login details, or you
will be required to register and use an Athens login:
http://tinyurl.com/ollathens
The easiest way to locate relevant content and journal articles in the Online
Library is to use the Summon search engine.
If you are having trouble finding an article listed in a reading list, try
removing any punctuation from the title, such as single quotation marks,
question marks and colons.
For further advice, please see the online help pages:
www.external.shl.lon.ac.uk/summon/about.php
1.4 Using the guide
We have already mentioned that this guide is not a textbook. It is
important that you read textbooks in conjunction with the guide and
that you try problems from them. The Learning activities, and the sample
questions at the end of the chapters, in this guide are a very useful
resource. You should try them all once you think you have mastered a
particular chapter. Do really try them: dont just simply read the solutions
where provided. Make a serious attempt before consulting the solutions.
Note that the solutions are often just sketch solutions, to indicate to
you how to answer the questions, but in the examination, you must
show all your calculations. It is vital that you develop and enhance
your problem-solving skills and the only way to do this is to try lots of
examples.
Finally, we often use the symbol to denote the end of a proof, where we
have finished explaining why a particular result is true. This is just to make
it clear where the proof ends and the following text begins.
5
1.5 Examination
Important: Please note that subject guides may be used for several years.
Because of this we strongly advise you to always check both the current
Regulations, for relevant information about the examination, and the VLE
where you should be advised of any forthcoming changes. You should also
carefully check the rubric/instructions on the paper you actually sit and
follow those instructions.
A Sample examination paper is given as an appendix to this guide. There
are no optional topics in this course: you should do them all.
Please do not imagine for a moment that the questions in a real
examination will necessarily be very similar to these sample questions.
An examination is designed (by definition) to test you. You will get
examination questions that are unlike any questions in this guide. The
whole point of examining is to see whether you can apply knowledge
in both familiar and unfamiliar settings. The Examiners (nice people
though they are) have an obligation to surprise you! For this reason, it
is important that you try as many examples as possible: from the guide
and from the textbooks. This is not so that you can cover any possible
type of question the Examiners can think of! Its so that you get used to
confronting unfamiliar questions, grappling with them, and finally coming
up with the solution.
Do not panic if you cannot completely solve an examination question.
There are many marks to be awarded for using the correct approach or
method.
Remember, it is important to check the VLE for:
up-to-date information on examination and assessment arrangements
for this course
where available, past examination papers and Examiners commentaries
for this course which give advice on how each question might best be
answered.
1.6 The use of calculators
You will not be permitted to use calculators of any type in the
examination. This is not something that you should panic about: the
examiners are interested in assessing that you understand the key
concepts, ideas, methods and techniques, and will set questions which do
not require the use of a calculator.
Notes
6
Chapter 2
Matrices, vectors, and
systems of linear
equations
Contents
2.1 Introduction . 7
2.2 Vectors and
matrices . . . 7
2.3 Rank, range,
null space,
and linear
equations . . . 15
2.4 Learning out-
comes . . . . 22
2.5 Sample exam-
ination questions 23
2.6 Answers to or
comments on
selected activ-
ities . . . . . 24
2.7 Sketch an-
swers to or
comments
on selected
questions . . . 26
Reading
There is no specic essential reading for this chapter. It is essential
that you do some reading, but the topics discussed in this chapter are
adequately covered in so many texts on linear algebra that it would be
articial and unnecessarily limiting to specify precise passages from
precise texts. The list below gives examples of relevant reading. (For
full publication details, see Chapter 1.)
Anton, H. Elementary Linear Algebra. Chapter 1 and Chapter 2.
Lay, David C. Linear Algebra and its Applications. Chapter 1,
Chapter 2 and Chapter 3.
Simon, C.P. and Blume, L. Mathematics for Economists.
Chapter 7, sections 7.17.4; Chapter 9, sections 9.19.2.
Anthony, M. and Biggs, N. Mathematics for Economics and
Finance. Chapters 1518, 20 (for revision).
2.1 Introduction
In this chapter we rst very briey revise some basics about vectors and
matrices, which will be familiar from Mathematics 2. We then explore
some new topics, in particular the rank of a matrix, and the null space
and range of a matrix.
2.2 Vectors and matrices
An n-vector v is a list of n numbers, written either as a row-vector
(v
1
, v
2
, . . . , v
n
),
7
05B Mathematics 2. We then explore
or a column-vector
_
_
_
_
v
1
v
2
.
.
.
v
n
_
_
_
_
.
The numbers v
1
, v
2
, and so on are known as the components, entries
or coordinates of v. The zero vector, denoted 0, is the vector with all
of its entries equal to 0.
The set R
n
denotes the set of all vectors of length n, and we usually
think of these as column vectors.
We can dene addition of two n-vectors by the rule
_
_
_
_
w
1
w
2
.
.
.
w
n
_
_
_
_
+
_
_
_
_
v
1
v
2
.
.
.
v
n
_
_
_
_
=
_
_
_
_
w
1
+v
1
w
2
+v
2
.
.
.
w
n
+v
n
_
_
_
_
.
Also, we can multiply a vector by any single number (usually called a
scalar in this context) by the following rule:
_
_
_
_
v
1
v
2
.
.
.
v
n
_
_
_
_
=
_
_
_
_
v
1
v
2
.
.
.
v
n
_
_
_
_
.
For vectors v
1
, v
2
, . . . , v
k
and numbers
1
,
2
, . . . ,
k
, the vector
1
v
1
+ +
k
v
k
is known as a linear combination of the vectors
v
1
, . . . , v
k
.
A matrix is an array of numbers
_
_
_
_
a
11
a
12
. . . a
1n
a
21
a
22
. . . a
2n
.
.
.
.
.
.
.
.
.
.
.
.
a
m1
a
m2
. . . a
mn
_
_
_
_
.
We denote this array by the single letter A, or by (a
ij
), and we say
that A has m rows and n columns, or that it is an mn matrix. We
also say that A is a matrix of size mn. If m = n, the matrix is said
to be square. The number a
ij
is known as the (i, j)th entry of A. The
row vector (a
i1
, a
i2
, . . . , a
in
) is row i of A, or the ith row of A, and
the column vector
_
_
_
_
a
1j
a
2j
.
.
.
a
mj
_
_
_
_
is column j of A, or the jth column of A.
It is useful to think of row and column vectors as matrices. For
example, we may think of the row vector (1, 2, 4) as being the same as
the 1 3 matrix ( 1 2 4 ). (Indeed, the only visible dierence is that
the vector has commas and the matrix does not, and this is merely a
notational dierence.)
8
Vectors and matrices
Recall that if A and B are two matrices of the same size then we
dene A+B to be the matrix whose elements are the sums of the
corresponding elements in A and B. Formally, the (i, j)th entry of the
matrix A+B is a
ij
+b
ij
where a
ij
and b
ij
are the (i, j)th entries of A
and B, respectively. Also, if c is a number, we dene cA to be the
matrix whose elements are c times those of A; that is, cA has (i, j)th
entry ca
ij
. If A and B are matrices such that the number (say n) of
columns of A is equal to the number of rows of B, we dene the
product C = AB to be the matrix whose elements are
c
ij
= a
i1
b
1j
+a
i2
b
2j
+ +a
in
b
nj
.
The transpose of a matrix
A = (a
ij
) =
_
_
_
_
a
11
a
12
. . . a
1n
a
21
a
22
. . . a
2n
.
.
.
.
.
.
.
.
.
.
.
.
a
m1
a
m2
. . . a
mn
_
_
_
_
is the matrix
A
T
= (a
ji
) =
_
_
_
_
a
11
a
21
. . . a
m1
a
12
a
22
. . . a
m2
.
.
.
.
.
.
.
.
.
.
.
.
a
1n
a
2n
. . . a
mn
_
_
_
_
that is obtained from A by interchanging rows and columns.
Example: For example, if A =
_
1 2
3 4
_
and B = ( 1 5 3 ) , then
A
T
=
_
1 3
2 4
_
, B
T
=
_
_
1
5
3
_
_
.
The transpose has the following properties:
1 1
See Anton or Simon and
Blume.
(A
T
)
T
= A, (A+B)
T
= A
T
+B
T
, (AB)
T
= B
T
A
T
.
The rst two properties follow immediately from the denition. The
last one can be stated as: The transpose of the product of two
matrices is the product of the transposes in the reverse order.
Learning activity 2.1
Check these. For the last property, if A is an mn matrix and B is
n p, look at the dimensions of the matrices (AB)
T
, A
T
, B
T
and
B
T
A
T
to understand why the multiplication of the transposes must be
in the reverse order.
9
We will often write a column vector in the text as the transpose of a
row vector, either
v =
_
_
_
_
x
1
x
2
.
.
.
x
n
_
_
_
_
= ( x
1
x
2
x
n
)
T
, or v = (x
1
, x
2
, , x
n
)
T
.
2.2.1 Row operations
Recall from Mathematics 2 (for it is not going to be reiterated here in
all its gory detail!) that there are three types of elementary row
operation one can perform on a matrix. These are:
multiply a row by non-zero constant
add a multiple of one row to another
interchange (swap) two rows.
In Mathematics 2, row operations were used to solve systems of linear
equations by reducing the augmented matrix of the system to echelon
form. (Yes, we are afraid you do have to remember this!) For example,
the following form is an echelon matrix:
_
_
_
_
_
_
1 . . .
0 0 1 . . .
0 0 0 1 . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 0 . . . 0
.
.
.
0
_
_
_
_
_
_
,
where the entries denote any numbers. The 1s which have been
indicated are called leading ones.
A matrix in echelon form has the following properties:
(1) every non-zero row begins with a leading one
(2) a leading one in a lower row is further to the right
(3) zero rows are at the bottom of the matrix.
Once the augmented matrix is in echelon form, you can either solve the
resulting equations using back substitution, or continue with row
operations to further reduce the matrix to reduced echelon form.
A matrix is in reduced echelon form if, in addition to echelon form, it
has the property that
(4) every column with a leading one has zeros elsewhere.
The example above has reduced echelon form,
_
_
_
_
_
_
1 0 0 . . .
0 0 1 0 . . .
0 0 0 1 . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 0 . . . 0
.
.
.
0
_
_
_
_
_
_
,
10
05B Mathematics 2 (for it is not going to be reiterated here in
05B Mathematics 2, row operations were used to solve systems of linear
where the entries denote any numbers and the indicated 1s are
leading ones.
To achieve this using row operations, reduce the augmented matrix to
echelon form. Then beginning with the last row add suitable multiples
to each row above to get zeros above the leading ones.
Example: Consider the system of equations
x
1
+x
2
+x
3
= 3
2x
1
+x
2
+x
3
= 4
x
1
x
2
+ 2x
3
= 5.
We solve this system by reducing the augmented matrix using
elementary row operations. (Here, R
i
indicates the ith row and the row
operations are as indicated.)
( A|b) =
_
_
1 1 1 3
2 1 1 4
1 1 2 5
_
_
R
2
2R
1
R
3
R
1
_
_
1 1 1 3
0 1 1 2
0 2 1 2
_
_
(1)R
2
_
_
1 1 1 3
0 1 1 2
0 2 1 2
_
_
R
3
+ 2R
2
_
_
1 1 1 3
0 1 1 2
0 0 3 6
_
_
(
1
3
)R
3
_
_
1 1 1 3
0 1 1 2
0 0 1 2
_
_
.
This matrix is in echelon form, continue to reduced echelon form,
R
1
R
3
R
2
R
3
_
_
1 1 0 1
0 1 0 0
0 0 1 2
_
_

R
1
R
2
_
_
1 0 0 1
0 1 0 0
0 0 1 2
_
_
.
The solution can now be read directly from the matrix. This system of
equations is equivalent to
x
1
= 1, x
2
= 0, x
3
= 2.
The system has a unique solution.
2.2.2 Determinants
The determinant of a square matrix A is a particular number
associated with A, written det A or |A|. When A is a 2 2 matrix, the
11
determinant is given by the formula
det
_
a b
c d
_
=
a b
c d
= ad bc.
For example,
det
_
1 2
2 3
_
= 1 3 2 2 = 1.
To extend this denition to n n matrices, we need two denitions.
Denition 2.1 The (i, j) minor of A, denoted by M
ij
, is the
determinant of the (n 1) (n 1) matrix obtained by removing the
ith row and jth column of A.
Denition 2.2 The (i, j) cofactor of a matrix A is
C
ij
= (1)
i+j
M
ij
.
Example: Let
A =
_
_
1 2 3
4 1 1
1 3 0
_
_
.
Then the minor M
23
and the cofactor C
23
are
M
23
=
1 2
1 3
= 5 and C
23
= (1)
(2+3)
M
23
=
1 2
1 3
= 5.
There is a simple way to associate the cofactor C
ij
with the entry a
ij
of
the matrix. Locate the entry a
ij
and cross out the row and the column
containing a
ij
. Then evaluate the determinant of the (n 1) (n 1)
matrix which remains. This is the minor, M
ij
. Then give it a + or
sign according to the position of a
ij
on the following pattern:
_
_
+ + . . .
+ + . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
_
_
.
If A is an n n matrix, the determinant of A, is given by
|A| =
a
11
a
12
. . . a
1n
a
21
a
22
. . . a
2n
.
.
.
.
.
.
.
.
.
.
.
.
a
n1
a
n2
. . . a
nn
def
= a
11
C
11
+a
12
C
12
+. . . +a
1n
C
1n
.
(Here, X
def
= Y means that X is dened to be equal to Y .) This is
called the cofactor expansion of det(A) by row one. It is a recursive
denition, meaning that the determinant of a 3 3 matrix is dened as
a sum of 2 2 determinants, and the determinant of an n n matrix
is given in terms of (n 1) (n 1) determinants.
Example: We calculate the determinant of the matrix A above:
|A| = 1C
11
+ 2C
12
+ 3C
13
= 1
1 1
3 0
4 1
1 0
+ 3
4 1
1 3
= 1(3) 2(1) + 3(13) = 34.

12
' '
' '
For very large matrices this method can involve too many
computations, and it is preferable to use row operations to reduce the
matrix to an upper triangular matrix as shown in Mathematics 2.
However, it turns out that the cofactor expansion is much more useful
than it rst appears, due to the following theorem.
2 2
For a proof, see Anton.
Theorem 2.1 If A is an n n matrix, then the determinant of A can
be computed by multiplying the entries of any row (or column) by their
cofactors and summing the resulting products:
|A| = a
i1
C
i1
+a
i2
C
i2
+. . . +a
in
C
in
(cofactor expansion by row i)
|A| = a
1j
C
1j
+a
2j
C
2j
+. . . +a
nj
C
nj
(cofactor expansion by column j).
This allows you to choose any row or any column of a matrix to nd its
determinant using a cofactor expansion.
Example: In the example above, instead of using the cofactor
expansion by row one as shown, we can choose to evaluate the
determinant of the matrix A by row three or column three, which will
involve fewer calculations since a
33
= 0. To check the result |A| = 34,
we will evaluate the determinant again using column three. Remember
the correct cofactor signs,
|A| =
1 2 3
4 1 1
1 3 0
= 3
4 1
1 3
1 2
1 3
+ 0 = 3(13) (5) = 34.

Calculate the determinant of each of the following matrices. For A, use
repeated cofactor expansions along appropriate rows; for B use row
operations, or a combination of both methods.
A =
_
_
_
7 5 2 3
2 0 0 0
23 57 1 1
11 2 0 0
_
_
_ B =
_
_
_
1 4 3 2
2 7 5 1
1 2 6 0
2 10 14 4
_
_
_.
2.2.3 Invertible matrices
In Mathematics 2 you learned two methods to calculate the inverse of
a 3 3 matrix either using determinants or using row operations.
Both of these methods generalise to n n matrices. Let A be an
n n matrix.
Inverse using determinants: If |A| = 0, then A
1
= (1/|A|) adj(A),
where adj(A) is the transpose of the matrix of cofactors, and is known
13
05B Mathematics 2.
2
05B Mathematics 2 you learned two methods to calculate the inverse of
as the adjoint matrix of A
3
or the adjugate matrix of A (according to
3
See Anton.
which textbook you are reading):
A
1
=
1
|A|
_
_
_
_
C
11
C
21
. . . C
n1
C
12
C
22
. . . C
n2
.
.
.
.
.
.
.
.
.
.
.
.
C
1n
C
2n
. . . C
nn
_
_
_
_
.
Note that the (i, j) entry of adj(A) is the cofactor C
ji
.
Inverse using row operations: Form the matrix (A I) by placing the
n n identity matrix next to the matrix A. If (A I) can be reduced to
(I B) using row operations, then A is invertible and A
1
= B. If A
cannot be reduced to the identity matrix, then A is not invertible.
You should also recall the fact that the system of equations Ax = b
has a unique solution for any b R
n
if and only if A is invertible. In
particular, the system Ax = 0 has only the solution x = 0 (known as
the trivial solution) if and only if A is invertible. We collect these
results in the following theorem.
Theorem 2.2 If A is an n n matrix, then the following statements
are equivalent (meaning if any one of these statements is true for A,
then all the statements are true).
A
1
exists.
Ax = b has a unique solution for any b R
n
.
Ax = 0 has only the trivial solution, x = 0.
The reduced echelon form of A is I.
|A| = 0.
Review the linear algebra section of Mathematics 2. In particular,
note how the above statements are connected there, and why one
implies the other.
If A is the matrix
A =
_
_
5 4 2
2 4 3
3 1 0
_
_
,
show that the system of equations Ax = b has a unique solution for
any b R
3
.
Find A
1
.
14
05B Mathematics 2. In particular,
Rank, range, null space, and linear equations
2.3 Rank, range, null space, and linear equations
2.3.1 The rank of a matrix
There are several ways of dening the rank of a matrix, and we shall
meet some other (more sophisticated) ways later. All are equivalent.
We begin with the following denition.
Denition 2.3 (Rank of a matrix) The rank, rank(A), of a matrix A
is the number of non-zero rows in an echelon matrix obtained from A
by elementary row operations.
By a non-zero row, we simply mean one that contains entries other
than 0. Since every non-zero row of the echelon form begins with a
leading one, this is equivalent to the following denition.
Denition 2.4 The rank, rank(A), of a matrix A is the number of
leading ones in an echelon matrix obtained from A by elementary row
operations.
Example: Consider the matrix
A =
_
_
1 2 1
2 2 0
3 4 1
1
2
3
_
_
.
Reducing this to echelon form using elementary row operations, we
have:
_
_
1 2 1
2 2 0
3 4 1
1
2
3
_
_
_
_
1 2 1
0 2 2
0 2 2
1
0
0
_
_
_
_
1 2 1
0 1 1
0 2 2
1
0
0
_
_
_
_
1 2 1
0 1 1
0 0 0
1
0
0
_
_
.
This last matrix is in echelon form and has two non-zero rows (and two
leading ones), so the matrix A has rank 2.
Prove that the matrix
A =
_
_
3 1 1
1 1 1
1 2 2
3
1
1
_
_
has rank 2.
Generally, if A is an mn matrix, then the number of non-zero rows
(the number of leading ones) in a row echelon form of A can certainly
15
be no more than the total number of rows, m. Furthermore, since the
leading ones must be in dierent columns, the number of leading ones
in the echelon form can be no more than the total number, n, of
columns. Thus we have:
Theorem 2.3 For an mn matrix A, rank(A) min{m, n}, where
min{m, n} denotes the smaller of the two integers m and n.
If a square matrix A of size n n has rank n, then its reduced echelon
form has a leading one in every row and a leading one in every column.
Since every column with a leading one has zeros elsewhere, it follows
that the reduced echelon form of A must be I, the n n identity
matrix. We therefore have one more equivalent statement to add to
our theorem:
Theorem 2.4 If A is an n n matrix, then the following statements
are equivalent.
A
1
exists.
Ax = b has a unique solution for any b R
n
.
Ax = 0 has only the trivial solution, x = 0.
The reduced echelon form of A is I.
|A| = 0.
The rank of A is n.
2.3.2 Rank and systems of linear equations
Recall that to solve a system of linear equations, one forms the
augmented matrix and reduces it to echelon form by using elementary
row operations.
Example: Consider the system of equations
x
1
+ 2x
2
+x
3
= 1
2x
1
+ 2x
2
= 2
3x
1
+ 4x
2
+x
3
= 2.
Using row operations to reduce the augmented matrix to echelon form,
we obtain
_
_
1 2 1 1
2 2 0 2
3 4 1 2
_
_
_
_
1 2 1 1
0 2 2 0
0 2 2 1
_
_
_
_
1 2 1 1
0 1 1 0
0 2 2 1
_
_
_
_
1 2 1 1
0 1 1 0
0 0 0 1
_
_
.
Thus the original system of equations is equivalent to the system
x
1
+ 2x
2
+x
3
= 1
x
2
+x
3
= 0
0x
1
+ 0x
2
+ 0x
3
= 1.
16
But this system has no solutions, since there are no values of x
1
, x
2
, x
3
that satisfy the last equation. It reduces to the false statement
0 = 1, whatever values we give the unknowns. We deduce,
therefore, that the original system has no solutions. Notice that in this
case there is no reason to reduce the matrix further.
If, as in the example just given, the echelon form of an augmented
matrix has a row of the kind (0 0 . . . 0 a), with a = 0, then the
original system is equivalent to one in which there is an equation
0x
1
+ 0x
2
+ + 0x
n
= a (a = 0).
Clearly this equation cannot be satised by any values of the x
i
s, and
we say that such a system is inconsistent.
If there is no row of this kind, the system has at least one solution, and
we say that it is consistent.
The number of non-zero rows in the echelon form of the augmented
matrix is known as the rank of the system. Thus the rank of the system
is the rank of the augmented matrix. Note, however, that if the system
is consistent then there can be no leading one in the last column of the
reduced augmented matrix, for that would mean there was a row of the
form (0 0 . . . 0 1). Thus, in fact, the rank of a consistent system
Ax = b is precisely the same as the rank of the matrix A.
Suppose we have a consistent system, and suppose that the rank r is
strictly less than n, the number of unknowns. Then the system in
echelon form (and hence the original one) does not provide enough
information to specify the values of x
1
, x
2
, . . . , x
n
uniquely.
Example: Suppose we are given a system for which the augmented
matrix reduces to the echelon form
_
_
_
1 3 2 0 2 0
0 0 1 2 0 3
0 0 0 0 0 1
0 0 0 0 0 0
0
1
5
0
_
_
_.
Here the rank (number of non-zero rows) is r = 3 which is strictly less
than the number of unknowns, n = 6.
Continuing to reduced echelon form, we obtain the matrix
_
_
_
1 3 0 4 2 0 28
0 0 1 2 0 0 14
0 0 0 0 0 1 5
0 0 0 0 0 0 0
_
_
_.
Verify this. What are the additional two row operations which need to
be carried out?
17
The corresponding system is
x
1
+ 3x
2
+ 4x
4
+ 2x
5
= 28
x
3
+ 2x
4
= 14
x
6
= 5.
The variables x
1
, x
3
and x
6
correspond to the columns with the
leading ones and are called leading variables. The other variables are
called non-leading variables.
Solving for the leading variables x
1
, x
3
and x
6
in terms of x
2
, x
4
and
x
5
we get
x
6
= 5, x
3
= 14 2x
4
, x
1
= 28 3x
2
4x
4
2x
5
.
The form of these equations tells us that we can assign any values to
x
2
, x
4
and x
5
, and then the leading variables will be determined.
Explicitly, if we give x
2
, x
4
, x
5
the arbitrary values s, t, u, where s, t, u
represent any real numbers, the solution is given by
x
1
= 283s4t2u, x
2
= s, x
3
= 142t, x
4
= t, x
5
= u, x
6
= 5.
Observe that there are innitely many solutions, because the so-called
free unknowns x
2
, x
4
, x
5
can take any values s, t, u R.
Generally, we can describe what happens when the echelon form has
r < n non-zero rows (0 0 . . . 0 1 . . . ). If the leading 1 is in the
kth column it is the coecient of the unknown x
k
. So if the rank is r
and the leading 1s occur in columns c
1
, c
2
, . . . , c
r
then the general
solution to the system can be expressed in a form where the unknowns
x
c
1
, x
c
2
, . . . , x
c
r
(the leading variables) are given in terms of the other
n r unknowns (the non-leading variables), and those n r unknowns
are free to take any values. In the preceding example, we have n = 6
and r = 3, and the 3 unknowns x
1
, x
3
, x
6
can be expressed in terms of
the 6 3 = 3 free unknowns x
2
, x
4
, x
5
.
In the case r = n, where the number of non-zero rows r in the echelon
form is equal to the number of unknowns n, there is only one solution
to the system for there is a leading one in every column since the
leading 1s move one step to the right as we go down the rows. In this
case there is a unique solution obtained from the reduced echelon form.
In fact, this can be thought of as a special case of the more general one
discussed above: since r = n there are n r = 0 free unknowns, and
the solution is therefore unique.
We can now summarise our conclusions concerning a general linear
system.
If the echelon form has a row (0 0 . . . 0 a), with a = 0, the
original system is inconsistent; it has no solutions.
If the echelon form has no rows of the above type it is consistent,
and the general solution involves n r free unknowns, where r is
the rank. When r < n there are innitely many solutions, but when
r = n there are no free unknowns and so there is a unique solution.
18
2.3.3 General solution of a linear system in vector notation
Consider the system solved above. We found that the general solution
in terms of the three free unknowns, or parameters, s, t, u is
x
1
= 283s4t2u, x
2
= s, x
3
= 142t, x
4
= t, x
5
= u, x
6
= 5.
If we write x as a column vector,
x =
_
_
_
_
_
_
_
x
1
x
2
x
3
x
4
x
5
x
6
_
_
_
_
_
_
_
,
then
x =
_
_
_
_
_
_
_
28 3s 4t 2u
s
14 2t
t
u
5
_
_
_
_
_
_
_
=
_
_
_
_
_
_
_
28
0
14
0
0
5
_
_
_
_
_
_
_
+
_
_
_
_
_
_
_
3s
s
0
0
0
0
_
_
_
_
_
_
_
+
_
_
_
_
_
_
_
4t
0
2t
t
0
0
_
_
_
_
_
_
_
+
_
_
_
_
_
_
_
2u
0
0
0
u
0
_
_
_
_
_
_
_
.
That is, the general solution is
x = v +su
1
+tu
2
+uu
3
, s, t, u R,
where
v =
_
_
_
_
_
_
_
28
0
14
0
0
5
_
_
_
_
_
_
_
, u
1
=
_
_
_
_
_
_
_
3
1
0
0
0
0
_
_
_
_
_
_
_
, u
2
=
_
_
_
_
_
_
_
4
0
2
1
0
0
_
_
_
_
_
_
_
, u
3
=
_
_
_
_
_
_
_
2
0
0
0
1
0
_
_
_
_
_
_
_
.
Applying the same method generally to a consistent system of rank r
with n unknowns, we can express the general solution of a consistent
system Ax = b in the form
x = v +s
1
u
1
+s
2
u
2
+ +s
nr
u
nr
.
Note that, if we put all the s
i
s equal to 0, we get a solution x = v,
which means that Av = b, so v is a particular solution of the system.
Putting s
1
= 1 and the remaining s
i
s equal to zero, we get a solution
x = v +u
1
, which means that A(v +u
1
) = b. Thus
b = A(v +u
1
) = Av +Au
1
= b +Au
1
.
Comparing the rst and last expressions, we see that Au
1
is the zero
vector 0. Clearly, the same equation holds for u
2
, . . . , u
nr
. So we
have proved the following.
If A is an mn matrix of rank r, the general solution of Ax = b is
the sum of:
a particular solution v of the system Ax = b and
19
a linear combination s
1
u
1
+s
2
u
2
+ +s
nr
u
nr
of solutions
u
1
, u
2
, . . . , u
nr
of the system Ax = 0.
If A has rank n, then Ax = 0 only has the solution x = 0, and so
Ax = b has a unique solution: v +0 = v.
Solve the following system of equations Ax = b by reducing the
augmented matrix to reduced echelon form:
x
1
x
2
+x
3
+x
4
+ 2x
5
= 4
x
1
+x
2
+x
4
x
5
= 3
x
1
x
2
+ 2x
3
+ 3x
4
+ 4x
5
= 7.
Show that your solution can be written in the form p +su
1
+tu
2
where Ap = b, Au
1
= 0 and Au
2
= 0.
2.3.4 Null space
It is clear from what we have just seen that the general solution to a
consistent linear system involves solutions to the system Ax = 0. This
set of solutions is given a special name: the null space or kernel of a
matrix A. This null space, denoted N(A), is the set of all solutions x
to Ax = 0, where 0 is the zero vector. That is,
Denition 2.5 (Null space) For an mn matrix A, the null space of
A is the subset
N(A) = {x R
n
: Ax = 0}
of R
n
, where 0 = (0, 0, . . . , 0)
T
is the zero vector of R
m
.
Note that the linear system Ax = 0 is always consistent since, for
example, 0 is a solution. It follows from the discussion above that the
general solution to Ax = 0 involves n r free unknowns, where r is
the rank of A (and that, if r = n, then there is just one solution,
namely x = 0).
Example: Referring back to the example on page 16, consider the
system of equations Ax = b given by
x
1
+ 2x
2
+x
3
= 1
2x
1
+ 2x
2
= 2
3x
1
+ 4x
2
+x
3
= 2.
We showed that this system is inconsistent. Now consider instead the
system of equations Ax = 0. Using the same row operations to reduce
20
the matrix A to echelon form as we did previously for the augmented
matrix, and continuing to reduced row echelon form, we obtain
A =
_
_
1 2 1
2 2 0
3 4 1
_
_
_
_
1 2 1
0 2 2
0 2 2
_
_
_
_
1 2 1
0 1 1
0 0 0
_
_
_
_
1 0 1
0 1 1
0 0 0
_
_
Setting the non-leading variable x
3
= t, we nd that the null space of
A consists of all vectors, x,
x = t
_
_
1
1
1
_
_
t R.
The system of equations Ax = 0 has innitely many solutions.
We now formalise the connection between the solution set of a general
consistent linear system, and the null space of the matrix determining
the system.
Theorem 2.5 Suppose that A is an mn matrix, that b R
n
, and
that the system Ax = b is consistent. Suppose that x
0
is any solution
of Ax = b. Then the set of all solutions of Ax = b consists precisely
of the vectors x
0
+z for z N(A); that is,
{x : Ax = b} = {x
0
+z : z N(A)}.
Proof To show the two sets are equal, we show that each is a subset
of the other. Suppose, rst, that x is any solution of Ax = b. Because
x
0
is also a solution, we have
A(x x
0
) = Ax Ax
0
= b b = 0,
so the vector z = x x
0
is a solution of the system Az = 0; in other
words, z N(A). But then x = x
0
+ (x x
0
) = x
0
+z where
z N(A). This shows that
{x : Ax = b} {x
0
+z : z N(A)}.
Conversely, if z N(A) then
A(x
0
+z) = Ax
0
+Az = b +0 = b,
so x
0
+z {x : Ax = b}. This shows that
{x
0
+z : z N(A)} {x : Ax = b}.
So the two sets are equal, as required. 2
2.3.5 Range
The range of a matrix A is dened as follows.
Denition 2.6 (Range of a matrix) Suppose that A is an mn
matrix. Then the range of A, denoted by R(A), is the subset
R(A) = {Ax : x R
n
}
of R
m
. That is, the range is the set of all vectors y R
m
of the form
y = Ax for some x R
n
.
21
Suppose that the columns of A are c
1
, c
2
, . . . , c
n
. Then we may write
A = (c
1
c
2
. . . c
n
). If x = (
1
,
2
, . . . ,
n
)
T
R
n
, then the product
Ax is equal to
1
c
1
+
2
c
2
+ +
n
c
n
.
Convince yourself of this last statement. Write out each side using
c
i
= (c
1i
, c
2i
, . . . , c
mi
)
T
to show that
Ax =
1
c
1
+
2
c
2
+ +
n
c
n
.
This is a very important result which will be used many times in this
course, so make sure you understand how it works.
So, R(A), as the set of all such products, is the set of all linear
combinations of the columns of A. For this reason R(A) is also called
the column space of A. (More on this in the next chapter.)
Example: Suppose that A =
_
_
1 2
1 3
2 1
_
_
. Then for x = (
1
,
2
)
T
,
Ax =
_
_
1 2
1 3
2 1
_
_
_
2
_
=
_
_
1
+ 2
2
1
+ 3
2
2
1
+
2
_
_
,
so
R(A) =
_
_
_
_
_
1
+ 2
2
1
+ 3
2
2
1
+
2
_
_
:
1
,
2
R
_
_
_
.
This may also be written as
R(A) = {
1
c
1
+
2
c
2
:
1
,
2
R} ,
where
c
1
=
_
_
1
1
2
_
_
, c
2
=
_
_
2
3
1
_
_
are the columns of A.
2.4 Learning outcomes
At the end of this chapter and the relevant reading, you should be able
to:
solve systems of linear equations and calculate determinants (all of
this being revision)
compute the rank of a matrix and understand how the rank of a
matrix relates to the solution set of a corresponding system of
linear equations
22
Sample examination questions
determine the solution to a linear system using vector notation
explain what is meant by the range and null space of a matrix and
be able to determine these for a given matrix.
2.5 Sample examination questions
The following are typical exam questions, or parts of such questions.
Question 2.1 Find the general solution of the following system of
linear equations. Express your answer in the form x = v +su, where v
and u are in R
3
.
3x
1
+x
2
+x
3
= 3
x
1
x
2
x
3
= 1
x
1
+ 2x
2
+ 2x
3
= 1.
Question 2.2 Write down the augmented matrix for the following
system of equations, and use it to solve the system by reducing the
augmented matrix to reduced echelon form. Express your answer in
vector form.
x
1
+x
2
+x
3
+x
5
= 1
3x
1
+ 3x
2
+ 6x
3
+ 3x
4
+ 9x
5
= 6
2x
1
+ 2x
2
+ 4x
3
+x
4
+ 6x
5
= 5.
Question 2.3 Find the rank of the matrix
C =
_
_
3 2 1 3
2 1 1 0
6 2 4 6
_
_
.
(a) If C is the augmented matrix of a system of equations Ax = b,
C = (A|b), nd all possible solutions.
(b) If C is the coecient matrix of a system of homogeneous
equations, Cx = 0, nd all possible solutions.
Question 2.4 Find the rank of the matrix
A =
_
_
_
1 0 1 0 2
2 1 1 1 3
1 3 1 2 2
0 3 2 2 0
_
_
_.
Determine N(A), the null space of A and R(A) the range of A.
Question 2.5 Consider the system of linear equations Ax = b, where
and are constants:
A =
_
_
1 2 0
5 1
1 1 1
_
_
x =
_
_
x
y
z
_
_
b =
_
_
2
7
_
_
Compute the determinant of A, |A|. Determine for which values of
and this system has:
23
(a) a unique solution
(b) no solutions
(c) innitely many solutions.
In case (a), use Cramers rule to nd the value of z in terms of
and . In case (c), solve the system using row operations and
express the solution in vector form, x = p +tv.
2.6 Answers to or comments on selected
activities
Activity 2.2 The answers are |A| = 20 and |B| = 150.
For |A|, expand rst by row 2 and then by row 3:
|A| =
7 5 2 3
2 0 0 0
23 57 1 1
11 2 0 0
= 2
5 2 3
57 1 1
2 0 0
= 2(2)
2 3
1 1
= 20
For |B|, using row operations and cofactor expansion:
|B| =
1 4 3 2
2 7 5 1
1 2 6 0
2 10 14 4
1 4 3 2
0 1 1 3
0 6 3 2
0 2 8 0
1 1 3
6 3 2
2 8 0
1 1 3
0 9 16
0 6 6
9 16
6 6
= 6
9 16
1 1
= 6(25) = 150
Activity 2.4 By Theorem 2.2, there are a number of ways of proving
this. For example, we can check that the determinant (expanding by
the rst row) is
|A| = 5(3) 4(9) 2(10) = 1 = 0
and the Theorem now applies, since having a nonzero determinant is
equivalent to there being a unique solution to the system Ax = b for
any b.
The inverse matrix is
A
1
=
_
_
3 2 4
9 6 11
10 7 12
_
_
.
You should already know that you have the correct answer, as you
should have checked that AA
1
= I !
Activity 2.5 Using row operations, we reduce the matrix as follows.
_
_
3 1 1
1 1 1
1 2 2
3
1
1
_
_
_
_
1 1 1
3 1 1
1 2 2
1
3
1
_
_
_
_
1 1 1
0 4 4
0 3 3
1
0
0
_
_
24
Answers to or comments on selected activities
_
_
1 1 1
0 1 1
0 1 1
1
0
0
_
_
_
_
1 1 1
0 1 1
0 0 0
1
0
0
_
_
.
Since there are two non-zero rows in this echelon form, rank(A) = 2.
Activity 2.6 The next row operation is R
2
3R
3
followed by
R1 + 2R
2
.
Activity 2.7 We reduce the augmented matrix to reduced echelon
form:
(A|b) =
_
_
1 1 1 1 2 4
1 1 0 1 1 3
1 1 2 3 4 7
_
_
R
2
+R
1
R
3
R
1
_
_
1 1 1 1 2 4
0 0 1 2 1 1
0 0 1 2 2 3
_
_
R
3
R
2
_
_
1 1 1 1 2 4
0 0 1 2 1 1
0 0 0 0 1 2
_
_
R
2
R
3
R
1
2R
3
_
_
1 1 1 1 0 0
0 0 1 2 0 1
0 0 0 0 1 2
_
_
R
1
R
2
_
_
1 1 0 1 0 1
0 0 1 2 0 1
0 0 0 0 1 2
_
_
.
The leading variables are x
1
, x
3
and x
5
. Set the non-leading variables
to arbitrary constants: x
2
= s, x
4
= t, and solve for the leading
variables in terms of these parameters, starting with the bottom row,
= x
5
= 2, x
4
= t, x
3
= 12t, x
2
= s, x
1
= 1+s+t
= x =
_
_
_
_
_
x
1
x
2
x
3
x
4
x
5
_
_
_
_
_
=
_
_
_
_
_
1 +s +t
s
1 2t
t
2
_
_
_
_
_
=
_
_
_
_
_
1
0
1
0
2
_
_
_
_
_
+s
_
_
_
_
_
1
1
0
0
0
_
_
_
_
_
+t
_
_
_
_
_
1
0
2
1
0
_
_
_
_
_
= p +su
1
+tu
2
,
where s, t R. Note that p is a particular solution and that u
1
and u
2
are solutions of the system Ax = 0, for
_
_
1 1 1 1 2
1 1 0 1 1
1 1 2 3 4
_
_
_
_
_
_
_
1
0
1
0
2
_
_
_
_
_
=
_
_
4
3
7
_
_
_
_
1 1 1 1 2
1 1 0 1 1
1 1 2 3 4
_
_
_
_
_
_
_
1
1
0
0
0
_
_
_
_
_
=
_
_
0
0
0
_
_
,
_
_
1 1 1 1 2
1 1 0 1 1
1 1 2 3 4
_
_
_
_
_
_
_
1
0
2
1
0
_
_
_
_
_
=
_
_
0
0
0
_
_
.
25
2.7 Sketch answers to or comments on selected
questions
Question 2.1 The general solution takes the form v +su where
v = (1, 0, 0)
T
and u = (0, 1, 1)
T
.
Question 2.2 The reduced echelon form of the augmented matrix is
_
_
1 1 0 0 1 1
0 0 1 0 2 2
0 0 0 1 0 1
_
_
x
2
and x
5
are non-leading variables. Set x
2
= r and x
5
= s.
Using the equations, solve for the other variables in terms of these.
_
_
_
_
_
x
1
x
2
x
3
x
4
x
5
_
_
_
_
_
=
_
_
_
_
_
1 r +s
r
2 2s
1
s
_
_
_
_
_
=
_
_
_
_
_
1
0
2
1
0
_
_
_
_
_
+r
_
_
_
_
_
1
1
0
0
0
_
_
_
_
_
+s
_
_
_
_
_
1
0
2
0
1
_
_
_
_
_
, r, s R.
Question 2.3 The rank of C is 3. (a) If C is the augmented matrix
of a system Ax = b then the system is inconsistent. There is no
solution (but if there had been a solution x it would have been a vector
in R
3
). (b) The general solution of Cx = 0 is
x =
_
_
_
x
1
x
2
x
3
x
4
_
_
_ =
_
_
_
t
t
t
0
_
_
_ = t
_
_
_
1
1
1
0
_
_
_, t R
These vectors are in R
4
.
Question 2.4 Using row operations, the matrix A reduces to echelon
form
A . . .
_
_
_
1 0 1 0 2
0 1 1 1 1
0 0 1 1 3
0 0 0 0 0
_
_
_.
There are three nonzero rows (three leading ones), so the rank of A is
3.
To nd N(A) we need to solve Ax = 0, which is a system of 4
equations in 5 unknowns. Call them x
1
, x
2
, . . . , x
5
. Continuing to
reduced echelon form,
A . . .
_
_
_
1 0 0 1 1
0 1 0 0 2
0 0 1 1 3
0 0 0 0 0
_
_
_.
The leading variables are x
1
, x
2
, and x
3
. Set the non-leading variables
x
4
= s and x
5
= t. Then the solution is
_
_
_
_
_
x
1
x
2
x
3
x
4
x
5
_
_
_
_
_
=
_
_
_
_
_
s +t
2t
s 3t
s
t
_
_
_
_
_
= s
_
_
_
_
_
1
0
1
1
0
_
_
_
_
_
+ t
_
_
_
_
_
1
2
3
0
1
_
_
_
_
_
, s, t R.
26
Sketch answers to or comments on selected questions
So the null space consists of all vectors of the form x = sv
1
+tv
2
,
where v
1
and v
2
are the vectors displayed above. It is a subset of R
5
.
The range of A can be described as the set of all linear combinations of
the columns of A,
R(A) = {
1
c
1
+
2
c
2
+
3
c
3
+
4
c
4
+
5
c
5
:
i
R}
where
c
1
=
_
_
_
1
2
1
0
_
_
_, c
2
=
_
_
_
0
1
3
3
_
_
_, c
3
=
_
_
_
1
1
1
2
_
_
_, c
4
=
_
_
_
0
1
2
2
_
_
_, c
5
=
_
_
_
2
3
2
0
_
_
_.
This is a subset of R
4
. We will nd a better way to describe this set
when we look at the column space of a matrix in the next chapter.
Question 2.5 |A| = 3 9. (a) If |A| = 0, that is, if = 3, then the
system will have a unique solution. In this case, using Cramers rule,
4 4
See the Mathematics 2
guide to review this.
z = (3 3)/( 3).
To answer (b) and (c), reduce the augmented matrix to echelon form
with = 3
(A|b) =
_
_
1 2 0 2
5 1 3 7
1 1 1
_
_

_
_
1 2 0 2
0 9 3 3
0 3 1 2
_
_
_
_
1 2 0 2
0 3 1 1
0 3 1 2
_
_

_
_
1 2 0 2
0 3 1 1
0 0 0 1
_
_
.
So if = 3, this system will be inconsistent if = 1, which answers
(b).
If = 3 and = 1 we have (c) innitely many solutions. Setting
= 1 and continuing to reduced echelon form,
(A|b) . . .
_
_
1 2 0 2
0 1
1
3
1
3
0 0 0 0
_
_

_
_
1 0
2
3
4
3
0 1
1
3
1
3
0 0 0 0
_
_
.
The solution can now be read from the matrix. Setting the non-leading
variable z = t,
_
_
x
y
z
_
_
=
_
_
4
3

2
3
t
1
3
+
1
3
t
t
_
_
=
_
_
4
3
1
3
0
_
_
+t
_
_
2
3
1
3
1
_
_
= p +tv, t R.
27
05B Mathematics 2
28
Chapter 3
Vector spaces, basis,
dimension
Contents
3.1 Introduction . 29
3.2 Vector spaces 29
3.3 Subspaces . . 32
3.4 Subspaces
connected
with matrices 35
3.5 Linear span . 36
3.6 Linear inde-
pendence . . . 38
3.7 Testing linear
independence
in R
n
. . . . . 40
3.8 Bases and di-
mension . . . 43
3.9 Finding a ba-
sis for a linear
span in R
n
. . 48
3.10 Basis and
dimension of
range and null
space . . . . . 49
3.11 Learning out-
comes . . . . 52
3.12 Sample exam-
ination questions 52
3.13 Answers to or
comments on
selected activ-
ities . . . . . 54
3.14 Sketch an-
swers to or
comments
on selected
questions . . . 58
Reading
There is no specic essential reading for this chapter. It is essential
that you do some reading, but the topics discussed in this chapter are
adequately covered in so many texts on linear algebra that it would be
articial and unnecessarily limiting to specify precise passages from
precise texts. The list below gives examples of relevant reading. (For
full publication details, see Chapter 1.)
Anton, H. Elementary Linear Algebra. Chapters 3, 4 and 5.
Lay, David C. Linear Algebra and its Applications. Chapter 4,
sections 4.14.6.
Ostaszewski, A. Advanced Mathematical Methods. Chapter 1.
Simon, C.P. and Blume, L. Mathematics for Economists.
Chapter 11, and Chapter 27.
3.1 Introduction
In this chapter we study the important theoretical concept of a vector
space.
3.2 Vector spaces
3.2.1 Denition of a vector space
We know that vectors of R
n
can be added together and that they can
be scaled by real numbers. That is, for every x, y R
n
and every
R, it makes sense to talk about x +y and x. Furthermore, these
operations of addition and multiplication by a scalar (that is,
multiplication by a real number) behave and interact sensibly, in that,
29
for example,
(x +y) = x +y,
(x) = ()x,
x +y = y +x,
and so on.
But it is not only vectors in R
n
that can be added and multiplied by
scalars. There are other sets of objects for which this is possible.
Consider the set V of all functions from R to R. Then any two of these
functions can be added: given f, g V we simply dene the function
f +g by
(f +g)(x) = f(x) +g(x).
Also, for any R, the function f is given by
(f)(x) = (f(x)).
These operations of addition and scalar multiplication are sometimes
said to be pointwise addition and pointwise scalar multiplication. This
might seem a bit abstract, but think about what the functions x +x
2
and 2x represent: the former is the function x plus the function x
2
,
and the latter is the function x multiplied by the scalar 2. So this is
just a dierent way of looking at something you are already familiar
with. It turns out that V and its rules for addition and multiplication
by a scalar satisfy the same key properties as does the set of vectors in
R
n
with its addition and scalar multiplication. We refer to a set with
an addition and scalar multiplication which behave appropriately as a
vector space. We now give the formal denition of a vector space.
Denition 3.1 (Vector space) A (real) vector space V is a
non-empty set equipped with an addition operation and a scalar
multiplication operation such that for all , R and all u, v, w V ,
1. u +v V (closure under addition)
2. u +v = v +u (the commutative law for addition)
3. u + (v +w) = (u +v) +w (the associative law for addition)
4. there is a single member 0 of V , called the zero vector, such that
for all v V , v +0 = v
5. for every v V there is an element w V (usually written as
v), called the negative of v, such that v +w = 0
6. v V (closure under scalar multiplication)
7. (u +v) = u +v (distributive law)
8. ( +)v = v +v (distributive law)
9. (v) = ()v (associative law for scalar multiplication)
10. 1v = v.
Other properties follow from those listed in the denition. For instance,
we can see that 0x = 0 for all x, as follows:
0x = (0 + 0)x = 0x + 0x,
30
Vector spaces
so, adding the negative 0x of 0x to each side,
0 = 0x+(0x) = (0x+0x)+(0x) = 0x+(0x+(0x)) = 0x+0 = 0x.
(A bit sneaky, but just remember the result: 0x = 0.)
Prove that (1)x = x, the negative of the vector x, using a similar
argument with 0 = 1 + (1).
(Note that this denition says nothing at all about multiplying
together two vectors: the only operations with which the denition is
concerned are addition and scalar multiplication.)
A vector space as we have dened it is called a real vector space, to
emphasise that the scalars , and so on are real numbers rather than
complex numbers. There is a notion of complex vector space, where
the scalars are complex numbers. We shall look at complex vector
spaces in a later chapter. Until then, all scalars will be real numbers.
3.2.2 Examples
Example: The set R
n
is a vector space with the usual way of adding
and scalar multiplying vectors.
Example: The set V of functions from R to R with pointwise addition
and scalar multiplication (described earlier in this section) is a vector
space. Note that the zero vector in this space is the function that maps
every real number to 0that is, the identically-zero function.
Show that all 10 properties of a vector space are satised. In particular,
if the function f is a vector in this space, what is the vector f?
Similarly, if S is any set then the set V of functions f : S R is a
vector space. However, if S is any proper subset of the real numbers
(meaning that S R, but S = R) and S = {0}, then the set V of
functions f : R S is not a vector space. Why not? Well, let f be
any function other than the identically-zero function. Then for some
a R, f(a) = 0. Now, for some R, the real number f(a) will not
belong to S. (Since S is not the whole of R, there is b R such that
b S. Since f(a) = 0, we can take = (b/f(a)), so that f(a) = b.)
This all shows that the function f is not in V , since it is not a
function from R to S.
31
Example: The set of mn matrices with real entries is a vector
space, with the usual addition and scalar multiplication of matrices.
The zero vector in this vector space is the zero mn matrix which
has all entries equal to 0.
Example: Let V be the set of all 3-vectors with third entry equal to 0,
that is,
V =
_
_
_
_
_
x
y
0
_
_
: x, y R
_
_
_
.
Then V is a vector space with the usual addition and scalar
multiplication. To verify this, we need only check that V is closed
under addition and scalar multiplication. The associative, commutative
and distributive laws (properties 2, 3, 7, 8, 9 ,10) will hold for vectors
in V because they hold for all vectors in R
3
(and all linear
combinations of vectors in V are in V ). Furthermore, if we can show
that V is closed under scalar multiplication, then for any particular
v V , 0v = 0 V and (1)v = v V . So we simply need to
check that V = (V is non-empty), that if u, v V then u +v V ,
and if R and v V then v V . Each of these is easy to check.
Verify that for u, v V and R, u +v V and v V .
3.3 Subspaces
The last example above is informative. Arguing as we did there, if V is
a vector space and W V is non-empty and closed under scalar
multiplication and addition, then W too is a vector space (and we do
not need to verify that all the other properties hold). The formal
denition of a subspace is as follows.
Denition 3.2 (Subspace) A subspace W of a vector space V is a
non-empty subset of V that is itself a vector space (under the same
operations of addition and scalar multiplication as V ).
The discussion given justies the following important result.
Theorem 3.1 Suppose V is a vector space. Then a non-empty subset
W of V is a subspace if and only if:
for all u, v W, u +v W (W is closed under addition), and
for all v W and R, v W (W is closed under scalar
multiplication).
32
Subspaces
Example: In R
2
, the lines y = 2x and y = 2x + 1 can be dened as
the sets of vectors,
S =
__
x
y
_
: y = 2x, x R
_
U =
__
x
y
_
: y = 2x + 1, x R
_
.
Each vector in one of the sets is the position vector of a point on that
line. We will show that the set S is a subspace of R
2
, and that the set
U is not a subspace of R
2
.
If v =
_
1
2
_
and p =
_
0
1
_
, these sets can equally well be expressed as,
S = {x : x = tv, t R} U = {x : x = p +tv, t R}.
Show that the two descriptions of S describe the same set of vectors.
To show S is a subspace, we need to show it is closed under addition
and closed under scalar multiplication using any vectors in S and any
scalar in R. Working with the second set of denitions, let u, w be
any vectors in S and let R. Then
u = s
_
1
2
_
w = t
_
1
2
_
for some s, t R.
closure under addition:
u +w = s
_
1
2
_
+t
_
1
2
_
= (s +t)
_
1
2
_
S (since s +t R).
closure under scalar multiplication:
u =
_
s
_
1
2
__
= (s)
_
1
2
_
S (since s R).
This shows that S is a subspace of R
2
.
To show U is not a subspace, any one of the three following
statements (counterexamples) will suce.
1. 0 / U.
2. U is not closed under addition:
_
0
1
_
U
_
1
3
_
U but
_
0
1
_
+
_
1
3
_
=
_
1
4
_
/ U
since 4 = 2(1) + 1.
3. U is not closed under scalar multiplication:
_
0
1
_
U, 2 R but 2
_
0
1
_
=
_
0
2
_
/ U
33
Show that 0 / U. Explain why this suces to show that U is not a
subspace.
The line y = 2x + 1 is an example of an ane set, a translation of a
subspace.
Let v be any non-zero vector in a vector space V . Show that the set
S = {v : R}
is a subspace of V . The set S denes a line through the origin in V .
If V is a vector space, the sets V and {0} are subspaces of V . The set
{0} is not empty, it contains one vector, namely the zero vector. It is a
subspace because 0 +0 = 0 and 0 = 0 for any R.
Given any subset S of a vector space V , how do you decide if it is a
subspace? First check that 0 S. Then using some vectors in the
subset, see if adding them and scalar multiplying them will give you
another vector in S. To prove that S is a subspace, you will need to
verify that it is closed under addition and closed under scalar
multiplication for any vectors in S, so you will need to use letters to
represent general vectors, or components of general vectors, in the set.
That is, using letters show that the sum u +v and the scalar product
u of vectors in S also satisfy the denition of a vector in S.
To prove a set S is not a subspace you only need to nd one
counterexample, one or two particular vectors (use numbers) for which
the sum or the scalar product does not satisfy the denition of S.
Note that if 0 is not in the set, it cannot be a subspace.
Write down a general vector (using letters) and a particular vector
(using numbers) for each of the following subsets. Show that one of
the sets is a subspace of R
3
and the other is not.
S
1
=
_
_
_
_
_
x
x
2
0
_
_
: x R
_
_
_
S
2
=
_
_
_
_
_
x
2x
0
_
_
: x R
_
_
_
34
Subspaces connected with matrices
3.3.1 An alternative characterisation of a subspace
We have seen that a subspace is a non-empty subset W of a vector
space that is closed under addition and scalar multiplication, meaning
that if u, v W and R, then both u +v and v are in W. Now,
it is fairly easy to see that the following equivalent property
characterises when W will be a subspace:
Theorem 3.2 A non-empty subset W of a vector space is a subspace
if and only if for all u, v W and all , R, we have u+v W.
That is, W is a subspace if it is non-empty and closed under linear
combination.
3.4 Subspaces connected with matrices
3.4.1 Null space
Suppose that A is an mn matrix. Then the null space N(A), the set
of solutions to the homogeneous linear system Ax = 0, is a subspace
of R
n
.
Theorem 3.3 For any mn matrix A, N(A) is a subspace of R
n
.
Proof To prove this we have to verify that N(A) = , and that if
u, v N(A) and R, then u +v N(A) and u N(A). Since
A0 = 0, 0 N(A) and hence N(A) = . Suppose u, v N(A).
Then to show u +v N(A) and u N(A), we must show that
u +v and u are solutions of Ax = 0. We have
A(u +v) = Au +Av = 0 +0 = 0
and
A(u) = (Au) = 0 = 0,
so we have shown what we needed. 2
The null space is the set of solutions to the homogeneous linear
system. If we instead consider the set of solutions S to a general
system Ax = b, S is not a subspace of R
n
if b = 0 (that is, if the
system is not homogeneous). This is because 0 does not belong to S.
However, as we saw in the previous chapter (theorem 2.5), there is a
relationship between S and N(A): if x
0
is any solution of Ax = b then
S = {x
0
+z : z N(A)}, which we may write as x
0
+N(A). S is an
ane set, a translation of the subspace N(A).
Generally, if W is a subspace of a vector space V and x V then the
set x +W dened by
x +W = {x +w : w W}
is called an ane subset of V . An ane subset is not generally a
subspace (although every subspace is an ane subset, as we can see by
taking x = 0).
35
3.4.2 Range
Recall that the range of an mn matrix is
R(A) = {Ax : x R
n
}.
Theorem 3.4 For any mn matrix A, R(A) is a subspace of R
m
.
Proof The set R(A) is non-empty as A0 = 0 R(A). We need to
show that if u, v R(A) then u +v R(A) and, for any R,
v R(A). So suppose u, v R(A). Then for some y
1
, y
2
R
n
,
u = Ay
1
, v = Ay
2
. We need to show that u +v = Ay for some y.
Well,
u +v = Ay
1
+Ay
2
= A(y
1
+y
2
),
so we may take y = y
1
+y
2
to see that, indeed, u +v R(A). Next,
v = (Ay
1
) = A(y
1
),
so v = Ay for some y (namely y = y
1
) and hence v R(A). 2
3.5 Linear span
Recall that by a linear combination of vectors v
1
, v
2
, . . . , v
k
we mean a
vector of the form
v =
1
v
1
+
2
v
2
+ +
k
v
k
.
If we add together two vectors of this form, we get another linear
combination of the vectors v
1
, v
2
, . . . , v
k
. The same is true of any
scalar multiple of v.
Show this; show that if v =
1
v
1
+
2
v
2
+ +
k
v
k
and
w =
1
v
1
+
2
v
2
+ +
k
v
k
then v +w and sv, s R, are also
linear combinations of the vectors v
1
, v
2
, . . . , v
k
.
The set of all linear combinations of a given set of vectors of a vector
space V forms a subspace, and we give it a special name.
Denition 3.3 (Linear span) Suppose that V is a vector space and
that v
1
, v
2
, . . . , v
k
V . The linear span of X = {v
1
, . . . , v
k
} is the
set of all linear combinations of the vectors v
1
, . . . , v
k
, denoted by
Lin{v
1
, v
2
, . . . , v
k
} or Lin(X) . That is,
Lin{v
1
, v
2
, . . . , v
k
} = {
1
v
1
+ +
k
v
k
:
1
,
2
, . . . ,
k
R}.
36
Linear span
Theorem 3.5 If X = {v
1
, . . . , v
k
} is a set of vectors of a vector space
V , then Lin(X) is a subspace of V . It is the smallest subspace
containing the vectors v
1
, v
2
, . . . , v
k
.
If you have carefully carried out the learning activity 3.8 above, then
you have shown that Lin(X) is a subspace of V (why?). Any vector
space which contains the vectors v
1
, v
2
, . . . , v
k
must also contain all
linear combinations of these vectors, so it must contain Lin(X). That
is, Lin(X) is the smallest subspace of V containing v
1
, v
2
, . . . , v
k
.
This subspace is also known as the subspace spanned by the set
X = {v
1
, . . . , v
k
}, or, simply, as the span, of {v
1
, v
2
, . . . , v
k
}.
Dierent texts use dierent notations. For example, Anton uses
span{v
1
, v
2
, . . . , v
k
} and Simon and Blume use L[v
1
, v
2
, . . . , v
n
].
Notation is important, but it is nothing to get anxious about: just
always make it clear what you mean by your notation: use words as
well as symbols!
3.5.1 Lines and Planes in R
3
In R
3
a plane is dened as the set of all vectors x whose components
satisfy a single Cartesian equation, ax +by +cz = d.
1
If d = 0, the
1
See, for example, Anton.
plane contains the origin, and the position vectors of points on the
plane form a subspace of R
3
. This subspace is the linear span of two
vectors. To see this, lets look at a specic example.
Example: Let S be the set given by
S =
_
_
_
_
_
x
y
z
_
_
: 3x 2y +z = 0
_
_
_
.
Then for x S,
x =
_
_
x
y
z
_
_
=
_
_
x
y
2y 3x
_
_
=
_
_
x
0
3x
_
_
+
_
_
0
y
2y
_
_
= x
_
_
1
0
3
_
_
+y
_
_
0
1
2
_
_
.
That is, x = xv
1
+yv
2
where x, y can be any real numbers and v
1
, v
2
are the vectors given above. Since S is the linear span of two vectors,
it is a subspace of R
3
. Of course, you can show directly that S is a
subspace by showing it is closed under addition and scalar
multiplication.
If d = 0 then the plane is not a subspace. It is an ane set, a
translation of a linear space.
Show this in general, that the set
S =
_
_
_
_
_
x
y
z
_
_
: ax +by +cz = d
_
_
_
37
is a subspace if d = 0 and it is not a subspace if d = 0.
Conversely, if v
1
, v
2
R
3
are two non-parallel vectors, then the set
K = Lin{v
1
, v
2
} is a plane, and we can obtain its Cartesian equation.
Consider the previous example.
Example: The plane is the set of all linear combinations of v
1
and v
2
,
that is, all vectors x such that
x =
_
_
x
y
z
_
_
= s
_
_
1
0
3
_
_
+t
_
_
0
1
2
_
_
s, t R.
This yields three equations in the two unknowns, s and t. Eliminating s
and t from these equations yields a single Cartesian equation between
the variables x, y, z:
x = s
y = t
z = 3s + 2t
_
_
_
= z = 3x + 2y or 3x 2y +z = 0.
What is the set Lin{v} of a single non-zero vector v? We have already
seen that this denes a line through the origin in R
3
. (See Learning
activity 3.6 in section 3.3.)
3.5.2 Row space and column space
In the previous chapter we observed that the range R(A) of an mn
matrix A is equal to the set of all linear combinations of its columns.
(See 2.3.5.) In other words, R(A) is the span of the columns of A and
is often called the column space of A and denoted by CS(A).
It is also possible to consider the row space RS(A) of a matrix: this is
the span of the rows of A. If A is an mn matrix the row space will
be a subspace of R
n
and the column space will be a subspace of R
m
.
3.6 Linear independence
Linear independence is a central idea in the theory of vector spaces. If
{v
1
, v
2
, . . . , v
k
} is a set of vectors in a vector space V , then the vector
equation
1
v
1
+
2
v
2
+ +
r
v
k
= 0
always has the trivial solution,
1
=
2
= =
k
= 0.
We say that vectors x
1
, x
2
, . . . , x
k
in R
n
are linearly dependent (LD) if
there are numbers
1
,
2
, . . . ,
m
, not all zero, such that
1
x
1
+
2
x
2
+ +
k
x
k
= 0.
38
Linear independence
In this case the left-hand side is termed a non-trivial linear
combination. The vectors are linearly independent (LI) if they are not
linearly dependent; that is, if no non-trivial linear combination of them
is the zero vector or, equivalently, whenever
1
x
1
+
2
x
2
+ +
k
x
k
= 0,
then, necessarily,
1
=
2
= =
k
= 0. We have been talking
about R
n
, but the same denitions can be used for any vector space V .
We state them formally now.
Denition 3.4 (Linear independence) Let V be a vector space and
v
1
, . . . , v
k
V . Then v
1
, v
2
, . . . , v
k
form a linearly independent set or
are linearly independent if and only if
1
v
1
+
2
v
2
+ +
k
v
k
= 0 =
1
=
2
= =
k
= 0 :
that is, if and only if no non-trivial linear combination of v
1
, v
2
, . . . , v
k
equals the zero vector.
Denition 3.5 (Linear dependence) Let V be a vector space and
v
1
, v
2
, . . . , v
k
V . Then v
1
, v
2
, . . . , v
k
form a linearly dependent set
or are linearly dependent if and only if there are real numbers
1
,
2
, . . . ,
k
, not all zero, such that
1
v
1
+
2
v
2
+ +
k
v
k
= 0,
that is, if and only if some non-trivial linear combination of the vectors
is the zero vector.
Example: In R
3
, the following vectors are linearly dependent:
v
1
=
_
_
1
2
3
_
_
, v
2
=
_
_
2
1
5
_
_
, v
3
=
_
_
4
5
11
_
_
.
This is because
2v
1
+v
2
v
3
= 0.
Note that this can also be written as v
3
= 2v
1
+v
2
.
This example illustrates the following general result. Try to prove it
yourself before looking at the proof.
Theorem 3.6 The set {v
1
, v
2
, . . . , v
k
} V is linearly dependent if
and only if some vector v
i
is a linear combination of the other vectors.
Proof Since this is an if and only if statement, we must prove it
both ways. If {v
1
, v
2
, . . . , v
k
} is linearly dependent, the equation
1
v
1
+
2
v
2
+ +
r
v
k
= 0 has a solution with some
i
= 0. Then
we can solve for the vector v
i
:
v
i
=
i
v
1

2
i
v
2

i1
i
v
i1

i+1
i
v
i+1

k
i
v
k
,
which expresses v
i
as a linear combination of the other vectors in the
set.
39
If v
i
is a linear combination of the other vectors, say,
v
i
=
1
v
1
+ +
i1
v
i1
+
i+1
v
i+1
+ +
k
v
k
, then
1
v
1
+ +
i1
v
i1
v
i
+
i+1
v
i+1
+
k
v
k
= 0
is a non-trivial linear combination of the vectors that is equal to the
zero vector, since the coecient of v
i
is 1 = 0. Therefore, the
vectors are linearly dependent. 2
It follows from this theorem that a set of two vectors is linearly
dependent if and only if one vector is a scalar multiple of the other.
Example: The vectors
v
1
=
_
_
1
2
3
_
_
, v
2
=
_
_
2
1
5
_
_
,
in the example above are linearly independent, since one is not a scalar
multiple of the other.
Show that, for any vector v in a vector space V , the set of vectors
{v, 0} is linearly dependent.
3.7 Testing linear independence in R
n
Given k vectors v
1
, . . . , v
k
R
n
, the vector equation
1
v
1
+
2
v
2
+ +
r
v
k
= 0
is a homogeneous system of n linear equations in k unknowns. This
can be written in matrix form, Ax = 0, where A is the n k matrix
A = (v
1
v
2
v
k
) with the vectors v
1
, v
2
, . . . v
k
as its columns, and
x is the vector (or k 1 matrix),
x =
_
_
_
_
2
.
.
.
k
_
_
_
_
.
Recall (learning activity, page 22) that the matrix product Ax is
exactly the linear combination
1
v
1
+v
2
+ +
k
v
k
.
Then the question of whether or not a set of vectors in R
n
is linearly
independent can be answered by looking at the solutions of the
homogeneous system Ax = 0.
Theorem 3.7 The vectors v
1
, v
2
, . . . , v
k
are linearly dependent if and
only if the linear system Ax = 0 has a solution other than x = 0,
where A is the matrix A = (v
1
v
2
v
k
). Equivalently, the vectors
are linearly independent precisely when the only solution to the system
is x = 0.
40
Testing linear independence in R
n
If the vectors are linearly dependent, then any solution x = 0 of the
system Ax = 0 will directly give a linear combination of the vectors
that equals the zero vector.
Show that the vectors
v
1
=
_
1
2
_
, v
2
=
_
1
1
_
, v
3
=
_
2
5
_
are linearly dependent by solving Ax = 0. Use your solution to give a
non-trivial linear combination of the vectors that equals the zero vector.
Now, we know from our experience of solving linear systems with row
operations that the system Ax = 0 will have precisely the one solution
x = 0 if and only if we obtain from the n k matrix A an echelon
matrix in which there are k leading ones. That is, if and only if
rank(A) = k. (Think about this!) Thus, we have the following result.
Theorem 3.8 Suppose that v
1
, . . . , v
k
R
n
. Then the set
{v
1
, . . . , v
k
} is linearly independent if and only if the matrix
(v
1
v
2
v
k
) has rank k.
But the rank is always at most the number of rows, so we certainly need
to have k n. Also, there is a set of n linearly independent vectors in
R
n
. In fact, there are innitely many such sets, but an obvious one is
{e
1
, e
2
, . . . , e
n
} ,
where e
i
is the vector with every entry equal to 0 except for the ith
entry, which is 1. Thus, we have the following result.
Theorem 3.9 The maximum size of a linearly independent set of
vectors in R
n
is n.
So any set of more than n vectors in R
n
is linearly dependent. On the
other hand, it should not be imagined that any set of n or fewer is
linearly independent: that isnt true.
Example: In R
4
, which of the following sets of vectors are linearly
independent?
L
1
=
_
_
_
_
_
1
0
1
0
_
_
_,
_
_
_
1
2
9
2
_
_
_,
_
_
_
2
1
3
1
_
_
_,
_
_
_
0
0
1
0
_
_
_,
_
_
_
2
5
9
1
_
_
_
_
_
,
L
2
=
_
_
_
_
_
1
0
1
0
_
_
_,
_
_
_
1
2
9
2
_
_
_
_
_
,
41
L
3
=
_
_
_
_
_
1
0
1
0
_
_
_,
_
_
_
1
2
9
2
_
_
_,
_
_
_
2
1
3
1
_
_
_
_
_
,
L
4
=
_
_
_
_
_
1
0
1
0
_
_
_,
_
_
_
1
2
9
2
_
_
_,
_
_
_
2
1
3
1
_
_
_,
_
_
_
0
0
1
0
_
_
_
_
_
.
Try this yourself before reading the answers.
The set L
1
is linearly dependent because it consists of ve vectors in
R
4
. The set L
2
is linearly independent because neither vector is a
scalar multiple of the other. To see that the set L
3
is linearly
dependent, write the vectors as the columns of a matrix A and reduce
A to echelon form to nd that the rank of A is 2. This means that
there is a non-trivial linear combination of the vectors which is equal to
0, or equivalently, that one of the vectors is a linear combination of the
other two. The last set, L
4
contains the set L
3
and is therefore also
linearly dependent, since it is still true that one of the vectors is a linear
combination of the others.
For the set L
3
above, nd the solution of the corresponding
homogeneous system Ax = 0 where A is the matrix whose columns are
the vectors of L
3
. Use the solution to write down a non-trivial linear
combination of the vectors that is equal to the zero vector. Express one
of the vectors as a linear combination of the other two.
There is an important property of linearly independent sets of vectors
which holds for any vector space V .
Theorem 3.10 If x
1
, x
2
, . . . , x
m
are linearly independent in V and
c
1
x
1
+c
2
x
2
+ +c
m
x
m
= c
1
x
1
+c
2
x
2
+ +c
m
x
m
then
c
1
= c
1
, c
2
= c
2
, . . . , c
m
= c
m
.
Prove this. Use the fact that
c
1
x
1
+c
2
x
2
+ +c
m
x
m
= c
1
x
1
+c
2
x
2
+ +c
m
x
m
if and only if
(c
1
c
1
)x
1
+ (c
2
c
2
)x
2
+ + (c
m
c
m
)x
m
= 0.
42
Bases and dimension
What does this theorem say about x = c
1
x
1
+c
2
x
2
+ +c
m
x
m
?
(Think about this before you continue reading.)
It says that if a vector x can be expressed as a linear combination of a
set of linearly independent vectors, then this can be done in only one
way. The linear combination is unique.
3.8 Bases and dimension
The following result about R
n
is very important in the theory of vector
spaces. It says that a linearly independent set of n vectors in R
n
spans
R
n
.
Theorem 3.11 If x
1
, x
2
, . . . , x
n
are linearly independent vectors in
R
n
, then for any x in R
n
, x can be written as a unique linear
combination of x
1
, . . . , x
n
.
Proof Because x
1
, . . . , x
n
are linearly independent, the n n matrix
A = (x
1
x
2
. . . x
n
)
has rank(A) = n. In other words, A reduces to an echelon matrix with
exactly n leading ones. Suppose now that x is any vector in R
n
and
consider the system Az = x. By Theorem 2.4 this system has a unique
solution. But lets spell it out. Because A has rank n, it can be
reduced (by row operations) to an echelon matrix with n leading ones,
one in each row and one in each column. The augmented matrix (Ax)
can therefore be reduced to a matrix (Es) where E is an n n echelon
matrix with n leading ones. Solving by back-substitution, or continuing
to reduced echelon form, we can nd a solution to Az = x. This shows
that any vector x can be expressed in the form
x = Az = (x
1
x
2
. . . x
n
)
_
_
_
_
2
.
.
.
n
_
_
_
_
,
where we have written z as (
1
,
2
, . . . ,
n
)
T
. Expanding this matrix
product, we have that any x R
n
can be expressed as a linear
combination
x =
1
x
1
+
2
x
2
+ +
n
x
n
,
as required. This linear combination is unique since the vectors are
linearly independent (or, because there is a leading one in every column
of the echelon matrix, so there are no free variables). 2
It follows from this theorem that if we have n linearly independent
vectors in R
n
, then any vector in R
n
can be expressed in exactly one
way as a linear combination of the n vectors. We say that the n
vectors form a basis of R
n
. The formal denition of a (nite) basis for
a vector space is as follows.
Denition 3.6 ((Finite) Basis) Let V be a vector space. Then the
subset B = {v
1
, v
2
, . . . , v
n
} of V is said to be a basis for (or of) V if:
43
B is a linearly independent set of vectors, and
V = Lin(B).
An alternative characterisation of a basis can be given: B is a basis of
V if every vector in V can be expressed in exactly one way as a linear
combination of the vectors in B. The set B spans V if and only if a
linear combination exists, and B is linearly independent if and only if
any linear combination is unique. We have therefore shown
Theorem 3.12 B = {v
1
, v
2
, . . . , v
n
} is a basis of V if and only if any
v V is a unique linear combination of v
1
, v
2
, . . . , v
n
.
Example: The vector space R
n
has the basis {e
1
, e
2
, . . . , e
n
} where e
i
is (as earlier) the vector with every entry equal to 0 except for the ith
entry, which is 1. Its clear that the vectors are linearly independent,
and there are n of them, so we know straight away that they form a
basis. In fact, its easy to see that they span the whole of R
n
, since for
any x = (x
1
, x
2
, . . . , x
n
)
T
R
n
,
x = x
1
e
1
+x
2
e
2
+ +x
n
e
n
.
The basis {e
1
, e
2
, . . . , e
n
} is called the standard basis of R
n
.
Prove that the vectors in the standard basis of R
n
are linearly
independent and that they span the whole of R
n
.
Example: Find a basis of the subspace of R
3
given by
W =
_
_
_
_
_
x
y
z
_
_
: x +y 3z = 0
_
_
_
.
Solution. If x = (x, y, z)
T
is any vector in W, then its components
must satisfy y = x + 3z, and we can express x as
x =
_
_
x
y
z
_
_
=
_
_
x
x + 3z
z
_
_
= x
_
_
1
1
0
_
_
+z
_
_
0
3
1
_
_
= xv+zw x, z R.
This shows that the set {v, w} spans W. The set is linearly
independent. Why? Because of the positions of the zeros and ones, if
v +w = 0 then necessarily = 0 and = 0.
Example: The set
S =
__
1
2
_
,
_
1
1
__
is a basis of R
2
. To show this we have to show it spans R
2
and is
linearly independent, or equivalently, that any vector b R
2
is a
44
Bases and dimension
unique linear combination of these two vectors. Writing the vectors as
the columns of a matrix A, we nd that |A| = 0, so this is true.
(Theorem 2.4, page 16.)
Which of these sets is a basis of R
3
?
U =
_
_
_
_
_
1
0
1
_
_
,
_
_
1
2
3
_
_
,
_
_
1
2
5
_
_
_
_
_
, W =
_
_
_
_
_
1
0
1
_
_
,
_
_
1
2
3
_
_
,
_
_
1
2
5
_
_
_
_
_
.
Show that one of these sets is a basis of R
3
and that one of these sets
spans a plane in R
3
. Find a basis for this plane. Then nd a Cartesian
equation for the plane.
3.8.1 Coordinates
What is the importance of a basis? If S = {v
1
, v
2
, . . . , v
n
} is a basis
of R
n
, then any vector v R
n
can be expressed uniquely as
v =
1
v
1
+
2
v
2
+ +
n
v
n
. The real numbers
1
,
2
, ,
n
are
the coordinates of v with respect to the basis, S.
Denition 3.7 (Coordinates) If S = {v
1
, v
2
, . . . , v
n
} is a basis of a
vector space V and v =
1
v
1
+
2
v
2
+ +
n
v
n
, then the real
numbers
1
,
2
, ,
n
are the coordinates of v with respect to the
basis, S. We use the notation
[v]
S
=
_
2
.
.
.
n
_
_
S
to denote the coordinate vector of v in the basis S.
Example: The sets B = {e
1
, e
2
} and S = {v
1
, v
2
} where
B =
__
1
0
_
,
_
0
1
__
and S =
__
1
2
_
,
_
1
1
__
are each a basis of R
2
. The coordinates of the vector v = (2, 5)
T
in
each basis are given by the coordinate vectors,
[v]
B
=
_
2
5
_
B
and [v]
S
=
_
1
3
_
S
.
In the standard basis, the coordinates of v are precisely the
components of the vector v. In the basis S, the components of v are
given by writing
v = 1
_
1
2
_
+ 3
_
1
1
_
=
_
2
5
_
.
45
For the example above, sketch the vector v on graph paper and show it
as the sum of the vectors given by each of the linear combinations:
v = 2e
1
5e
2
and v = 1v
1
+ 3v
2
.
3.8.2 Dimensions
A fundamental result is that if a vector space V has a nite basis, then
all bases of V are of the same size.
2 2
Theorem 3.13 Suppose that the vector space V has a nite basis
consisting of d vectors. Then any basis of V consists of exactly d
vectors.
This enables us, nally, to dene exactly what we mean by the
dimension of a vector space V .
Denition 3.8 (Dimension) The number d of vectors in a nite basis
of a vector space V is the dimension of V , and is denoted dim(V ).
The vector space V = {0} is dened to have dimension 0.
A vector space which has a nite basis is said to be nite-dimensional.
Not all vector spaces are nite-dimensional. (For example, the vector
space of real functions with pointwise addition and scalar multiplication
has no nite basis. Such a vector space is said to be innite
dimensional.)
Example: We already know R
n
has a basis of size n. (For example,
the standard basis consists of n vectors.) So R
n
has dimension n
(which is reassuring, since it is often referred to as n-dimensional
Euclidean space).
If we know the dimension of a vector space V , then we know how
many vectors we need for a basis. If we have the correct number of
vectors for a basis and we know either that the vectors span V , or that
they are linearly independent, then we can conclude that both must be
true and they form a basis, as is shown in the following theorem. That
is, we do not need to show both.
3 3
Theorem 3.14 Let V be a nite-dimensional vector space of
dimension d. Then:
d is the largest size of a linearly independent set of vectors in V .
Furthermore, any set of d linearly independent vectors is necessarily
a basis of V ;
46
Bases and dimension
d is the smallest size of a spanning set of vectors for V .
Furthermore, any nite set of d vectors that spans V is necessarily
a basis.
Thus, d = dim(V ) is the largest possible size of a linearly independent
set of vectors in V , and the smallest possible size of a spanning set of
vectors (a set of vectors whose linear span is V ).
Example: We know that the plane W in R
3
,
W =
_
_
_
_
_
x
y
z
_
_
: x +y 3z = 0
_
_
_
has dimension 2, because we found a basis for it consisting of two
vectors. If we choose any set of 2 linearly independent vectors in W,
then that set will be a basis of W. For example, the vectors
v
1
= (1, 2, 1)
T
and v
2
= (3, 0, 1)
T
are linearly independent (why?),
so by the theorem, S = {v
1
, v
2
} is a basis of W.
3.8.3 Dimension and bases of subspaces
Suppose that W is a subspace of the nite-dimensional vector space V .
Any set of linearly independent vectors in W is also a linearly
independent set in V .
Prove this last statement.
Now, the dimension of W is the largest size of a linearly independent
set of vectors in W, so there is a set of dim(W) linearly independent
vectors in V . But then this means that dim(W) dim(V ), since the
largest possible size of a linearly independent set in V is dim(V ). There
is another important relationship between bases of W and V : this is
that any basis of W can be extended to one of V . The following result
states this precisely.
4 4
Theorem 3.15 Suppose that V is a nite-dimensional vector space
and that W is a subspace of V . Then dim(W) dim(V ).
Furthermore, if {w
1
, w
2
, . . . , w
r
} is a basis of W then there are
s = dim(V ) dim(W) vectors v
1
, v
2
, . . . , v
s
V such that
{w
1
, w
2
, . . . , w
r
, v
1
, v
2
, . . . , v
s
} is a basis of V . (In the case W = V ,
the basis of W is already a basis of V .) That is, we can obtain a basis
of the whole space V by adding vectors of V to any basis of W.
Example: The plane W in R
3
, W = {x : x +y 3z = 0} has a
basis consisting of the vectors v
1
= (1, 2, 1)
T
and v
2
= (3, 0, 1)
T
. If
v
3
is any vector which is not in this plane, for example,
v
3
= (1, 0, 0)
T
, then the set S = {v
1
, v
2
, v
3
} is a basis of R
3
.
47
3.9 Finding a basis for a linear span in R
n
Suppose we are given k vectors x
1
, x
2
. . . , x
k
in R
n
, and we want to
nd a basis for the linear span Lin{x
1
, . . . , x
k
}. The point is that the k
vectors themselves might not form a linearly independent set (and
hence not a basis).
A useful technique is to form a matrix with the x
T
i
as rows, and to
perform row operations until the resulting matrix is in echelon form.
Then a basis of the linear span is given by the transposed non-zero
rows of the echelon matrix (which, it should be noted, will not
generally be among the initial given vectors). The reason this works is
that: (i) row operations are such that at any stage in the resulting
procedure, the row space of the matrix is equal to the row space of the
original matrix, which is precisely the linear span of the original set of
vectors, and (ii) the non-zero rows of an echelon matrix are linearly
independent (which is clear, since each has a one in a position where
the vectors below it all have zero).
Example: We nd a basis for the subspace of R
5
spanned by the
vectors
x
1
=
_
_
_
_
_
1
1
2
1
1
_
_
_
_
_
, x
2
=
_
_
_
_
_
2
1
2
2
2
_
_
_
_
_
, x
3
=
_
_
_
_
_
1
2
4
1
1
_
_
_
_
_
, x
4
=
_
_
_
_
_
3
0
0
3
3
_
_
_
_
_
.
Write the vectors as the columns of a matrix A and then take A
T
(eectively writing each column vector as a row).
A
T
=
_
_
_
1 1 2 1
2 1 2 2
1 2 4 1
3 0 0 3
1
2
1
3
_
_
_.
Reducing this to echelon form by elementary row operations,
_
_
_
1 1 2 1
2 1 2 2
1 2 4 1
3 0 0 3
1
2
1
3
_
_
_
_
_
_
1 1 2 1
0 3 6 0
0 1 2 0
0 3 6 0
1
0
0
0
_
_
_
_
_
_
1 1 2 1
0 1 2 0
0 1 2 0
0 1 2 0
1
0
0
0
_
_
_
_
_
_
1 1 2 1
0 1 2 0
0 0 0 0
0 0 0 0
1
0
0
0
_
_
_.
The echelon matrix at the end of this tells us that a basis for
Lin{x
1
, x
2
, x
3
, x
4
} is formed from the rst two rows, transposed, of
the echelon matrix, that is,
_
_
_
_
_
_
_
1
1
2
1
1
_
_
_
_
_
,
_
_
_
_
_
0
1
2
0
0
_
_
_
_
_
_
_
.
48
Basis and dimension of range and null space
If we want to nd a basis that consists of a subset of the original
vectors, then we need to take those vectors that correspond to the
nal non-zero rows in the echelon matrix. By this, we mean the rows of
the original matrix that have ended up as non-zero rows in the echelon
matrix. For instance, in the example just given, the rst and second
rows of the original matrix correspond to the non-zero rows of the
echelon matrix, so a basis of the span is {x
1
, x
2
}. On the other hand,
if we interchange rows, the correspondence wont be so obvious.
A better method to obtain such a basis is given in the next section,
using the matrix A whose columns are the vectors x
1
, x
2
. . . , x
k
.
Then, as we have seen, Lin{x
1
, . . . , x
k
} = R(A). That is,
Lin{x
1
, . . . , x
k
} is the range or column space of the matrix A.
3.10 Basis and dimension of range and null space
We have shown that the range and null space of an mn matrix are
subspaces of R
m
and R
n
respectively (section 3.4). Their dimensions
are so important that they are given special names.
Denition 3.9 (Rank and nullity) The rank of a matrix A is
rank(A) = dim(R(A))
and the nullity is
nullity(A) = dim(N(A)).
We have, of course, already used the word rank, so it had better be
the case that the usage just given coincides with the earlier one.
Fortunately it does. In fact, we have the following connection.
Theorem 3.16 Suppose that A is an mn matrix with columns
c
1
, c
2
, . . . , c
n
, and that an echelon form obtained from A has leading
ones in columns i
1
, i
2
, . . . , i
r
. Then a basis for R(A) is
B = {c
i
1
, c
i
2
, . . . , c
i
r
}.
Note that the basis is formed from columns of A, not columns of the
echelon matrix: the basis consists of those columns of A corresponding
to the leading ones in the echelon matrix.
We will outline a proof of this theorem, so you can see how it works.
5 5
see, also, Anton.
We have already seen that a solution x = (
1
,
2
, . . . ,
n
) of Ax = 0
gives a linear combination of the columns of A which is equal to the
zero vector,
0 =
1
c
1
+
2
c
2
+. . . +
n
c
n
.
If E denotes the reduced echelon form of A, and if c
1
, c
2
, . . . , c
n
denote the columns of E, then exactly the same relationship holds:
0 =
1
c
1
+
2
c
2
+. . . +
n
c
n
.
49
In fact, we use E to obtain the solution x = (
1
,
2
, . . . ,
n
). So the
linear dependence relations are the same for the columns of both
matrices, which means that the linearly independent columns of A
correspond precisely to the linearly independent columns of E. Which
columns of E are linearly independent? The columns which contain the
leading ones.
If E is a matrix in reduced echelon form, prove that the set of columns
with the leading ones is a basis of the column space of E.
We have already seen that a matrix A and its reduced echelon form
have the same row space, and that the non-zero rows form a basis of
this row space. So the dimension of the row space of A, RS(A), and
the dimension of the column space of A, R(A), are each equal to the
number of leading ones in an echelon form of A, that is both are equal
to rank(A).
Example: Let A be the matrix,
A =
_
_
1 1 2 1
2 0 1 1
9 1 3 4
_
_
.
The reduced echelon form of the matrix is (verify this!)
E =
_
_
1 0
1
2
1
2
0 1
3
2
1
2
0 0 0 0
_
_
.
The leading ones in this echelon matrix are in the rst and second
columns, so a basis for R(A) can be obtained by taking the rst and
second columns of A. (Note: columns of A, not of the echelon
matrix!) Therefore a basis for R(A) is
_
_
_
_
_
1
2
9
_
_
,
_
_
1
0
1
_
_
_
_
_
.
A basis of the row space of A consists of the two non-zero rows of the
reduced matrix, or the rst two rows of the original matrix,
_
_
_
_
_
_
1
0
1
2
1
2
_
_
_
_
,
_
_
_
_
0
1
3
2
1
2
_
_
_
_
_
_
or
_
_
_
_
_
_
1
1
2
1
_
_
_
_
,
_
_
_
_
2
0
1
1
_
_
_
_
_
_
.
Note that the column space is a 2-dimensional subspace of R
3
(a
plane) and the row space is a 2-dimensional subspace of R
4
. The
columns of A and E satisfy the same linear dependence relations,
which can be easily read from the reduced echelon form of the matrix,
c
3
=
1
2
c
1
+
3
2
c
2
, c
4
=
1
2
c
1
+
1
2
c
2
.
50
Basis and dimension of range and null space
Check that the columns of A satisfy these same linear dependence
relations.
There is a very important relationship between the rank and nullity of a
matrix. We have already seen some indication of it in our
considerations of linear systems. Recall that if an mn matrix A has
rank r then the general solution to the (consistent) system Ax = 0
involves n r free parameters. Specically (noting that 0 is a
particular solution, and using a characterisation obtained earlier in the
previous chapter), the general solution takes the form
x = s
1
u
1
+s
2
u
2
+ +s
nr
u
nr
,
where u
1
, u
2
, . . . , u
nr
are themselves solutions of the system Ax = 0.
But the set of solutions of Ax = 0 is precisely the null space N(A).
Thus, the null space is spanned by the n r vectors u
1
, . . . , u
nr
, and
so its dimension is at most n r. In fact, it turns out that its
dimension is precisely n r. That is,
nullity(A) = n rank(A).
To see this, we need to show that the vectors u
1
, . . . , u
nr
are linearly
independent. Because of the way in which these vectors arise (look at
the example we worked through), it will be the case that for each of
them, there is some position where that vector will have an entry equal
to 1 and the entry in that same position of all the other vectors will be
0. From this we can see that no non-trivial linear combination of them
can be the zero vector, so they are linearly independent. We have
therefore proved the following central result.
Theorem 3.17 (Rank-nullity theorem) For an mn matrix A,
rank(A) + nullity(A) = n.
Find a basis of the null space of the matrix
A =
_
_
1 1 2 1
2 0 1 1
9 1 3 4
_
_
from the previous example. Verify the rank-nullity theorem for this
matrix.
51
52
3.11 Learning outcomes
At the end of this chapter and the relevant reading, you should be able
to:
explain what is meant by a vector space and a subspace
prove that a given set is a vector space, or a subspace of a given
vector space
explain what is meant by the linear span of a set of vectors
explain what is meant by linear independence and linear
dependence
determine whether a given set of vectors is linearly independent or
linearly dependent, and in the latter case, nd a non-trivial linear
combination of the vectors which equals the zero vector
explain what is meant by a basis, and by the dimension of a
nite-dimensional vector space
explain what is meant by coordinates with respect to a basis and be able
to use this definition
nd a basis for a linear span
nd a basis for the null space, range and row space of a matrix
from its reduced row echelon form
explain how rank and nullity are dened, and the relationship
between them (the rank-nullity theorem).
3.12 Sample examination questions
The following are typical exam questions, or parts of questions.
Question 3.1 Let
S =
x
1
x
2
: x
2
= 3x
1
.
Prove that S is a subspace of R
2
(where the operations of addition and
scalar multiplication are the usual ones).
Question 3.2 Let V be the vector space of all functions from R R
with pointwise addition and scalar multiplication. Let n be a xed
positive integer and let W be the set of all real polynomial functions of
degree at most n; that is, W consists of all functions of the form
f(x) = a
0
+a
1
x +a
2
x
2
+ +a
n
x
n
, where a
0
, a
1
, . . . , a
n
R.
Prove that W is a subspace of V , under the usual pointwise addition
and scalar multiplication for real functions. Show that W is
nite-dimensional by presenting a nite set of functions which spans W.
Sample examination questions
Question 3.3 Show that the set S
1
spans R
3
, but any vector v R
3
can be written as a linear combination of the vectors in S
1
in innitely
many ways. Show that S
2
and S
3
do not span R
3
.
S
1
=
_
_
_
_
_
1
2
3
_
_
,
_
_
1
0
1
_
_
,
_
_
0
1
1
_
_
,
_
_
1
1
0
_
_
_
_
_
, S
2
=
_
_
_
_
_
1
0
1
_
_
,
_
_
2
1
3
_
_
,
_
_
1
2
9
_
_
_
_
_
,
S
3
=
_
_
_
_
_
1
1
1
_
_
,
_
_
2
0
1
_
_
_
_
_
.
Question 3.4 Show that the following three vectors are linearly
independent:
_
_
2
1
1
_
_
,
_
_
3
4
6
_
_
,
_
_
2
3
2
_
_
.
Express the vector v =
_
_
4
7
3
_
_
as a linear combination of these three
vectors.
Question 3.5 Let
x
1
=
_
_
2
3
5
_
_
, x
2
=
_
_
1
1
2
_
_
, v =
_
_
a
b
c
_
_
.
Find a vector x
3
such that {x
1
, x
2
, x
3
} is a linearly independent set of
vectors.
Find a condition that a, b, c must satisfy for the set of vectors
{x
1
, x
2
, v} to be linearly dependent.
Question 3.6 Show that the following vectors are linearly dependent
by nding a non-trivial linear combination of the vectors that equals
the zero vector.
_
_
_
1
2
1
2
_
_
_,
_
_
_
0
1
3
4
_
_
_,
_
_
_
4
11
5
1
_
_
_,
_
_
_
9
2
1
3
_
_
_.
Question 3.7 Let A be any matrix. Let v
1
and v
2
be two non-zero
vectors and suppose that Av
1
= 2v
1
and Av
2
= 5v
2
. Prove that
{v
1
, v
2
} is linearly independent.
(Hint: Assume
1
v
1
+
2
v
2
= 0; multiply this equation through by A
to get a second equation for v
1
and v
2
; then solve the two equations
simultaneously.)
Can you generalize this result?
Question 3.8 Let B be the basis {v
1
, v
2
, v
3
}, where v
1
= (0, 1, 0)
T
,
v
2
= (4/5, 0, 3/5)
T
, v
3
= (3/5, 0, 4/5)
T
, and let
x = (1, 1/5, 7/5)
T
. Find the coordinates of x with respect to B.
53
Question 3.9 For each of the sets S
i
of vectors given below, nd a
basis of the vector space Lin(S
i
) and state its dimension.
S
1
=
__
1
2
_
,
_
2
3
__
S
2
=
__
1
1
_
,
_
0
0
_
,
_
2
2
_
,
_
3
3
__
S
3
=
_
_
_
_
_
1
0
1
_
_
,
_
_
2
1
3
_
_
,
_
_
1
2
9
_
_
_
_
_
S
4
=
_
_
_
_
_
1
2
1
3
_
_
_,
_
_
_
2
0
1
2
_
_
_,
_
_
_
4
4
1
8
_
_
_,
_
_
_
1
1
1
2
_
_
_
_
_
Question 3.10 Which of the following sets are a basis for R
3
? (State
reasons for your answers.)
S
1
=
_
_
_
_
_
1
2
3
_
_
,
_
_
2
1
0
_
_
,
_
_
4
1
0
_
_
,
_
_
7
2
1
_
_
_
_
_
S
2
=
_
_
_
_
_
1
0
1
_
_
,
_
_
1
1
1
_
_
_
_
_
S
3
=
_
_
_
_
_
2
1
1
_
_
,
_
_
1
2
1
_
_
,
_
_
3
3
0
_
_
_
_
_
S
4
=
_
_
_
_
_
1
1
1
_
_
,
_
_
1
1
0
_
_
,
_
_
2
3
3
_
_
_
_
_
For any set which is a basis of R
3
, nd the coordinates of the vector
w = (1, 2, 1) in that basis.
Question 3.11 Write down a basis for the yz plane in R
3
.
Question 3.12 Find a basis for the null space of the matrix
_
1 1 1 0
2 1 0 1
_
.
Question 3.13 Let
A =
_
_
_
1 2 1 1 2
1 3 0 2 2
0 1 1 3 4
1 2 5 13 5
_
_
_.
Find a basis for the column space of A.
Question 3.14 Find a basis of the row space, a basis of the range
(column space), and a basis of the null space of the matrix
B =
_
_
1 2 1 3 0
0 1 1 1 1
1 3 2 0 1
_
_
.
Find the rank of B and verify the rank-nullity theorem.
Let b = c
1
+c
5
, the sum of the rst and last column of the matrix B.
Without solving the system, use the information you have obtained to
write down a general solution of the system of equations Bx = b.
3.13 Answers to or comments on selected
activities
Activity 3.1 For any x,
0 = 0x = (1 + (1))x = 1x + (1)x = x + (1)x
54
so adding the negative x of x to each side, and using properties 3
and 4 of the denition of a vector space,
x = x +0 = x +x + (1)x = (1)x
which proves that x = (1)x.
Activity 3.2 The negative of a function f is the function f given by
(f)(x) = (f(x)) for all x.
Activity 3.3 Suppose
u =
_
_
x
y
0
_
_
, v =
_
_
x
0
_
_
V,
and that R. Then
u +v =
_
_
x +x
y +y
0
_
_
V
and
v =
_
_
x
y
0
_
_
V.
Activity 3.5 0 is not in the set U as
0 =
_
0
0
_
=
_
0
1
_
+t
_
1
2
_
for any t R,
so property 4 of the denition of a vector space is not satised
(denition 3.1).
Activity 3.6 Note rst that S is non-empty because 0 S. Suppose
that x, y S. (Why are we not using the usual symbols u and v? Can
you see? Its because v is already used in the denition of the set S.)
Suppose also that R. Now, because x and y belong to S, there
are ,
R such that x = v and y =
v. Then,
x +y = v +
v = ( +
)v,
which is in S since it is a scalar multiple of v. Also,
x = (v) = ()v S
and it follows that S is a subspace.
Activity 3.7 A general vector in S
1
is of the form
_
_
x
x
2
0
_
_
, x R, and
one particular vector, taking x = 1, is
_
_
1
1
0
_
_
. To show S
1
is not a
subspace you need to nd one counterexample, one or two particular
vectors in S
1
which do not satisfy the closure property, such as
_
_
1
1
0
_
_
S
1
but
_
_
1
1
0
_
_
+
_
_
1
1
0
_
_
=
_
_
2
2
0
_
_
/ S
1
.
55
A general vector in S
2
is of the form
_
_
x
2x
0
_
_
, x R, and one particular
vector, taking x = 1, is
_
_
1
2
0
_
_
. S
2
is a subspace. We show it is closed
under addition and scalar multiplication using general vectors: if
u, v S
2
, R,
u+v =
_
_
a
2a
0
_
_
+
_
_
b
2b
0
_
_
=
_
_
a +b
2(a +b)
0
_
_
S
2
a, b R a+b R,
u =
_
_
a
2(a)
0
_
_
S
2
a R.
Activity 3.8 Any two such vectors will be of the form
v =
1
v
1
+
2
v
2
+ +
k
v
k
and
v
1
v
1
+
2
v
2
+ +
k
v
k
and we will have
v +v
= (
1
+
1
)v
1
+ (
2
+
2
)v
2
+ + (
k
+
k
)v
k
,
which is a linear combination of the vectors v
1
, v
2
, , v
k
. Also,
v = (
1
v
1
+
2
v
2
+ +
k
v
k
) = (
1
)v
1
+(
2
)v
2
+ +(
k
)v
k
,
is a linear combination of the vectors v
1
, v
2
, , v
k
.
Activity 3.9 Suppose d = 0. Let u, v S and R. Then
u =
_
_
x
y
z
_
_
, v =
_
_
x
_
_
,
where ax +by +cz = 0 and ax
+by
+cz
= 0. Consider u +v. This

equals
_
_
X
Y
Z
_
_
=
_
_
x +x
y +y
z +z
_
_
and we want to show this belongs to S. Now, this is the case, because
aX +bY +cZ = a(x +x
) +b(y +y
) +c(z +z
)
= (ax +by +cz) + (ax
+by
+cz
) = 0 + 0 = 0,
and similarly it can be shown that, for any R, v S. So, in this
case, S is a subspace. You can see why this argument fails when d is
not 0; for, then aX +bY +cZ will equal 2d, which will not be the
same as d. So we will not have u +v S. (Similarly, we could see
that v will not be in S if = 1.)
Activity 3.10 The non-trivial linear combination 0v + 10 is 0.
56
Activity 3.11 Let A =
_
1 1 2
2 1 5
_
. Then, using row operations,
it can be seen that the general solution x to Ax = 0 is
x = (r, 3r, r)
T
for r R. In particular, taking r = 1, and
multiplying out the equation Ax = 0, we have that
_
1
2
_
+ 3
_
1
1
_
_
2
5
_
=
_
0
0
_
.
Activity 3.12 The general solution to the system is
x =
_
_
x
y
z
_
_
=
_
_
3r/2
r/2
r
_
_
,
for r R. Taking r = 1 and multiplying out the equation Ax = 0,
we see that
3
2
_
_
_
1
0
1
0
_
_
_+
1
2
_
_
_
1
2
9
2
_
_
_
_
_
_
2
1
3
1
_
_
_ = 0,
and hence
_
_
_
2
1
3
1
_
_
_ =
3
2
_
_
_
1
0
1
0
_
_
_+
1
2
_
_
_
1
2
9
2
_
_
_.
Activity 3.13 As noted,
c
1
x
1
+c
2
x
2
+ +c
m
x
m
= c
1
x
1
+c
2
x
2
+ +c
m
x
m
if and only if
(c
1
c
1
)x
1
+ (c
2
c
2
)x
2
+ + (c
m
c
m
)x
m
= 0.
But since the vectors are linearly independent, this can be true only if
c
1
c
1
= 0, c
2
c
2
= 0, and so on. That is, for each i, we must have
c
i
= c
i
.
Activity 3.15 Write each set of vectors as the columns of a matrix:
B =
_
_
1 1 1
0 2 2
1 3 5
_
_
, A =
_
_
1 1 1
0 2 2
1 3 5
_
_
.
|A| = 0, so W is a basis of R
3
. |B| = 0, so U is not. Because the set
U is linearly dependent, one of the vectors is a linear combination of
the other two. Also, any two vectors of U are linearly independent
since no one of them is a scalar multiple of any other. Therefore any
two vectors from the set U are a basis of Lin(U). Since Lin(U) is a
two-dimensional subspace of R
3
, it is plane.
Using the rst two vectors in U for the basis, if x U,
x =
_
_
x
y
z
_
_
= s
_
_
1
0
1
_
_
+t
_
_
1
2
3
_
_
, s, t R.
57
Equating components, you obtain three equations in the two unknowns
s and t. Eliminating s and t between the three equations you will
obtain a single equation relating x, y, and z. Explicitly, we have
x = s +t, y = 2t, z = s + 3t,
so
t =
y
2
, s = t x =
y
2
x
and
z = s + 3t =
_
y
2
x
_
+
3
2
y,
so we have x 2y +z = 0. This is the Cartesian equation of the plane.
Note that a Cartesian equation could equally well have been obtained
by writing the two basis vectors and the vector x as the columns of a
matrix M and using the fact that |M| = 0 if and only if the columns of
M are linearly dependent. That is,
1 1 x
0 2 y
1 3 z
= 2x + 4y 2z = 0.
Activity 3.20 A general solution of the system of equations Ax = 0 is
x = s
1
_
_
_
_
1
2
3
2
1
0
_
_
_
_
+s
2
_
_
_
_
1
2
1
2
0
1
_
_
_
_
= s
1
u
1
+s
2
u
2
.
The set {u
1
, u
2
} is a basis of the null space of A, so dim(N(A)) = 2.
From the example, rank(A) = 2. The matrix A has n = 4 columns.
rank(A) + nullity(A) = 2 + 2 = 4 = n.
Note that the basis vectors of the null space give precisely the same
linear dependence relations between the column vectors as those given
in the example. Since Au
1
= 0 and Au
2
= 0,
Au
1
=
1
2
c
1
3
2
c
2
+c
3
= 0 and Au
2
=
1
2
c
1
1
2
c
2
+c
4
= 0.
3.14 Sketch answers to or comments on selected
questions
Question 3.1 You need to show that for any R and u, v S,
u S and u +v S. Both are reasonably straightforward, and the
details are omitted. Also, 0 S, so S = .
Question 3.2 Clearly, W = . We need to show that for any R
and f, g W, f W and f +g W. Suppose that
f(x) = a
0
+a
1
x+a
2
x
2
+ +a
n
x
n
, g(x) = b
0
+b
1
x+b
2
x
2
+ +b
n
x
n
.
Then
(f +g)(x) = f(x) +g(x) = (a
0
+b
0
) +(a
1
+b
1
)x+ +(a
n
+b
n
)x
n
,
58
so f +g is also a polynomial of degree at most n and therefore
f +g W. Similarly,
(f)(x) = (f(x)) = (a
0
) + (a
1
)x + + (a
n
)x
n
,
so f W also. It can be seen that the set of functions
{1, x, x
2
, . . . , x
n
} spans W, where 1 denotes the function that is
identically equal to 1 (that is, the function f with f(x) = 1 for all x).
Question 3.3 To show that
_
_
1
2
3
_
_
+
_
_
1
0
1
_
_
+
_
_
0
1
1
_
_
+
_
_
1
1
0
_
_
=
_
_
x
y
z
_
_
has innitely many solutions for any vector b R
3
, and therefore
spans R
3
, one can solve the system Ax = b:
A =
_
_
1 1 0 1
2 0 1 1
3 1 1 0
_
_
x =
_
_
_
_
_
_ b =
_
_
x
y
z
_
_
.
If the row echelon form of the coecient matrix A has three leading
ones, then there will always be innitely many solutions: a solution
will exist since a leading one in each row means an augmented system
cannot be inconsistent and there will be innitely many solutions since
there is one free (non-leading) variable.
Row reducing A (not all steps are shown),
A =
_
_
1 1 0 1
2 0 1 1
3 1 1 0
_
_

_
_
1 1 0 1
0 2 1 1
0 4 1 3
_
_

_
_
1 1 0 1
0 1
1
2
1
2
0 0 1 1
_
_
The row echelon form of the augmented matrix (A|b) has three
leading ones.
Therefore, the set S
1
spans R
3
, and any vector b R
3
can be written
as a linear combination of the vectors in S
1
in innitely many ways.
S
2
does not span R
3
. Since
|B| =
1 2 1
0 1 2
1 3 9
= 1(9 6) 1(4 1) = 0
the REF (reduced echelon form) of B will contain a row of zeros.
There will be some b R
3
for which the system is inconsistent.
For example, the vector b = ( 0 0 1 )
T
cannot be expressed as a
linear combination of the vectors in S
2
as the system Bx = b has no
solution:
_
_
1 2 1 0
0 1 2 0
1 3 9 1
_
_
_
_
1 2 1 0
0 1 2 0
0 5 10 1
_
_
_
_
1 2 1 0
0 1 2 0
0 0 0 1
_
_
S
3
does not span R
3
. At least three vectors are required to span R
3
.
59
Question 3.4 Call the vectors x
1
, x
2
, x
3
. To show linear
independence, show that the matrix A = (x
1
x
2
x
3
) has rank 3 using
row operations, or show |A| = 0. To express the fourth vector x
4
as a
linear combination of the rst 3, solve Ax = x
4
. You should obtain the
solution x = (217/55, 18/55, 16/11)
T
, so
_
_
4
7
3
_
_
=
217
55
_
_
2
1
1
_
_
18
55
_
_
3
4
6
_
_
+
16
11
_
_
2
3
2
_
_
.
Question 3.5 There are many ways of solving this problem, and there
are innitely many possible x
3
s. We can solve it by trying to nd a
vector x
3
such that the matrix (x
1
x
2
x
3
) has rank 3.
Another approach is to answer the second part of the question rst,
determine what vectors v form a linearly dependent set with x
1
and x
2
(nd the condition on a, b, c as asked) and then write down any
vector whose components do not satisfy the condition.
Write the three vectors as the columns of a matrix and row reduce it,
A =
_
_
1 2 a
1 3 b
2 5 c
_
_
.
Notice that you can choose to order x
1
and x
2
so that the row
reduction will be easier since it makes no dierence in this question.
The vectors {x
1
, x
2
, v} will be linearly dependent if the row echelon
form of A has a row of zeros.
A =
_
_
1 2 a
1 3 b
2 5 c
_
_
R
2
R
1
R
3
2R
1
_
_
1 2 a
0 1 b a
0 1 c 2a
_
_
R
3
R
2
_
_
1 2 a
0 1 b a
0 0 c a b
_
_
The vectors will be linearly dependent if and only if the components of
v satisfy
a +b c = 0
So, choose any vector for x
3
which does not satisfy this equation, such
as x
3
= ( 1 0 0 )
T
.
Note that this condition is the equation of a plane in R
3
determined by
the vectors x
1
and x
2
. The set {x
1
, x
2
, v} is linearly dependent if and
only if v is the position vector of a point in this plane.
Question 3.6 Let
A =
_
_
_
1 0 4 9
2 1 11 2
1 3 5 1
2 4 1 3
_
_
_,
the matrix with columns equal to the given vectors. If we only needed
to show that the vectors were linearly dependent, it would suce to
show, using row operations, that rank(A) < 4. But were asked for
more: we have to nd an explicit non-trivial linear combination that
equals the zero vector. So we need to nd a non-trivial solution of
60
Ax = 0. One solution is x = (5, 3, 1, 1)
T
. (You should use row
operations to nd this. The details are omitted here.) This means that
5
_
_
_
1
2
1
2
_
_
_3
_
_
_
0
1
3
4
_
_
_+
_
_
_
4
11
5
1
_
_
_
_
_
_
9
2
1
3
_
_
_ =
_
_
_
0
0
0
0
_
_
_.
Question 3.7 To prove that {v
1
, v
2
} is linearly independent, assume
that
1
and
2
are scalars such that
()
1
v
1
+
2
v
2
= 0.
Then
A(
1
v
1
+
2
v
2
) = 0
1
Av
1
+
2
Av
2
= 0
1
(2v
1
) +
2
(5v
2
) = 0
2
1
v
1
+ 5
2
v
2
= 0 .
Add this last equation to 2 times equation () to obtain 3
2
v
2
= 0.
Since v
2
= 0, we must have
2
= 0. Substituting back into either
equation gives
1
v
1
= 0, so that
1
= 0 since v
1
= 0. This shows
that v
1
, v
2
are linearly independent.
Generalisation 1. The same proof works for any constants, Av
1
= v
1
,
Av
2
= v
2
provided = .
Generalisation 2. It also extends to 3 (or more) non-zero vectors,
Av
1
= v
1
, Av
2
= v
2
, Av
3
= v
3
with , , distinct constants
(that is, no two are equal).
Question 3.8 The coordinates are [x]
B
= (1/5, 1/25, 43/25)
T
.
Question 3.9 S
1
spans R
2
since
1 2
2 3
= 0. A basis of R
2
is S
2
or
{e
1
, e
2
}. R
2
has dimension 2.
A basis of S
2
is
__
1
1
__
(since all other vectors are scalar
multiples of it). The set spans the line x = t
_
1
1
_
with Cartesian
equation y = x. This is a one-dimensional subspace of R
2
.
To nd a basis of S
3
or S
4
, write the vectors as the columns of a
matrix. You can either row reduce A
T
, and the non-zero rows will be a
basis, or, you can row reduce A and the columns of A corresponding to
the columns with leading ones in the echelon form will be a basis.
Using the rst method for S
3
(row reduce A
T
),
_
_
1 0 1
2 1 3
1 2 9
_
_

_
_
1 0 1
0 1 5
0 2 10
_
_

_
_
1 0 1
0 1 5
0 0 0
_
_
.
Then a basis is given by the top two vectors. The set S
3
spans a plane
(through the origin) in R
3
, Lin(S
3
) has dimension 2.
61
Using the second method for S
4
, write the vectors as the columns of a
matrix A and reduce A to echelon form. Columns 1, 2 and 4 have
leading ones, so the rst, second and fourth vector form a basis of
Lin(S
4
), which is a 3-dimensional subspace of R
4
.
Question 3.10 S
1
is not a basis. No set of four vectors in R
3
can be
linearly independent.
S
2
is not a basis. Two vectors cannot span R
3
: there must be at least
three.
S
3
is not a basis. Either notice that the third vector is the sum of the
rst two, or reduce A
T
to echelon form and show it has a row of zeros,
or show |A| = 0. The set is not linearly independent.
S
4
is a basis. You can reduce A
T
to echelon form or you can
compute |A| = 5 = 0, which shows that Ax = b has a unique solution
for all b R
3
.
To nd the coordinates of w =
_
_
1
2
1
_
_
in the basis S
4
you need to
nd the unique solution of
_
_
1
1
1
_
_
+
_
_
1
1
0
_
_
+
_
_
2
3
3
_
_
=
_
_
1
2
1
_
_
.
Reduce the augmented matrix to reduced echelon form to nd
= 2, = 3, = 1, so that
[w]
S
4
=
_
_
2
3
1
_
_
S
4
.
Question 3.11 The yz plane is the set of all vectors of the form
(0, y, z)
T
. So the set of vectors {e
2
, e
3
} is a basis of the yz plane.
Question 3.12 We omit the details. A basis for the null space is
_
(1, 2, 1, 0)
T
, (1, 1, 0, 1)
T
_
. There are many other possible answers.
Question 3.13 A basis for the column space is
_
_
_
_
_
1
1
0
1
_
_
_,
_
_
_
2
3
1
2
_
_
_,
_
_
_
2
2
4
5
_
_
_
_
_
.
Details of the calculations are omitted.
Question 3.14 Reduce the matrix B to reduced echelon form,
B =
_
_
1 2 1 3 0
0 1 1 1 1
1 3 2 0 1
_
_

_
_
1 2 1 3 0
0 1 1 1 1
0 1 1 3 1
_
_

_
_
1 2 1 3 0
0 1 1 1 1
0 0 0 4 2
_
_
_
_
1 2 1 3 0
0 1 1 1 1
0 0 0 1
1
2
_
_

_
_
1 2 1 0
3
2
0 1 1 0
1
2
0 0 0 1
1
2
_
_

_
_
1 0 1 0
5
2
0 1 1 0
1
2
0 0 0 1
1
2
_
_
.
62
The leading ones are in the rst, second and fourth columns. For the
solutions of Bx = 0, set the non-leading variables x
3
= s and x
5
= t.
A basis of the row space is:
_
_
_
_
_
_
_
1
0
1
0
5
2
_
_
_
_
_
,
_
_
_
_
_
0
1
1
0
1
2
_
_
_
_
_
,
_
_
_
_
_
0
0
0
1
1
2
_
_
_
_
_
_
_
R
5
.
A basis of the range of B is:
_
_
_
_
_
1
0
1
_
_
,
_
_
2
1
3
_
_
,
_
_
3
1
0
_
_
_
_
_
R
3
.
So the range of B is all of R
3
.
From the reduced echelon form of B, the solution of Bx = 0 is
_
_
_
_
_
_
x
1
x
2
x
3
x
4
x
5
_
_
_
_
_
_
=
_
_
_
_
_
_
s
5
2
t
s +
1
2
t
s
1
2
t
t
_
_
_
_
_
_
=
_
_
_
_
_
_
1
1
1
0
0
_
_
_
_
_
_
+
_
_
_
_
_
_
5
1
0
1
2
_
_
_
_
_
_
= x
1
+x
2
.
The set {x
1
, x
2
} R
5
is a basis of the null space.
Therefore rank(B) = 3 and nullity(B) = 2. The number of columns
is n = 5, so that
rank(B) + nullity(B) = 3 + 2 = 5 = n.
If b = c
1
+c
5
, then p = ( 1 0 0 0 1 )
T
is a solution of Bx = b.
(Why?) Combining this with the solution of the system Bx = 0, a
general solution of Bx = b is
x =
_
_
_
_
_
1
0
0
0
1
_
_
_
_
_
+
_
_
_
_
_
1
1
1
0
0
_
_
_
_
_
+
_
_
_
_
_
5
1
0
1
2
_
_
_
_
_
= p +x
1
+x
2
.
63
64

mt2118 ch1-3

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

mt2118 ch1-3

Uploaded by

Copyright:

Available Formats

Undergraduate study in

= 1(3) 2(1) + 3(13) = 34.

+ 0 = 3(13) (5) = 34.

R such that x = v and y =

= 0. Consider u +v. This

You might also like