Now, I go to here, LVA trainer.

This is
where I'm going to train the system. I'm
going to take these 80 dogs and cats and I
figure out. What I mean by training means
what I'm going to do is project the common
features of these, get the modes out of
this, so that I can do this linear
discrimination analysis. This is how I got
those pictures of those cats and dogs,
those Igon dog faces, okay? Was through
this LDA trainer. so it was already done
in the last lecture. But let me highlight
it for you what goes on in there. What you
send in is your dogs, your cats, and the
number of features. So you throw in all
your dog, data, wavelet data, all your cat
wavelet data, and the number of features
you going to keep. Right now we have
twenty, okay? We'll play around with that.
And I go get the length of this dog data,
the length of the cat data. We already
know it's 80. But again trying to robut,
get a robust algorithm you should always
if find out for itself. Oh this is big,
this is what I got to do. Okay, these are
the by the way good programming tips to
help'cuz all the sudden if you write. It's
subroutines within subroutines. And if,
all of a sudden, you get down to a level
where you are putting in stuff by hand.
And someone gives you a data set with 85
cats and dogs. You're going to have to
track down everywhere you had an 80 in
various subroutines, which is a little
amateur. Okay if they're not equal. Dogs
and cats are not equal. But a little
warning comes up. Why is that important? .
Presumably, you know, if I were to take.
100 cat pictures and put only ten dog
pictures in. What do you think is going to
happen when I do an SVD? Everything's
going to be weighted towards those cats,
cuz I have a lot more. Remember it's
correlating all this data together. If you
put the equal number of cats and dogs in,
somehow, says look, I have an equal number
of dogs and cats. Which means when I build
these. I can mode faces of dogs and cats.
It doesn't sort of favor one or the other,
okay? It's sort of equal, equal ground.
There's the SBD. Whoomp. Take my dog data,
my cat data, stack ''em on top of each
other. I do an economy SVD which is the
zero there. The economy SVD, remember how
the SVD worked? if you remember, we fill
in these extra rows and columns with
silent rows and columns. That was from a
while ago, I, I don't expect you to
remember that. But, you don't have to do
that. Zero says don't do that. Just gimme
what I need, okay? So, SPD, I now have
these three matrices that we've been
talking about the whole time. And. You can
sort the data. Which by the way it should
already be sorted. But, there you go. This
is just in case. What's the orientation of
all the 0's there 85? I think they're by
80 by 32 squared. Okay. Yeah. I'm not
positive. I'm pretty sure. So it's 80 by
32^2. And you stack it on top of another
80 by 32^2. So all of a sudden, now, you
have 160 by 32^2. And the correlates all
that. And now what you have is you have
this beast here where you're going to
basically have it sorted. You shouldn't
have to do the sorting. It's already
sorted anyway the SVD. but this, this. If
you had to sort it, this would get
everything sorted by descending order.
Okay, now Important. Right there. Each
animal, how is it going to project onto
the U basis. Nor the SPD does. The V
vector carries information of the
individual dogs for cats. How do you work
it out? Well, V multiplies the weighting,
and then you multiply them by the modes.
So what this is, is to take each dog and
cat but. It's a little bit a fake measure,
right? Because if I take how does the, how
does an individual cat or dog project onto
the modes there's this pre factor that I
have to take into account. It's V1 is so
much but it has to be multiplied by the
first diagonal element, that gives the
appropriate weighting. So what you're
really doing in this animals vector is
giving the appropriate weight is S times V
hat. Okay, this gives you the right
projection and what I'm going to do now
for you is I'm going to rewrite U as U is
the first. Twenty columns. So the, the
right now, the u matrix has, if I have 160
total dogs and cats, u matrix has 160
columns, 160 modes, and I'm going to say
take the first twenty. And what I want to
do, is now project my dogs and cats. So
here. What I'm going to do with that is
take. Okay. With my animals vector.
Remember the way I stacked them is the
first one through 80. Are thoughts, the
next 81 to 160 are cats, and so what I'm
going to do is take, all right, how do I
do the corresponding corresponent, qa, I,
ca, components for this, so what I got to
do is take my first, one through ND. Cats
projected on to the feature space.
Remember that animals is giving me the
projection vector. Right. Telling me how
the waiting happens on these, on of all
the modes and what I would say is I don't
want all the modes. I want just the first,
first twenty modes. First twenty features.
So what cats and dogs and cats vectors is,
is the projection of my dogs and cats onto
the first twenty features, okay? Now, by
the way, we already went, this is, this is
just reviewing what actually happened
Wednesday. So I'm kind of just
highlighting so it, I, I think I spent
more time on that in that class I think. I
hope. All right. and here. Now once we
have that, let me just tell you, you find
the means of the dogs and cats. What we're
going to do is subtract out the mean,
because that's technically what principal
component analysis does. You subtract out
the mean of the data. If you, you wouldn't
have to this, and you'd still get good
results. But if you want to really do,
strictly speaking, talk about principal
components, you subtract out the means.
And the means are subtracted out here,
where what this is, this is how you
project using a linear discrimination
analysis. And I won't have time to go over
it. But. If you look at the last lecture,
Wednesday's. This is basically, what this
code does here is it figures out what is
this plane? I have the clumping of data
now of all my cats and dogs. In wavelet
space, projected down, and then that gives
me my little, balls in space, and I figure
out from that, where the best projection
plane is. I project everything down. Okay.
That's what this whole thing here does.
Okay. Now the rest of this here, does,
does some plotting and so forth, but what
it does, is actually, the, the result of
all this is it gets me a number. The only
number I'm interested in is this one right
there. I want the plane. And I want that
number. With those two things, I'm going
to bring in new cat and dog datas. And
what I'm going to do with the new cat and
dog data. Is, I'm going to basically take
my new cat. And I'm going to say, okay,
I've got a new cat. I'm going to wavelet
transform it. I'm going to project it onto
the feature space. This U matrix, which
has twenty columns. Figure out, what is
the best coefficient to project onto the U
space. Once I get that, it's going to give
me my V values. Which I'm going to use
then to say where does that sit me in here
in parameter space and how do I drop it
down to here? And you just inter-product
it down with this vector and it's going to
give me a number and a score. And then we
say, is that a square on this side or this
side of that line. If it's over here, it's
a cat. If it's over here, it's a dog.
That's the whole algorithm. It combines
everything, okay? Alright, so let's go
back out to main. And that is what the
trainer does. Now, here's the thing. This
is an important thing, this trainer is a
one time deal. This is important. If you
are writing real al, realtime algorithms.
Let's say you want to try to do face
detection algorithm. A face detection of a
certain person. You want to put a camera
outside of Guggenheim Hall, anytime,
anytime Charles walks in you want this to
go oh there's Charles. Sends a little
warning light to everybody in the
building, Charles is here. So what would
you do? It's not like every time you do
this you're going to say okay I've got 200
pictures of Charles so I'm going to look
at his principle components and then I'm
going to project his picture on it. Every
time I do I'm going to view th is SVD of
Charles 200 times. I mean you do it once.
You get to 200 pictures, you get out his
principle components and all you got to do
now is take the new picture, project it on
to what you already did. So this is an
upfront cost. And by the way, the reason I
mention that, here it's not going to be a
problem. But you have large data sets. You
want to do this only once. It costs you
order N cubed, okay? Right now we're using
160. By 4,096 matrix I want to SVD that,
you can imagine then you have a lot more
data and you want a lot higher resolution
data you're going to be doing SVD of
ginormous stuff and that is very costly.
Cuz you want real time, you got to do it
once, it's an upfront cost, you should
never have to pay that cost again, okay?
So that is what we've got. Okay, now we
start with what we want to do next which
is the actual testing and here it is. So
what we're going to do is load pattern
wreck ants. So these are new, 38 new dogs
and cat pictures, never before seen by the
computer. And here is a question, how well
is it going to do on these new dogs and
cats? They're not part of the training
set. And by the way, very important, when
you do the training. When you have a bunch
of dogs and cats, pictures, and you want
to do a training on two types of things,
you train it on a certain data set and
then you bring a data set from elsewhere
in. You do not like, I've trained it on
this data set and now I'll use part of the
data set to ask if I did okay. You don't
do that. You always have aside for you
some other stuff. U2018kay? That's fresh.
Otherwise you're, it's called cheating.
Then you're going to Computer way of
thinking about. So I'm going to load 38
more pictures. And here's what I'm going
to do with'em. I'm going to put'em through
PP wavelet. What that's going to do, it's
going to take all 38 pictures, wavelet
transform them. Pull it back up for me.
Just like it did with all the dogs and
cats. And here's the key steps Two key
steps. Now that I have their wavelets,
what do I got to do? I've got to pr oject
these wavelet pictures onto the modes. How
do I do that? Right there. Take each one
of those pictures project it onto U. And
by the way, U has twenty. Columns or rows,
whatever. U is the Oh, the features ,
yeah. The U that came back out for us was
the one that had the twenty features. So
what I'm going to do is say take these
wavelet dogs and cats, project them onto
the twenty features. Where am I now? I
project I'm on the future , a spot in some
higher dimensional space. Boom, sitting
there. Now what I got to do is figure out
how do I project it down onto a plane.
Right? We're almost at the end. And the
way you project them onto the plane. Right
here. Take what you get left over, and
this W right here. That is your separation
space that we determined before. That is
your. The W, is this.You take a dot
product. How do you project a vector onto,
another vector? Dot product it. How do you
take a jet vector projection onto a plane?
You dot product it with that plane. That's
all we're doing. Take your point and
space, take an inner product; a dot
product with that, separation. The best
separation direction you can. Which is
what's guaranteed in the LDA. Get a spot.
That's it. And now all you have to do is
have an if statement in there. And if it's
on this side, dog. And if it's on that
side cat, okay? So you see it's pretty
nice. Right? I mean, actually, right, you
just project, project. You're using, the
dimensionality reduction of the data
through using correlations of the SVD to
get these nice projections. All the way