21. Eigenvalues and Eigenvectors

Well, this is the first lecture on

eigenvalues

​​and

eigenvectors

and that is an important topic that will take up most of the rest of the course. Again, the matrices are square and now we are looking for some special numbers, the

eigenvalues

​​and some special vectors.

eigenvectors

, so this lecture is mainly about what these numbers are and then the other lectures are how do we use them, why do we want them, so what is an eigenvector? Maybe I'll start with eigenvector, what is an eigenvector? So I have an array. What does a matrix do? It acts on the vectors, it multiplies the vectors . comes f from X here in linear algebra we are in more dimensions enter a vector vectors ax be at points in some different direction, but there are certain vectors where ax runs parallel to X and those are the eigenvectors, so ax parallel to Oh, very much. easier to just express it in an equation ax is a multiple and everyone calls that lambda multiple of X and in the same direction, I allow it to be in the opposite direction.

I allow lambda to be negative or zero. Well, I guess we've known eigenvectors that have zero eigenvalue, they're in the same direction but they're in a very special shape. So I understand that this is the eigenvector Let's take a second about zero eigenvalue from the eigenvalues ​​point of view, that's not a special problem, we have an eigenvector, if the eigenvalue was zero, that would mean that ax was zero. X in other words, zero, so what would X be? Where would we look? What are the X's? What are the X's? Eigenvectors with zero eigenvalue are the types in the null space ax is equal to zero so if our matrix is ​​singular then let me write this if a is singular then what does singular mean it takes some vector that's what, okay, the eigenvector will be at 0, so lambda equals 0 is an eigenvalue, but we're interested in all eigenvalues, now lambda equals 0 is not so special anymore, okay , then the question is how do we find these X and lambdas and Note that we no longer have an equation ax equals B.

I can't use elimination. I have two unknowns and, in fact, they multiply each other. Lambda and X are both unknowns here, so we need to. a good idea of ​​how to find them, but before you do that and that's where a certain comes in, so I can give you some matrices like here, take the matrix, a projection matrix, okay, let's assume we have a plane and our matrix. P is what I've called now, I'll call it P for the moment because I'm thinking, well, let's look at a projection matrix, what are the eigenvalues ​​of a projection matrix?

So that's my question, what are the eigenvectors of the X's? and the lambdas, the eigenvalues ​​of and now let me say a projection matrix. My point is that before we get into determinants and formulas and all that, let's take some matrices where we know what they do, we know that if we take a vector B, what this matrix projects and reduces to PB, so it's an eigenvector in that image, that vector is an eigenvector, no, no, so B is not an eigenvector because P is the projection in a different direction, so now tell me which vectors are eigenvectors of P, which vectors are projected onto the same direction they start in, so please answer, tell me some X's in this image, where could you start with a vector B or Well, that would happen. if the vector was already in that plane if the vector here because this is our family image, but now I'm going to say that B was not good for our purposes.

I'm interested in a vector X that is actually in the plane and I project it and what do I get? back X, of course, doesn't move, so any X in the plane doesn't change by P and what does that tell me? It tells me that x is an eigenvector and everything tells me what the eigenvalue is, which is simply comparing it to eigenvector. The value of the multiplier is just a good, so we actually have a complete plane of eigenvectors. Now I asked if there are other eigenvectors and I hope the answer is yes because I would like to get three if I am in three dimensions. expect three independent eigenvectors, two of them in the plane and one not in the plane, so this type B that I drew was not good what is the correct eigenvector that is not in the plane?

The good one is the one that is perpendicular to the plane, there is another good X because what is the projection for these to be eigenvectors, another type here would be another eigenvector but now here is another any That vector, what is the projection of this type perpendicular to the plane? is zero of course so there is the px null space and note that those types are 0 or 0 X if we want and the eigenvalue is 0 so my answer to the question is what are the eigenvalues ​​for our projection matrix? they are 1 and 0 ok, we know the projection matrices, we can write them as a transposition, an inverse transposition, but without doing that in the image we can see what the eigenvectors are, are there other matrices? take a second example, how about a permutation matrix?

What about the matrix? I'll call it now zero one one zero. Can you tell me a vector Look, we'll have a system soon, so I'd like to do these two. of examples just to see the picture before we let everything go into a system where that matrix is ​​nothing special because it is special and then what vector could you multiply it by and end up in the same direction? Can you detect an eigenvector for this type is a matrix that permutes x1 and x2 correctly, changes the first two components of What do we start? with, can you tell me an eigenvector here for this guy?

X is equal to what actually, can you tell me a vector that has an eigenvalue 1? So which vector would have an eigenvalue 1? So if I swap it, it doesn't change there. be 1 1 thanks 1 1 okay take that vector 1 1 that's going to be an eigenvector because if I do an ax I get 1 1 so that's the eigenvalue is 1 cool that's an eigenvalue but I have a 2 by matrix here 2 and I imagine it's going to be a second eigenvalue and eigenvector now, what's up with that? What is a vector? Well, maybe we can just guess it as a vector that the other one actually this one that I'm thinking of will be a vector that has an eigenvalue minus one, that's my.

This will be my other eigenvalue for this array. That's the notice that the array is positive or non-negative, but an eigenvalue will come out negative and can you guess which X will work for that? So I want a vector when I multiply by a. which inverts the two components. I want the thing to come out minus the original, so what should I send in that case? If I send negative 1 1, when I apply to get, I do that multiplication and I get 1 negative 1, so sign reversed, so ax is minus X lambda is minus 1 aah, so ax was X there and ax is minus .

Can I just jump ahead and point out a little special fact about the eigenvalues ​​of n by n matrices that will have n eigenvalues ​​and won't? So suppose that n is 3 or 4 or more, it is not so easy to find them, we would have an equation of the third degree, of the fourth degree or of the nth degree, but here there is an interesting fact, there is a pleasant fact: the sum of the eigenvalues is equal to I summarize the diagonal that is called trace and I specifically put it in the content of the lecture, so it is a clear fact, a fact that some of the lambdas sum the lambdas are equal.

There is something you would like me to do at home. I write it. What I mean in words is the sum of the diagonal of a. I need to write a 1 1 plus a 2 2 plus a in that, adding the diagonal entries in this example is 0, in other words, once I found this eigenvalue. of 1 I knew that the other one had to be minus 1 in this case of 2 times 2 because in the case of 2 times 2, which is good for playing with trace, it tells you right away what the other eigenvalue is, so if I say your 1 eigenvalue could you tell me the other we will have that we will see again ok now I can give more examples but maybe it is time to face the equation ax equals lambda X and calculate how we are we are going to find X and lambda ok so this is the question now it's how to find eigenvalues ​​and eigenvectors how to solve how to solve for x equals lambda is equal to 0 to the right.

I have an X minus lambda I know lambda and I don't know X, but I do know something here, what I know is if I'm going to be able to solve this for something tell what I know now is that this array must be what if I'm going to be if there is an matrix this special combination which is like matrix a with lambda shifted by lambda shifted by lambda I which has to be singular this matrix must be singular otherwise the only What do I know now about singular matrices? They are determinants is zero, so by the fact that it has to be singular, I know that the determinant of a minus lambda has to be zero and that now I have of eigenvalue and that, in other words, I am now in a position to find lambda. first, so this is the idea will be to find lambda first and in fact, I will know that I will not find a lambda.

I will find n different lambdas and maybe not in different lambdas, a lambda could be repeated repeated lambda is the source of all the problems in 1806, so let's hope for the moment that they are not repeated, they are there, they were different, right, one and less one in that, for that permutation, okay, and after I find this lambda, can I look ahead? How am I going to find X after I have found this lambda? The lambda is one of the numbers that makes this matrix singular. So of course finding X is just by elimination. Right now I have a singular array that I'm looking for. for null space we are experts in finding null space, you know you do elimination, identify dynamic columns etc., and give values ​​to free variables, there will probably only be one free variable that will give the value like there. and we find the other variable, okay, so let's find the second X, it will be a doable job.

Come on, let's look at the first job of finding lambda. Can I take another example and let's solve that one? Well, let me take the example. let me make it easy three three one and one so I made it easy I made it - by - I made it symmetric and I even made it constant on the diagonal so that the more special properties I put in the matrix, the more special result I get for the eigenvalues , for example, than this symmetric matrix. I know it will come out with real eigenvalues. The eigenvalues ​​will turn out to be good real numbers and in our previous example, that was a symmetric matrix, yes, actually, while we.

You're in it, it was a symmetric matrix, its eigenvalues ​​were nice real numbers 1 and minus 1 and you notice something about its eigenvectors, something particular about those two vectors 1 1 and -1 1, it just turns out that no, I can't say that It just happened to be because that's one point is that they had to be what are they? They are perpendicular to the vector when if I see a vector 1 1 and a 1 and a minus 1 1 my mind immediately takes that scalar product which is 0 those vectors are perpendicular, that will happen here very well, let's find the eigenvalues, actually, I must my too easy examples, my example is too easy, let me tell you in advance what is going to happen, can I or should I do the determinant of a minus lambda? and then point out at the end what you will remember me after finding the eigenvalues ​​to say why they were easy from the example we did well, let's do the work here, let's pretend, let's calculate the determinant of a minus lambda I so that's a determinant and what? what is this?

It's the matrix a with lambda removed from the diagonal, so the diagonal matrix is ​​shifted and then I'm taking the determinant, okay, so I multiplyThis, then, what is that determinant? I didn't remove lambda from all the inputs it's lambda so it crashed along the diagonal and I got three lambda squared and then minus 1 to the right and I want it to be 0 well I'm going to simplify it and what will I get , so if I multiply this I get lambda squared minus 6 lambda plus what else 8 and I'm going to set that to 0 and I'm going to solve it like this and it's a quadratic equation.

I can use factoring. I can use the quadratic formula. I'll get two lambdas before doing that. Tell me what that number six is ​​that appears in this equation. a trace that the number six is ​​three plus three and while we're at it, what is the number eight that appears in this equation? It's the determinant that our matrix has determined to be eight, so in a 2x2 case it's really good, it's lambda squared minus the trace. multiplied by lambda, the trace is the linear coefficient in plus the given constant term, okay, so can we find the roots?

I guess the easiest way is to factor that as something multiplied by something, if we couldn't factor it then we would have to use the old formula b squared minus 4ac, but I think we can factor that into lambda minus what multiplied by lambda minus what for do that factorization and for lambda minus four times lambda minus two so that the eigenvalues ​​are four and two so that the eigenvalues ​​are a lambda eigenvalue, one let's say it's four lambda and the other eigenvalue is two, the eigenvalues ​​are 4 and 2 and then I can go for the eigenvectors.

You see, first I got the eigenvalues ​​4 and 2, now for the eigenvectors, so what are the eigenvectors? There are these guys in the null space when I remove when I do the singular matrix taking 4i or Iowa, so we have to do them separately. Let me find the eigenvector for first, so I'll subtract 4, so minus 4i is like this. taking away 4 we will put minus ones in there and what is the point of that matrix if 4 is an eigenvalue then a minus 4. You better figure out what kind of singular matrix if that matrix is ​​not singular, the 4 was not correct but we are fine, that matrix is ​​singular and what is the So that's the goal, so now I'm making ax 1 lambda 1 vector X in null space, of course, is 1 1, so that's the eigenvector that goes with that.

Eigenvalue Now, how about the eigenvector that goes with the other eigenvalue? Can I do that by deleting? I take minus 2 I, so now I take 2 from the diagonal and that leaves me with a 1 and a 1, so a minus 2 I like again produced a singular matrix like it had to. I'm looking for the null space of that guy, what vector is in your null space, well of course a complete line of vectors, so when I say the eigenvector, I'm not speaking correctly, there is a set line of eigenvectors and you I only want a base and for a line I only want a vector, but I could have some freedom to choose that one, but choose a reasonable one, what is a vector in the null space of that well, the natural vector to choose as eigenvector with lambda 2 is minus 1 1 if I were to eliminate that vector and set the free variable to be 1 I would get minus 1 and I would get that eigenvector, so you see what I have? counter eigenvalue eigenvector eigenvalue for this for this matrix and now it becomes that I wanted to remember what is the relationship between that problem and let me write just above what we find here is equal to 0 1 1 0 which has eigenvalues ​​1 and minus 1 and eigenvectors 1 1 and eigenvector minus 1 1 and what do you notice?

How is this matrix related to that matrix? How are those 2 matrices related? 1 is only 3i more than the other, right? I just took that array and took this array. and I added 3i so my question is what happened to the eigenvalues ​​and what happened to the eigenvectors that is the question we keep asking now in this chapter if I do something to the Mae Trix what happens if I already know something about the matrix which is the conclusion for its eigenvectors and eigenvalues ​​because those eigenvalues ​​and eigenvectors are going to give us important information about the matrix and here, what are we seeing what's happening with these eigenvalues ​​1 and minus 1 when I add 3? i just added 3 to the eigenvalues ​​i got 4 and 2 3 plus 1 and minus 1 what happened to the eigenvectors nothing at all 1 1 is + - + 1 and minus 1 1 are still the eigenvectors in other words simple observation but useful if I add 3i to a matrix, its eigenvectors do not change and its eigenvalues ​​are 3 larger, let's see why let me keep all this on the same board, suppose I have a matrix a and ax equal to lambda to that matrix.

Do you see what their eigenvalues ​​are, the eigenvalues ​​and the eigenvectors are going to come out? If so, there is a if ax is equal to lambda I only have one ax which is lambda X and I have a 3x. of the 3x, so it's just I mean it's sitting there lambda plus 3x, so if you had the lambda eigenvalue of this size the lambda eigenvalue plus 3 and the well, that's great, of course, it's special. we obtained the new matrix by adding three. I guess I added another array. Suppose I know the eigenvalues ​​and eigenvectors of a.

So this is this little board that's not going to be so good. Suppose I have a matrix a and it has a matrix of its own. vector if here if ax is equal to lambda I'm going to use alpha for the eigenvalues ​​of B for no good reason, well you see what I'm going to ask how about a plus B, let me let me give you what you might think first, okay, if ax is equal to lambda X and if B has an alpha eigenvalue, so I can tell what is what is the matter with this argument, it's wrong what I'm going to write is wrong.

I'm going to say that BX is Alpha a more B but that's false a more B what when B was 3i that job worked very well but this is not so good and what's the problem with that? argument there, we have no reason to believe that X is also an eigenvector of B. B has some eigenvalues ​​but it has some different eigenvectors. Normally it is a different matrix. I don't know anything special if I don't know anything special then so far. like I know it has a different eigenvector Y and when I add I just get garbage, I mean I can add but I don't learn anything so it's not so good a plus B or a multiplied by B, usually the eigenvalues ​​of a plus B o a multiplied by B are not eigenvalues ​​of a plus the eigenvalues ​​of b eigenvalues ​​are not linear o and do not multiply because the eigenvectors are usually different and there is no way to find out what a plus B does to a vector, does OK? that's like a warning, don't do it if B is a multiple of the identity, great, but if B is a general matrix, then for a plus B you have to find that you have to solve the eigenvalue problem.

Okay, now I want to do another one. example that highlights another point about eigenvalues ​​let me make this example a rotation matrix okay so here's another example so a rotation oh I better call it Q I often use Q for four rotations because those are very special, very important examples of orthogonal matrices. Let me do a 90 degree rotation so that my matrix is ​​the one that rotates each vector 90 degrees. Remember that matrix is ​​the cosine of 90 degrees, which is zero? The 90 degree sine, which is one. minus the sine of 90 the cosine of 90 then that matrix deserves the letter Q it is a very very orthogonal matrix now I am interested in its eigenvalues ​​and eigenvectors two by two it cannot be that difficult we know that the eigenvalues ​​add up to zero in actually we already know something here the eigen what is the sum of the two eigenvalues ​​just tell me what I just said zero from that tracking business the sum of the eigenvalues ​​is going to come out zero and the product of the eigenvalues ​​Did I tell you that Is the determinant the product of the eigenvalues?

No, but it's good to know. We point out how that eight appeared in the quadratic equation. So let me say this, the trace is zero plus zero, obviously. and that is the sum that is lambda 1 plus lambda 2 now the other interesting fact is that the determinant what is the determinant of that matrix 1 and that is lambda 1 multiplied by lambda 2 in our example, the one that we calculated, the eigenvalues ​​came out four and two their product was eight, it had to be eight because we factored lambda minus four by lambda minus two, which gave us the constant term eight and that was the determinant.

Well, what I mean by this example is that something is going to go wrong. something goes wrong for a rotation because what vector can come out parallel to itself after a rotation if this matrix rotates each vector 90 degrees what could be an eigenvector? You see, we are going to have problems. I can, the vectors are our image of The eigenvectors that come out in the same direction that they came in there won't be any and with the eigenvalues ​​we're going to have problems with these equations. Let's see why I expect problems. The first equation says that the eigenvalues ​​add up to 0.

So there is a plus and a minus, but then the second equation says that the product is plus 1 and we are in trouble, but there is a way out, so how are we going to do the usual ? Look at the determinant of Q minus lambda. Well, I'm just going to follow the rules, I'm going to take the determinant, I'm going to subtract lambda from the diagonal where it had zeros, the remainder is equal, the remainder of Q was just copied, I'm going to calculate that determinant, okay, so what determines that? is equal to lambda squared minus -1 plus 1 What happens?

There's my equation. My equation for the eigenvalues ​​is lambda squared plus 1 equals 0. What are the eigenvalues ​​lambda 1 and lambda 2? Whatever and less, those are the correct numbers that add to 0 as the trace requires and multiply to 1 as the determinant requires, but they are not real numbers even though the matrix was perfectly real, so what this can happen. Complex numbers will have to enter 1806 right now, well, okay, if I choose right. matrices that have real eigenvalues ​​we can postpone that bad day, but so you see, I'll try to do that, but it's out there that a matrix, a perfectly real matrix, could have given a perfectly innocent looking quadratic thing, but the roots of that quadratics can be complex numbers and of course everyone knows that what do they know about complex numbers so now let's spend one more minute on this bad possibility of complex numbers.

We know a little information about the two complexes. the numbers are complex conjugates of each other if lambda is an eigenvalue then when I change when I go you remember what complex conjugates are you change the sign of the imaginary part well this was just imaginary it had no real part so we just change its side , then the eigenvalues ​​come in pairs like that, but they are complex, a complex conjugate pair and that can happen with a perfectly real matrix and in fact, that was Mike, my previous point that if the matrix were symmetric it wouldn't be .

This happens so if we stick to matrices that are symmetric or nearly symmetric then the eigenvalues ​​will still be real, but if we move away from the symmetric ones and that's as far as we can move because that matrix is ​​how the transpose relates Q with Q for that. matrix that matrix is ​​antisymmetric that the transposition is less that that is the complete opposite of symmetry when I rotate the diagonal I get I invert all the signs those are the types that have pure imaginary eigenvalues, so they are the extreme case and in the middle our matrices that are not symmetric or antisymmetric but have partly a symmetric part and an antisymmetric part, okay, so I'm doing a bunch of examples here to show the possibilities, the good possibilities are perpendicular eigenvectors, real eigenvalues, the bad possibilities.

Being complex own values, we could say that this is bad, there is something even worse. I'm getting over the bad things here today and then the next conference can be like pure bliss, okay, here's one more bad thing that could happen, so I'll do it again. with an example suppose my matrix is ​​suppose I take this three three one and change it to zero what are the eigenvalues ​​of that matrix what are the eigenvectors this is always our question of course the following sections will show why, for what or why do we care? But for now this conference presents them and we are going to find them.

What are the eigenvalues ​​of that matrix? Let me tell you that at a glance we could answer that question because the matrix is ​​triangular. It's really useful to know. If you have properties like a triangular matrix, it is very useful to know that you canread the eigenvalues ​​right there on the diagonal, so the eigenvalue is 3 and also 3 3 is a repeated eigenvalue, but let's see what happens, let me get this right. determinant of a minus lambda I what I always have to do is this determinate I take lambda from the diagonal I leave the rest I calculate the determinant so I get 3 minus lambda times 3 minus lambda and nothing then that's where the triangular part is it came in triangular part The only thing we know about triangular matrices is that the given is just the product on the diagonal and in this case, this is what repeats, so lambda 1 is 1, sorry, lambda 1 is 3 and lambda 2 is 3 , that was easy, I mean, no, why should I be pessimistic about an array whose eigenvalues ​​can be read immediately?

The problem with this matrix is ​​in the eigenvectors, so let's go to the eigenvectors, so how do I find the eigenvectors? I'm looking for a pair of eigenvectors, so I take the eigenvalue. What do I do now? We remember that I solve a minus lambda IX is equal to zero and what is it. a minus lambda IX, so take away three and I get this matrix zero zero zero one multiplied by and what kind of matrix am I supposed to have here singular right, it's supposed to be singular and then it has some vectors, which one is it, so it has a vector for that guy, tell me what is a vector It's cool now what is the other eigenvector?

What is the eigenvector that goes with lambda 2? Well, lambda 2 is 3 again, so I have the same thing again, give me another vector. I want it to be independent. If I'm going to write a x2, I'll never do it. let it depend on x1. I'm looking for independent eigenvectors and what is the conclusion: there are none. This is a degenerate matrix. It only has one line of eigenvectors instead of two. It is the possibility that a repeated eigenvalue opens up this additional possibility. of a paucity of eigenvectors and therefore there is no second independent eigenvector x 2, so it is a matrix, it is a two-by-two matrix, but with only one independent eigenvector, so it will be those the matrices in which there are or are not eigenvectors. full story ok, my lecture on monday will give the full story of all the other arrays.

Thank you, have a good weekend, a real New England weekend.