Linear Algebra Eigenvalues and Eigenvectors

Welcome back to educator.com and welcome back to linear algebra.0000

Today, we are going to start on a new topic. A very, very, very important topic. 0008 Probably the single most important topic both in terms of the underlying structure of linear mappings, matrices, and also of profoundly practical importance, in all areas of science and math... quantum mechanics, engineering, all areas of physics, all areas of mathematics.0004

Also, we are going to be discussing Eigenvalues and Eigenvectors. So, let us solve this and jump on in and see if we can make sense of this.0030

Okay. Recall if you will, so if... a is n by n... and for the discussion of Eigenvalues and Eigenvectors, we are always going to be talking about matrices that are n by n.0039

So, we are no longer going to be talking about 5 by 6, 3 by 9, it is always going to be 3 by 3, 4 by 4, 2 by 2... things like that.0056

Okay. So, if a is n by n, we know that the function L, which is a mapping from RN to RN defined by the multiplication of some vector x by that matrix... we know that it is a linear mapping.0063

So, this we know. That when we are given a n by n matrix, and we use that matrix to multiply on the left of some vector in RN, we know that what we get is a linear mapping. Okay.0107

What we want to do, so we wish to discuss this situation where the vector x and anything I do to x which is multiply it by the matrix on the left are parallel to each other... by parallel, what we are really saying is that they are scalar multiples of each other.0122

In other words, or when a × x is just a scalar multiple of x.0155

In other words, I do not just map it to a completely different vector all together, all I do is take the vector x and I either expand it or contract it or leave it the same length.0175

So, I am keeping it in its own space. I do not jump to another space. That is really what is going on here with this idea of Eigenvalue and Eigenvector.0184

It has to do with starting in a space, taking a vector in that space, multiplying it by a matrix, and instead of twisting it and turning that vector, turn it into something else... just -- you know -- dilating it, making it bigger or smaller.0193

That is all we are doing. Still staying in that space. Staying parallel. Okay.0206

Now, let us see what we have got. Let us start with a definition. Let a be an n by n matrix, the real number Λ.0213

It is always symbolized with a Λ this is traditional.0234

It is called an Eigenvalue of a if there exists a non-zero vector x such that the matrix a × x just gives me some scalar Λ × x.0240

So, again, all this is saying is that we are starting with a vector x, if I multiply it by a matrix, it is the same as multiplying that vector by some scalar multiple.0272

Instead of twisting it and turning it, all I have done is expand it, contract it, or left it the same. Okay.0285

Now, every non-zero vector satisfying this relation is called an Eigenvector... whoa, that was interesting... an Eigenvector of a associated with the Eigenvalue Λ.0292

Okay. So, our central equation here is this one, our definition. It basically says, again, if I take a vector x in a subspace, and if I multiply it by a matrix, and n by n matrix, I am going to be transforming that vector... turning it into something else.0346

If, all I do to it is expand or contract that vector x, and there is actually some number by which I expand or contract it.0362

That is called an Eigenvalue, and every vector that satisfies this condition... meaning every vector that when multiplied by the matrix only ends up being expanded or contracted or left the same.0371

That is called an Eigenvector associated with the Eigenvalue, associated with the matrix a.0384

Very, very important relation. Again we are staying in this space. We are not doing anything to it. We are just moving along that space in a parallel fashion.0391

Okay. One thing we definitely want to note here is that the 0 vector cannot be an Eigenvector, but 0, the real number can be an Eigenvalue.0401

So, once again, the 0 vector cannot be an Eigenvector. We just exclude that possibility, but the number 0 can be an Eigenvalue. Okay. That is the only caveat with respect to this.0427

Quick example... let us say that a is the matrix (0,1/2,1/2,0). Okay.0445

Well, if we take a × the vector -- let us just say (1,1) -- well that is equal to (0,1/2,1/2,0) × (1, 1), that is what this is.0458

That is equal to 0 × 1 + 1/2 × that is equal to 1/2, and then 1/2 ×1 + 0 × 1, 1/2... all that is equal to 1/2 × (1,1).0473

Notice what I have done here. a × the vector (1,1) is equal to 1/2 × (1,1).0486

My Eigenvalue Λ = 1/2, because that is all a did... just simply by virtue of this multiplication... all I did was shrink it by 1/2.0496

Λ = 1/2, and the vector (1,1) happens to be one of the Eigenvectors. It is an Eigenvector... not the only Eigenvector.0509

Oftentimes, for a given Eigenvalue you have an infinite number of Eigenvectors. We will show you why in a minute.0521

Again, ax does nothing but expand or contract a vector. Okay. Now, a given Λ can have many Eigenvectors.0528

Often, we are only interested in 1... we do not necessarily need to list them all... so 1 will do.0546

So a given Λ can have many Eigenvectors associated with it, and here is why.0557

Well, if I take a × some number R × x, if I just take any vector x and I multiply it by any number, that is an infinite number of vectors that I can get.0566

Then, if I multiply that by a, we can reverse this... we can do R × a × x is equal to r × Λ x... because a(x) is equal to Λx, right? Λ is an Eigenvalue.0580

Well, that is equal to Λ × r(x). Notice what I have got. a × R(x) = Λ × R(x).0595

If I have a given vector x, any scalar multiply of x is also an Eigenvector associated with that Eigenvalue.0603

Okay. Let us do a example again. This time, we will let a equal to (1,1) - 2 and 4. Okay.0614

This time we want to actually find the Eigenvalues and associated Eigenvectors of a.0635

So, a given matrix can have Eigenvalues and Eigenvectors associated with it. Okay. Now, what do we want. So, we want real numbers Λ and all of the variables x, which I will write in component form... x1, x2, such that, well, ax = Λx.0659

Well, a is (1,1,-2,4)... x is (x1,x2) = Λ × x1... Λ's x's, all these symbols everywhere... x2... okay.0694

When we actually multiply this out, we get the following system: x1 + x2 = Λx1, and we get -2x1 + 4x2 = Λx2... let us fiddle around with this a little bit.0714

Let me bring this over here, and this over here... and set it equal to 0, so I am going to write the equivalent version. It is going to be Λ - 1 × x1, right? I have Λ x1 - x1... I can pull out the x1 and I get Λ - 1 × x1 - x2 = 0.0742

I also get 2x1, moved it over to that side... + Λx2 - 4x2, which is Λ - 4 × x2 = 0. Right? Okay.0771

Now, take a look at this linear system right here. It is a homogeneous system, okay? 2 by 2.0794

Let me go back to my blue ink. This system has a non-trivial solution... remember the list of non-singular equivalences? It has a non-trivial solution, if and only if the determinant of the coefficient matrix is equal to 0.0802

So, if I have -- no, this one I definitely want to write as clear as possible... start again -- coefficient matrix is (Λ - 1, - 1, 2, Λ - 4)... the determinant = 0.0820

This homogeneous system has the non-trivial solution and the determinant is 0. Well, the determinant is this. The determinant of a 2 by 2 is this × this - that × that.0843

So, I end up with Λ - 1 × Λ - 4 -2 = 0. I get Λ² - 5Λ + 4 + 2. I get Λ² - 5Λ + 6 = 0. All I am doing is following the map... that is all I am doing.0856

Let me rewrite this, there are too many lines here... + 6 = 0... rewrite it again... go to red... Λ² - 5Λ + 6 = 0.0891

This factors into Λ - 2, Λ - 3, this implies that Λ1 = 2, Λ2 = 3. These are my Eigenvalues associated with that matrix, and all I did was solved this homogeneous system, right? Okay.0910

Now we want to find the Eigenvectors associated with the Eigenvalues. Well, I have 2 Eigenvalues, so I am going to be solving 2 systems to find the associated Eigenvectors.0934

Let me show you what I just did here. I started with ax = Λ × x. Bring this over here and set it equal to 0.0948

Λ... let us make the Λ look like a Λ and the x look like an x... Λx - a × x is equal to 0. So, let me put the 0 vector over here... so I am working on the right because that is our habit.0964

Let me factor out the x... well, Λ ×... we are talking about matrices here, so since this is a matrix a, Λ is a scalar... I just multiply that scalar by the identity matrix.0985

Remember what the identity matrix is... it is just that matrix with 1's all along the diagonals, because I need matrix subtraction to be defined.0997

This is the equation that I solve. So, for every Λ that I get... I put it into this equation which is the thing that I had in the previous page.1010

I put it into this equation, I solve the homogeneous system, I get my Eigenvectors for that Eigenvalue, and then I do the same for 3.1016

So, now let us actually go through the process. Okay. This if you recall, was this. Λ - 1 × x1 - x2 = 0, and it was 2x1 + Λ - 4x2 is equal to 0.1030

So, if I were going to take me Λ = 2 Eigenvalue, I would put this 2 in here, and solve the associated homogeneous system.1058

So, I would get 2 - 1 is 1. So, I would get x1 - x2 = 0, and I would get 2x1 + Λ is 2... 2 - 4 is -2, so it is -2x2 is equal to 0.1070

Well, that x1 is equal to x2 so which means that I can choose x2 anything that I want, so let us just call it R.1098

Therefore, any vector of the form (R,R), is an Eigenvector for this Eigenvalue 2.1111

Okay. Alright. What this means is if I take a, and if I take any vector of the form (R,R)... (1,1), (2,2), (3,3), (4,4)... all I end up doing is I end up multiplying it... that is what this is telling me.1125

All vectors of this form, that have the same entry... when I multiply by the matrix a, all I do is end up doubling its length. That is what this telling me. Only vectors of this form are associated with this Eigenvalue.1144

Now, let us do the Λ = 3. Well, Λ = 3... we end up putting it back into that original equations, so that is 3 - 1 × x1 - x2 = 0.1160

We have 2x1 + 3 - 4, because 3 is our Eigenvalue, x2 = 0. We end up with 2x1 - x2 = 0... 2x1 - 2x2 = 0... This tells us that 2x1 = x2... x1 = x2/2.1179

Therefore, our vector x is, well, if x2 is equal to R, then x1 = R/2.1212

So, every vector of the form (R/2,R), like for example (4,2), (8,4), (16,8), (24,12)... those are the Eigenvectors associated with the Eigenvalue 3... that is going to actually end up equaling 3 × (18,9).1224

That means that if I take the matrix a which was given, and if I take some vector like (18,9), which is of the form (R/2,R), all I am going to do is I am going to multiply that vector by a factor of 3. That is what is happening here.1243

Okay. Let us move on, we are going to have a little bit of a definition here. We just did this, so now we are going to actually... this equation that we came up with that we solved to get the Eigenvalue, we are going to give it a special name.1265

So, definition, we will let a_ij be an n by n matrix... the determinant of Λ × the identity matrix - a matrix, which is equal to this following determinant in symbolic form, Λ - (a1,1) - (a1,2) - (a1,3) - (a1,n).1283

Then, of course, - (a2,1) - (an,1) - (an,2)... Λ - (a2,2), Λ - (a_n,n)...1344

This determinant is called the characteristic polynomial... characteristic polynomial of a.1367

Now, when I set that characteristic polynomial, in other words the determinant of Λ × in - a, when I set it equal to 0, it is called the characteristic poly... it is called the characteristic equation -- I am sorry.1388

That is the polynomial... it is the characteristic equation... is the characteristic equation of a.1410

Okay. Let us do an example. Let a = 1 - 2, 1 - 2, 1, 1, 0, -1, 4, 4, -5.1421

Okay, so, we want to find the determinant of Λ × in - a, which is... so you see what this looks like, let me actually do this... this is 3 by 3.1446

So, it is going to be Λ × in - 1 - 2, 1, 1, 0, - 1, 4, 4, -5...1468

We are going to take the determinant of this thing... which means I have Λ, 0, 0, 0, Λ, 0, 0, 0, Λ, -1, -2, 1, 1, 0, -1, 4, 4, -5.1491

I end up with Λ, -1, - -2 is 2, -1, 0 - 1 is -1, Λ - 0 is Λ, - -1 is 1, -4, -4, Λ + 5.1520

Then I take the determinant of that, and when I actually end up doing that and going through it, I end up with Λ³ + 4Λ² - 3Λ - 6.1552

This is a characteristic polynomial. If I want to find the Eigenvalues, I have to find the roots of this characteristic polynomial. I set it equal to 0, that will give me the Eigenvalues.1572

When I get the Eigenvalues, I put it back into this form and I solve the homogeneous system to get the Eigenvectors. We will do more of that in just a minute.1584

Okay. So, let us close off this section with just a theorem.1594

An n by n matrix is singular... does not have an inverse if and only if 0 is an Eigenvalue of a.1604

In other words, if 0 was not an Eigenvalue of matrix a, that matrix is non-singular. It has an inverse, so this is one item that we are going to add to our list of non-singular equivalences.1628

We had 9 of them, now we have are going to have 10. We are going to add a 10th item.1640

Okay. That 10th item added to the list of non-singular equivalences... 0 is not an Eigenvalue of a... that is the same as saying that a is non-singular.1646

It is the same as saying that the determinant exists... all of those things that -- you know -- we have for those... for that list. So, this is the 10th equivalence.1665

Another theorem. The Eigenvalues of a are the real roots of the characteristic polynomial... okay. So, you might have a polynomial fifth degree... it has 5 roots.1681

Well, there is no guarantee that all of those 5 roots are going to be real. Some of them might become complex... if they are complex, they are going to come in complex conjugate pairs.1717

So, if you know one of them is complex, you know 2 of them are complex. That means only 3 of them can be real.1725

If you have 3 of them that are complex, that means the 4th is also complex. That means only 1 of them is going to be real. For caller algebra, polynomial equations, and solutions to polynomial equations, roots where the graph hits the x axis.1731

So, when you have a characteristic polynomial, it is the real values that are the Eigenvalues of that associated matrix. Okay.1748

Let us try something here. We will try an example. We will let a = (2, 2, 3, 1, 2, 1, 2, -2, 1,)... this is our matrix a.1756

Our characteristic polynomial when we set it up... again, this is something that you can do on the mathematical software... our characteristic polynomial... is Λ³ - 5Λ² + 2Λ + 8.1781

When we actually factor this out, we end up with Λ1 = 2... Λ2 = 4. Λ3 = -1. The degree of the polynomial is 3, which means we have 3 roots. We found those 3 roots... (2,4,-1), they are all real.1804

All of these are Eigenvalues. Okay. Now, let us find the Eigenvectors associated with these Eigenvalues. Let us actually find a specific Eigenvector, not like we did last time where we found a general Eigenvector.1832

Okay. In so doing, we are going to solve, of course, when we do... so we are going to solve this... Λ × i3 - the matrix a × x = 0.1845

This is the equation -- okay, this is not going to work... too many lines all over the place... this is too strange, let us try this again -- Λ × i3 - a... x = 0, the vector.1860

We are going to solve this equation, homogeneous system in order to find the associated Eigenvector.1882

So, 4Λ = 2... I get the following system... 0, -2, -3, 0, -1, 0, -1, 0... I am hoping to god my arithmetic is correct here... -2, 2, 1, 0... 2302 when I subject it to reduced row echelon... I want you to see at least one of them.1889

We end up with 1, 0, 1, 0, 0, 1, 3/2, 0, 0, 0, 0, 0. So, leading entries here and here. Leading entry not there. Therefore I can take x3 = R, x2 = well, x2 = -3/2R, and x1 = -R.1928

Well, I can set R to anything, so why do I not just like R = 1. So, a particular Eigenvector... a specific Eigenvector is -1, -3/2, and 1. This is an Eigenvector associated with the Eigenvalue 2.1962

There are an infinite number of them, just different values of R. That is all it is.1985

Okay. When I take the Λ, when my Eigenvalue is a -1, I get some matrix, I subject it to reduced row echelon form, and I get a vector of the form -R, 0, R.1991

Well, let us just take specific values. Let us just take R = 1, so (1,0,-1). This Eigenvector is associated with this Eigenvalue, Λ2.2010

Now, we will do Λ = 4... Λ3 = 4 -- get some notation here, make sure that I am correct -- Λ1, Λ2, and Λ3 = 4... yes, we are correct.2024

For that Eigenvalue, we end up with the general 4R, 5/2R, R, which gives us 4, 5/2, and 1.2040

So, given a certain matrix, we can find its Eigenvalues, we can solve the homogeneous system to find its Eigenvectors, so we had a nice structure developing here for a particular matrix.2060

So, let us do a quick recap. We have the definition of Eigenvalue, Eigenvector... If I have a matrix a, if I have a vector that I multiplied by, if what I end up with is some scalar multiple of that vector, well the scalar multiple is called an Eigenvalue.2073

The vectors that actually satisfy this condition are called Eigenvectors associated with that Eigenvalue.2101

I solve this for 0. I move this over to that side, and I end up with 0 = Λx - ax, let me just bring this 0 over here, so I end up with Λ × in - a... × x = 0.2111

Okay. For this to have a non-trivial solution, well, the determinant of this thing Λ i_n - a has to equal 0.2139

So I take the determinant of that... this matrix that I get, set it equal to 0, that gives me the Eigenvalues. Okay?2167

That is the characteristic polynomial. This is the characteristic equation, and for each Λ_i, for each Eigenvalue that I get, for each root, real root, of the characteristic polynomial.2177

We put each Λ_i back into this equation and we solve that homogeneous system and find our basis... our vectors that satisfy that.2195

We find the associated Eigenvectors by solving Λ_i × i_n - a × x = 0.2208

So, we have a matrix a. We set this up, we take the parameter Λ × the identity matrix... we subtract from it the a matrix.2233

Now, you can do it either way. You can go a - Λ, it does not matter. I did Λ - a because I like Λ to be positive, that is just a personal choice of mine.2246

You end up with this equation. Well, you take the determinant of the matrix that you get... this thing Λ × in - a, you set it equal to 0, you find the roots... those are the Eigenvalues.2256

When you take each of those Eigenvalues and put it in turn back into this equation, solve the homogeneous system to get the associated Eigenvector with that respective Eigenvalue.2269

So, that takes us through the basic structure of Eigenvalues and Eigenvectors. In our next lesson we are going to continue on and dig a little deeper into the structure of these things.2280

Thank you for joining us at Educator.com, we will see you next time.2289