Linear Algebra Orthogonal Complements, Part I

Welcome back to Educator.com and welcome back to linear algebra.0000

Today we are going to be talking about orthogonal complements.0004

So, rather than doing a preamble discussion of what it is, let us just jump into some definitions and it should make sense once we actually set it out in a definition form.0008

Okay. So, let us start with our definition. It is a little long, but nothing strange.0019

Let w be a subspace of RN, so we are definitely talking about N-space here. Okay.0032

A vector u which is a member of RN is said to be orthogonal to w, so notice orthogonal to w as a subspace.0049

Orthogonal to an entire subspace, if it is orthogonal to every vector in that subspace.0070

Okay. The set of all vectors in RN that are orthogonal to all vectors in w is called the orthogonal complement of w.0094

And... is symbolized by w with a little perpendicular mark on the top, and they call it w perp.0142

The top right. Okay. SO, let us look at this definition again.0157

So, w is a subspace of RN, okay? So it could be dimension 1, 2, 3, all the way up to N, because RN is a subspace of itself.0161

A vector u in RN is said to be orthogonal to that subspace if its orthogonal to every vector in that subspace.0170

So, the set of all vectors that are orthogonal, we call it the orthogonal complement of w.0180

It is symbolized by that w with a little perpendicular mark on the top right, called w perp.0187

Let us give a little bit of a picture so that we see what it is we are looking at.0194

So, let us just deal in R3, and let us say that, so let me draw a plane.0199

As you know, a plane is 2-dimensional so it is in R3, and then let me just draw some random vectors in this plane. Something like that.0205

Well, if I have some vector like that, which is perpendicular to this plane, so this plane... let us call that w.0214

So, that is some subspace of R3, and again, let me make sure that I write it down... so we are dealing with R3.0226

This two dimensional plane is a subspace of R3, and every single vector in there is of course... well, it is a vector in the plane.0231

Then, if I take this vector here, well every single vector that is perpendicular to it is going to be parallel to this vector, right?0241

So, when we speak about parallel vectors, we really only speak about 1 vector.0248

So, as it turns out, if this is w, this vector right here and all of the scalar multiples of it, like shortened versions of it, long versions of it, this is your w perp.0253

Because, this vector, any vector in here, is going to end up being perpendicular to every one of these vectors. This is the orthogonal complement of that.0268

So, it helps to use this picture working in R3, and working with either dimension 1 or 2, because we can actually picture it.0279

For something like R4 or R5, I mean I can go ahead and tell you what it is that you will be dealing with.0287

So let us say in R4 you have a subspace that is 2-dimensional, that is some kind of plane so to speak in R4.0294

Well, the orthogonal complement of that is going to be every vector that is going to be perpendicular to that 1 or 2 dimensions, that is actually going to end up being 2-dimensional.0302

The idea is we have this subspace and we have a bunch of vectors that are orthogonal to every vector in that subspace.0314

The set of all of those vectors that are orthogonal are called the orthogonal complement. That is all that it means.0324

Okay. Let us actually do a little bit of an example here.0331

So, let us say... well actually, you know what, let us just jump into a theorem and we will get into an example in a minute.0338

So, theorem... let w be a subspace of RN... okay.0348

Then, aw perp is a subspace of RN.0371

So, if w is a subspace, w perp, its orthogonal complement, is also a subspace.0383

We do not have to go through that procedure of checking whether it is a subspace.0388

And... it is interesting... that the intersection of w and w perp is the 0-vector.0393

So, again, they are subspaces so they have to include the 0 vector, both of them, but that is the only thing common between the two subspaces of w and w perp, its orthogonal complement.0402

The only thing they have in common. They actually pass through the origin.0413

Okay. So, now let us do our example.0419

Let us see. We will let w be a subspace of, this time we will work in R4, with basis w1, w2.0425

So, w1, w2, these two vectors form a basis for our subspace w.0447

And... w1 is going to be 1, 1, 0, 1, and I have just written this vector in horizontal form without the... not as a list without the commas, it does not really matter.0453

w2 is going to be the vector 0, -1, 1, 1, 1.0466

So, you have 2 vectors, they form a basis for the subspace w in R4.0474

Now, our task is find a basis for the orthogonal complement, w perp. Find a basis for the subspace of all of the vectors that are orthogonal to all of the vectors in w, that has these 2 vectors as a basis.0479

Okay, well, so, let us just take some random... okay... so we will let u, let us choose u equal to some random vector in R4.0498

a, b, c, d, we want to be as general as possible... a, b, c, d.0512

Well, so we are looking for the following. We want... actually, let me see, let this be -- I am sorry -- let this be a random vector in the orthogonal complement.0518

Okay. So, we are just going to look for some random vector, see if we can find values for a, b, c, d.0532

We are going to take a vector in the orthogonal complement, and we know that this is going to be true.0538

We know that because w perp and w are orthogonal to each other, we know that u · w1 = 0.0543

We know that u · w2... let me make this dot a little more clear, we do not want that.... is equal to 0, right?0555

So, because they are orthogonal complements, we know that they are orthogonal, which means that their dot product is equal to 0.0570

Well, these are just a couple of equations, so let us actually do this.0578

So, if I do u · w1, I get a + b + 0 + b = 0.0582

Then, if I do u · w2, I get 0 - b + c + d = 0.0595

When we solve this using the techniques that we have at our disposal... I am going to go ahead and do it over here.0608

So, this is just a homogeneous system, you set up the coefficient matrix, reduced row echelon form, the columns that have... that do not have a leading entry, those are your free variables... r, s, t, u, v, whatever you want.0618

Then you solve for the other variables that do have leading entries0629

When you do this, you end up with the following. You get a, b, c, and d, the vector takes on the form R × -1, 1, 1, 1, 0, + s × -2, 1, 0, 1.0633

So, those two vectors form a basis for the orthogonal complement w perp.0655

Therefore, we will set it up as c -- set notation, let me just write it and again -1, 1, 1, 1, 0... comma, -2, 1, 0, 1... is a basis for w perp.0663

So, that is it. We started with a basis of two vectors in R4.0682

Then, just by virtue of the fact that we know that the orthogonal complement is going to be orthogonal to every single vector in this, so it is certainly going to be orthogonal to these two... I pick a random vector in this orthogonal complement.0689

I write my equation... orthogonal just means the dot product equals 0, get a homogeneous system.0702

I solve the homogeneous system and I set it up a way where I can basically read off the basis for my solution space of this homogeneous system, which is the basis for, in this particular case, based on this problem, the orthogonal complement.0708

Nice, straight forward, nothing we have not done. We have seen dot product, we have seen homogeneous systems, we have seen basis, everything is new.0725

Now we are just supplying it to this new idea of 2 subspaces being orthogonal to each other. Being perpendicular to each other.0733

Of course, perpendicularity, of course you know from your geometric intuition, only makes sense in R2 and R3, which is why we do not use the word perpendicular, we use the word orthogonal, but it is the same thing in some sense0742

So, you might have a 6-dimensional subspace being orthogonal to a 3-dimensional subspace "whatever that means".0750

Well, geometrically, pictorially, we do not know what that means. We cannot actually picture that. We have no way of representing it geographically.0760

But, we know what it means algebraically. The dot product of two vectors in those spaces is equal to 0.0767

Okay. One of the things that I would like you to notice when we had R4, you notice that our w had dimension 2.0774

Its basis had 2 vectors, dimension 2... and you noticed when we had w perp, the orthogonal complement, we ended up with 2 vectors as a basis, also in dimension 2.0782

Notice that the dimension of the subspace w + the dimension of its orthogonal complement added up to 4, the actual dimension of the space. That is not a coincidence.0793

So, let us write down a theorem... Let w be a subspace of RN.0808

Then, RN, the actual space itself is made up of 2 w + w perp. SO, let me talk about this thing.0826

This little plus sign with a circle around it, it is called a direct sum, and I will speak about it in just a minute.0836

Okay. Essentially what this means is... we will have to speak a little bit more about it, but one of the things that it means is that w intersect w perp, the only thing they have in common is like we said before... the 0 vector.0846

These are both subspaces, so they have to have at least the 0 vector in common. They do not share anything else in common.0860

Okay. Yet another theorem, and I will talk about the sum in just a moment, but going back to the problem that we just did, this basically says that if I take the subspace w and its orthogonal complement, and if I somehow combine them -- which we will talk about it in a minute -- we will actually end up getting this space itself, the 4-dimensional space.0870

So if I had a 6-dimensional space and I know that I am dealing with a subspace of 2-dimensions, w, I know that the orthogonal complement is going to have dimension 4 because 2 + 4 has to equal 6, or 6 - 2 = 4, however you want to look at it.0890

Okay. Let us do another theorem here. Just a little bit of an informational theorem, which will make sense.0906

If w is a subspace of RN, then w perp perp = w.0915

This just says that if you take an orthogonal complement of some subspace and you take the orthogonal complement of that ,you are going to end up getting the original subspace.0932

Nothing new about that, I mean like a function... if you take the inverse of a function and then you take the inverse of the inverse, you get the function back. That is all it is. Very, very intuitive.0940

Okay. Now, let us discuss this symbol some more. This + symbol.0950

So, when we write... this direct sum symbol -- I am sorry -- when we write w + w perp, these are subspaces, okay?0956

We do not... this is a symbol for the addition of subspaces, we are not actually doing the operation of addition.0970

What this means, so this symbolizes the addition of a subspace. This whole thing is a space.0976

What this means is that something... it means that if I have some w1 -- no, let me make it a little bit more general, there are going to be too many w's floating around.0987

So, if I have a, this direct sum symbol, plus b, okay?1003

It is the space made up of vectors v, such that v is equal to some a + b, where the vector a comes from the space a and the vector b comes from the space b.1009

So, this symbol, this direct sum symbol... it means if I take some vector in the subspace a... and a vector in the subspace b, and i actually add those vectors like I normally would? I am going to get some vector.1040

Well, that vector belongs to this space. When I see this symbol, I am talking about a space. In some sense what I have done is I have taken 1 whole space and I have attached another space right to it.1055

In the case of the example that we did, we had a 2-dimensional subspace, we added a 2-dimensional orthogonal complement to it, and what I got was the entire space R4.1070

That is what is happening here. That is what this direct sum symbol means. It symbolizes the addition of spaces, the putting together of spaces.1078

But, these vectors are spaces that contain individual vectors.1088

Okay. Let us see. Let us do a little bit further here. Let us take R4, expand upon this...1097

Let us let w = ... well not equal, let us say it has a basis.1112

Let w have... has a basis vector (1,0,0,0), and (0,1,0,0).1123

So, let us say that w is the subspace that has these 2 vectors as a basis.1136

So, it is a 2-dimensional subspace, and we will let w perp have basis (0,0,1,0)... (0,0,0,1)... okay, as a basis.1141

Now, if I take w, the direct sum w perp, well, that is equal to R4... right?1165

So, a vector in R4... let us say for example... which is let us just say some random vector (4,3,-2,6), which is a vector in R4, it can be written as... well, it can be written as a vector from this subspace + a vector from this subspace.1180

Just like what we defined, that is what a direct sum means. This w + the w perp, means take a vector from here, add it to a vector from here, and you have a vector in the sum, which happens to be R4.1207

We can write it as (4,3,0,0)... this vector right here is in the space w.1220

We can add it to the vector (0,0,-2,6), which is a vector in w perp.1231

What is nice about this representation, this direct sum representation is that -- let us see -- this representation is unique.1240

So, when I write a particular vector as a direct sum of 2 individual subspaces, the way that I write it is unique. There is no other way of writing it.1262

Okay. So that gives us a nice basic idea of orthogonal complements to work with.1273

We will continue on next time some more with orthogonal complements.1277

Thank you for joining us at Educator.com and we will see you for the next instalment of linear algebra. Take care.1280