Linear Algebra Homogeneous Systems

Welcome back to Educator.com and welcome back to linear algebra.0000

We have been investigating the structure of vector spaces and subspaces recently getting a little bit more deeper into it to understand what it is that is actually going on in the vector space.0004

Today we are going to continue to talk about that of course, and we are going to talk about homogeneous systems.0015

As you gleaned already, homogeneous systems are profoundly important, not only in linear algebra, but they end up being of central importance in the theory of differential equations.0022

In fact, at the end of this lesson, I am going to take a little bit of a digression to talk about the nature of solutions in the field of differential equations as related to linear algebra.0033

Linear algebra and differential equations, they are very, very closely tied together. So, it is very difficult to separate one from the other.0043

Let us go ahead and get started.0050

Okay, so, we know that the null space of... so what we have is this homogeneous system, for example, ax = 0, where a is some m by n matrix.0054

This is just the matrix form of the normal linear system that we are used to, in this case a homogeneous system because everything on the right hand side is 0.0072

x is, again, is a vector, so it is a n by m, n by 1 vector that is a solution to this particular equation.0080

Now, we know that given this, we know that the null space, or the space of all solutions to this equation.0091

In other words, all of the vectors that satisfy, all of the vectors x that satisfy this equation is a subspace of RN.0102

Well, very important question, a very important problem is finding the basis for this subspace.0120

If you recall what a basis is, a basis is a set of vectors that actually spans the entire space that we are dealing with and is also linearly independent.0137

So, you remember, you might have a series of vectors that spans a space, but it might not be linearly independent.0148

Or, you might have some vectors that are linearly independent, but they might not span the space.0153

A basis is something that satisfies both of those properties.0158

Again, it spans the space that we are talking about, and the vectors in it are linearly independent.0162

In this case, the space that we are talking about is the null space. The space of solutions to the homogeneous system ax = 0.0167

So, the procedure for finding a basis for the null space of ax = 0, where is a n by m, of course I will not go ahead and mention that.0177

The first thing we do, well, we solve ax = 0 by Gauss Jordan elimination to reduced row echelon form, as always.0210

Now, if there are no arbitrary constants, in other words, if there are no columns that have no leading entries, then, the null space equals the set 0 vector.0229

What that means is that there is no basis, no null space. There is no null space, essentially -- well, there is, it is the 0 vector, but there is no basis for it.0263

In other words, there is no collection of vectors. Okay.0272

The dimension of the null space, which we called the nullity if you remember, equals 0.0277

Okay. Our second possibility is if arbitrary constants do exist after we reduce it to reduced row echelon form.0293

What that means is that if there are columns that do not have leading entries, those are... the x... the values corresponding to those columns, let us say it is the third and fifth column, so x3 and x5, I can give them any value I want. That is what the arbitrary constant means.0302

So, if arbitrary constants exist, then, write the solution x = c1x1 + c2x2 + ... + ckxk, however many of these vectors and constants there are.0321

Well, once you do that, the set s, which consists of this x1, this x2, all the way up to xk... x1, x2... xk... is a basis for the null space.0358

Let me write space over here. So, again, what we do when we want to find the basis of a null space of a homogeneous system.0390

We solve the homogeneous system with reduced row echelon form.0402

We check to see if there are no columns that do not have a leading entry, meaning if all of the columns have a leading entry, there are no arbitrary constants.0407

Our null space is the 0 vector. It has no basis, and the dimension of the null space, the nullity in other words equals 0.0414

Let me go ahead and put nullity here. Nullity is the dimension of the null space. In other words it is the number of vectors that span that space, it is 0.0423

If arbitrary constants do exist, meaning if there are columns that do not have a leading entry, then we can read off the solution for that homogeneous system.0435

We can write it this way, and the vectors that we get end up being our basis.0442

Let us do an example, and as always, it will make sense, hopefully.0450

Let us see, so our example... find, not find the... find a basis.0458

So, there is usually more than one basis. Find a basis. Basis is not necessarily unique.0476

Find a basis for, and the nullity of the null space for the following system.0484

(1,1,4,1,2), (0,1,2,1,1), this is a one, sorry about that, it looks like a 7, (0,0,0,1,2), (1,-1,0,0,2), and (2,1,6,0,1).0506

x1, x2, x3, x4, x5 = (0,0,0,0,0).0538

This is our homogeneous system. We are looking for this right here.0552

We are looking for all vectors, x1, x2, x... all vectors whose components x1, x2, x3, x4, x5... that is what we are looking for.0558

It is a solution space. So, we are going to try to find the solution, and when we do, we are going to try to write it as a linear combination of certain number of vectors.0566

Those certain number of vectors are going to be our basis for that solution space.0574

Okay. So, let us go ahead and just resubmit it to reduced row echelon form.0578

What we end up getting is the following... (1,0,2,1,0), (0,1,2,0 -- oops, I forgot a number here.0586

(1,0,2... let me go back... what I am doing here is... I am going to take the augmented matrix.0608

I am going to take this matrix and I am going to augment it with this one, so let me actually rewrite the whole thing.0627

It is (1,1,4,1,2,0), (0,1,2,1,1,0), (0,0,0,1,2,0), (1,-1,0,0,2,0), (2,1,6,0,1,0). Okay.0632

So, this is the matrix that we want to submit to reduced row echelon form.0655

I apologize, I ended up doing just the matrix a.0659

When we submit it to reduced row echelon form using our software, we get the following.0663

(1,0,2,0,1,0), (0,1,2,0,-1,0), (0,0,0,1,2,0), (0,0,0,0,0,0), and of course the last... not of course... but... there is no way of knowing ahead of time.0669

What we end up with is something like this. So, now let us check to see which columns actually do not have leading entries.0695

This first column does, this second column does, this third column does not... so, third.0702

The fourth column has a leading entry, it is right there... the fifth column... okay.0711

So the third and the fifth columns, they are arbitrary constants.0719

In other words, these of course correspond to x1, x2, x3, x4, and x5.0723

So, x3 and x5, I can let them be anything that I want.0732

I am going to give them arbitrary parameters. So, let me go ahead and write x3... I am going to call x3 R.0739

It just stands for any number.0748

And... x5 is going to be S, any parameter. Columns with non-leading entries.0752

Now, when I solve for, like for example if I take x1, it is going to end up being the following.0759

x1 = -2R - S, and here is why... that row... well this is x1 + 2x3 + x5.0767

So, x1 = -2x3 - x5.0785

Well, -2x3 -x5 is -R - S, that is where this x1 comes from.0793

I am just solving now for the x1, x2 and x4. That is where I am going to get the following equations.0800

Then, I am going to rewrite these in such a way that I can turn them into a vector that I can read right off.0809

So, x2 here is equal to -2R + S, because here, this is 2R over here, this is -S, it becomes +S when I move it over.0815

x5, our last one, is equal to -2S, and here is the reason why: because it is -- I am sorry, this is x4 not x5, x5 is S.0828

x4 + 2s = 0, therefore x = -2s.0843

So, this is our solution. Notice our infinite number of solutions, now we are going to rewrite this in order... x1, 2, 3, 4, 5 as opposed to how I wrote it here, which was just reading it off, make it as simple as possible.0851

So, let me move forward. What I end up with is x1 = -2R - S, x2 = -2R + S, x3 = R, x4 = -2S, and x5 = S.0866

So, now x1, 2, 3, 4, 5... this is the vector that I wanted. This is the arrangement.0896

Well, take a look. R and S are my parameters. This is equivalent to my following.0901

I can write this as this vector, which is just x1 x2 in vector form... is equal to... I pull out an R and this becomes (-2, -2, 1, 0, 0).0908

That is what this column right here is... -2R, -2R, R, 0, R, 0, R.0925

Then this one right here, I pull out an S... +S × (-1,1,0,-2,1). There you go.0935

This particular homogeneous system has the following solution, as it is some arbitrary number × this vector + some arbitrary number × this vector.0950

Well, these are just constants. Any sort of number that I can imagine from the real numbers... any number at all, so this is a linear combination.0960

Therefore, that vector and that vector form a basis for our solution space.0970

Therefore, we have our basis is the set (-2,-2,1,0,0), and (-1,1,0,2,1).0977

This is the basis of the null space. Well, the dimension is, well how many are there... there are 2.0997

So, the null space has a dimension 2. The nullity is 2.1005

So, notice what it is that we actually had. We had the system of 1, 2, 3, 4, 5... 1, 2, 3, 4, 5, we had this 5 by 5 system.1016

So, R5 all the way around. Well, within that 5 dimensional space, 2 of those dimensions are occupied by solutions to the homogeneous system.1026

That is what is going on here. If you think about it, what you have is any time you have 2 vectors, you have essentially a plane, what we call a hyper-plane.1040

Because, you know, we are talking about, you know, a 5-dimensional space.1051

But, that is all that is going on here. So, we solve a homogeneous system, we reduce the solution to this, and because it is a linear combination of vectors, the 2 vectors that we get actually form a basis for our solution space.1054

We could how many vectors we get to and that is the dimension of our null space, which is a subspace of all of it.1068

Let us do another example. Let us take the system 2, let us go (1,0,2), (2,1,3), (3,1,2) × x1, x2, x3 = (0,0,0).1081

We set up the augmented matrix for the homogeneous system which is (1,0,2,0, (2,1,3,0), (3,1,2,0), just to let you know, that is what the augment is.1110

We subject it to reduced row echelon form, and we get the following... We get (1,0,0,0), (0,1,0,0), (0,0,1,0).1130

There are no columns here that do not have leading entries. Basically what this is saying is that x1 is 0, x2 is 0, x3 is 0. That is our solution.1144

We have x1 = 0, x2 = 0, x3 = 0, all of these are equivalent the vector = 0 vector.1155

When we have the 0 vector as the solution space, only the trivial solution, we have no basis for this system.1168

The nullity is 0. The dimension of the solution space is 0. The only solution to this is the 0 vector.1176

Okay. Now, let us go ahead and talk... take a little bit of a digression and talk about the relation between the homogeneous and non-homogeneous systems.1186

So, we have been dealing with homogeneous systems, but as we know there is also the associated non-homogeneous system. Some vector v.1197

So, that is the non-homogeneous and this is the homogeneous version of it.1206

0 on the right, or some b on the right. There is a relationship that exists between the two. This relationship becomes profoundly important not only for linear algebra, but especially for differential equations.1213

Because, often times, we will have a particular solution to a differential equation, but we want to know some other solution.1224

But, maybe we cannot actually solve the equation. Believe it or not, the situation comes up all the time.1232

As it turns out, we do not have to solve the equation. We can solve a simpler equation. The homogeneous version, which is actually quite easy to solve for differential equations, and here is the relationship.1237

Okay. If, some vector x_b is a solution, some solution that I just happen to have, to the system ax = b, and x₀ is a solution to the associated homogeneous system, okay?1248

Then, if I add these two, x_b and x₀, then the some of those 2 as a solution is also the solution. I should say also.1285

Is also a solution to ax = b. So, if I happen to have the non-homogeneous system and if I happen to know some solution for it, and if I happen to know some solution to the homogeneous system, which is usually pretty easy to find, then I can add those 2 solutions.1306

That is a third solution, or that is a second solution to ax = b.1322

That is actually kind of extraordinary. There is no reason for it to actually be that way, and yet there it is.1328

What is more, every solution to the non-homogeneous system, ax = b, every single solution can be written as a particular solution, which we called x_b + something from the null space.1337

Symbolically, x_b + x₀, what this says is that when I have the system ax = b, and I happen to know a solution to it and I also happen to know the solution to the homogeneous system, all the solutions.1389

In other words, I happen to know the null space every single solution to the non-homogeneous system consists of some particular solution to the non-homogeneous system plus something from the null space.1406

So, if I have, I mean obviously I am going to have maybe an infinite number of solutions to the null space, all I have to do is take any one of those solutions from the null space, add it to the particular solution that I have for the non-homogeneous, and I have my collection of every single solution to the non-homogeneous system.1422

That is amazing. What makes this most amazing is that usually when we are dealing with a non-homogeneous system, more often than not, you can usually just guess a solution.1441

We can check it, and if it works, that is a good particular solution.1450

Then, what we do is instead of solving that equation, we actually end up solving the homogeneous system, which is pretty easy to solve.1453

Then we just take any solution we want from the homogeneous system, add it to the solution that we guessed, and we have any solution that we want for the particular problem at hand.1459

That is extraordinary. It is a technique that you as engineers and physicists will use all the time.1470

Very, very important for differential equations. Okay.1477

We will go ahead and stop it there today. Thank you for joining us at educator.com, we will see you next time.1480