Multivariable Calculus Directional Derivative

Hello and welcome back to educator.com and multivariable calculus.0000

So, today we are going to be talking about the directional derivative. Let us just jump right on in and start talking about it.0004

Let us go back to first... single variable calculus and let us talk about just a basic function.0012

Let f = x², well we know that df/dx = 2x, so the derivative is a rate of change.0019

It is telling you that if I change a little bit in the x direction, that the function is going to change a certain amount in the y-direction. That is what this means, that is all it means, at a given point.0031

In this particular case, let us say the point we pick is x = 3.0045

So, at x = 3, df/dx = 2 × 3, it equals 6. That means if I am at x=3, and if I actually move to the right a little bit, or to the left a little bit, the extent to which the function is actually going to change is measured by this thing right here, the derivative.0050

That is what the derivative does, it is a rate of change.0063

Now, notice... well, let me write all of this out here so that we have it available.0077

This is a rate of change. It means the rate at which f changes with each small change in x.0084

Sometimes we get so wrapped up in actually just doing the mathematics and getting through the symbolism, that we forget what is really happening. I mean that is what it is.0117

It is a measure of how the function changes when you change the other variable at a given point.0125

Of course since functions, curves, are kind of whacky, we are not talking about a straight line for the function itself, it is going to have different rates of change at different points along that curve. That is the whole idea.0130

But the change in f has a direction, which is implicit for functions of one variable.0146

That direction is the y direction, because that is it. I mean you have yourself a graph, okay?0178

In this particular case y = x² looks something like this. This is x and this is y.0183

Well, at a particular point if I move a little bit in this direction, my graph is going to change, you know, by a certain amount.0191

There are only two directions, the x direction - this way, that way - and the y direction. There is really no place else to go.0200

Our domain is the x-axis. This is a function from R to R. From the real number line to the real number line.0206

I only have one direction to move in, and the direction in which the function moves, just this way... it is implicit in the fact that this is a function of a single variable.0216

Now, let us go this way. So... let me actually just write this down. So as x changes in this direction, y changes... or not y, let us call it f, I do not necessarily... this is an actual function, the value of the function, we say y but it is actually the function.0228

So f changes in this direction that is it, there are only 2 directions available, and those directions are implicit on the fact that we are dealing with a function of a single variable.0256

Again, we do not have a lot of choices. This way, this way, or this way, this way.0268

Now, let f(x,y) = x² + y². Okay? So, now we have a function of 2 variables. Now our domain is the x,y plane.0272

We have a whole bunch of choices for x and y. The functional value, do not think of it in terms of the z value. Think of it just as the function.0289

In other words if I have 2 here and 3 here, my function is going to be a certain value.0302

If I change x a little bit and if I change y a little bit, my function is going to change accordingly.0308

Well, we have a way of finding the derivative of the function of these 2 variables. Let me just write the mapping representation.0313

This is a function from R2 to R. From 2-space to 1-space.0320

Well, the derivative which I am going to notate as d(f), capital D, the derivative of this function, well we know what that is - that is just the gradient.0326

That is what the gradient is, the derivative of a function of several variables... equals the gradient of f, okay?0335

That is equal to, well, we know that is df/dx, and that is df/dy. This is a vector, okay?0343

df/dy, which in this case is going to be 2x and 2y.0354

That means, if I pick a point in 2-space from my domain, okay? Let us just actually say that this is our domain.0362

If I pick this particular point and then I take the derivative of the function, the gradient, I am going to end up with a vector at that point if I actually evaluate this.0372

In other words if I put the x in there, and the y in there, I am going to have some vector pointing in some direction.0384

So, that is what is happening here. We have a change in f as 2 directions change.0391

What this gradient vector does is it measures, as I change in the x-direction, the function is going to change a little bit, and as I change in the y direction, it is going to change a little bit.0412

Well, this gradient vector is a measure of the combined change, if you will, of the x and the y. That means if I am at this point and I move in the direction of the gradient vector, how is the function going to change in that direction.0424

Now we have a little bit of a complication... well, not necessarily a complication, well, I mean, it is a complication but it is a really great complication.0442

Now our domain is 2-dimensional. We do not have to stick with just this direction. The only reason we have the directional derivative is because the x and y axes are our reference axes.0451

When we are dealing with coordinates we have to have a point of reference. Our coordinates x axis, y axis, y axis, whatever, those are our reference points. Our reference lines, which is why we take derivatives with respect to them.0462

But the fact of the matter is, if I am at this point, and I want to know how the function changes if I move in any direction, what if I want to move from this point, what if I want to move in that direction, or what if I want to move in this direction or that direction.0474

How is the function going to change, how is the value of the function going to change if I move in any one of these other directions instead of just the gradient.0488

The gradient is our basis if you will. It is a standard that is based on the fact that we have coordinate axes x, and y. It just gives us a nice little... but now that we have a domain, we can move it any direction that we want.0496

We are not confined to just move in one single direction, the x, like in single variable. That is what is happening here.0513

So, let me go ahead and finish off, but, so now we have a change in f as 2 directions change, x and y.0518

But, these are not the only 2 directions available for changes in the independent variables.0530

Ooh, wow, look at that crazy line. Let us make sure that we do not have those... in the independent variables x and y.0555

Again, they are independent, they do not rely on each other. It is not that x and y have to change uniformly, as one changes... one can move this way, one can move that way. They can move independently, they can change independently.0570

Now, that is why we take partial derivatives. The partial derivative as x changes, f changes, as y changes, f changes... again, the gradient is just a sort of a standard.0583

It is the gradient, it is the vector of the partial of f with respect to x, the partial of f with respect to y, it gives us 1 direction.0597

But, maybe I do not want to know, maybe I do not want to move in that direction. I want to know how the function changes if I move from that point in another direction. That is what we are doing.0605

Let us go ahead and move to another... so, if we want to know how a function changes as we move from a given point in any direction in the x,y plane, which is R2.0618

We use the gradient vector as a base -- I will put that in quotes -- as a base, and we form something called the directional derivative -- very, very important concept, the directional derivative.0674

So, again, let me just explain what is going on, just to make sure we have an idea of what is happening here.0712

When you take the gradient of a function of several variables, at a given point you are getting a vector that emanates from that point.0718

So, again, this is 2-space, let us say we have a function of 2 variables, let us say I have a certain point... if I take the gradient, which is df/dx, df/dy, and at that point when I put the x,y values in, I am going to get some vector.0725

This vector, it is giving me the rate of change of the function as I move from this point in that direction.0740

If I move in that direction, how is the function going to change? Well, that is just a standard. The fact of the matter is now that our domain is in 2-space, we can move in any direction that we want from this point. We are not limited.0748

This is what we want. We are going to use the gradient vector to find out what the derivative is in this direction, in this direction, in this direction, in this direction, in any number of directions in the x,y plane, or in x,y,z, space, or 4 space, or 5 space.0763

As it turns out, what we are going to define is valid in any number of dimensions. That is what we are doing.0778

What if I want the derivative that way, that way, this way, it is our basis. It is of 1 direction, and it is 1 direction based on the x and y axes, because they are our reference axes. We have to have a point of reference.0784

That is all that is going on here. So let us go ahead and write down what we have got.0797

So, let a be a unit vector, and I am going to write this in capitals, be a unit vector, very, very important.0803

The direction in which we choose is going to have to be... is going to be represented by a vector. It needs to be a unit vector.0813

Now, let p be a point in R2, for right now we will go ahead and define it for R2, but this is valid for any number of dimensions. This is our p.0822

Okay, now let f, the function f, be a mapping from R2 to R, so this is a function of 2 variables and differentiable.0840

Again, the idea of differentiable meaning that you can take the partial derivatives and the partial derivatives are continuous.0852

That is just going to be sort of implicit in the work that we do. We are not going to be really working with any functions that are not well behaved.0857

The directional derivative of f at p in the direction of a is the following. The notation is... the directional derivative in the direction of a of f at p = the gradient f at p dotted with a.0865

So, any time you dot something with the unit vector, what you are doing is you are actually forming the projection of the gradient vector onto the vector in the direction that you are interested in.0917

Now, let us go back to this thing right here. We said that this was our gradient vector. So, this was our ∇f.0929

Now, let us say I wanted to know what the directional derivative in that direction is. This is our vector a, okay? In that direction.0941

In other words, I want to know how the function changes not when I move along the gradient direction but when I move in this direction. My choice, any direction that I want.0952

What I do is I actually form the projection of the gradient vector onto a, which is... let us call this a unit vector.0961

We take a to be a unit vector, we form the dot product of the gradient of f at p · a, that is all that is. What that does, is that will give me the rate of change of the function as I move in the direction of a.0973

This is what I mean by we are using the gradient as the basis. We are using it as a base to find the derivative in any one of a million other directions, an infinite number of directions. That is what is going on here.0990

This is really, really fantastic because now we are not just limited to a derivative in 1 direction, we can choose a derivative in any direction that we want. This is profoundly, profoundly important.1003

Hopefully that makes sense. Now, okay, so this is true in... let me write this... so this is true for all n, okay? What I mean by that is RN, R, any function of any number of variables, 5, 10, 15, 30. This will always be the case.1016

Let us see what we are going to do here... basic reference we use the gradient as our base... here is what we are doing. Let us say we have... once again, I am going to draw this. Forgive me if I am redundant, but these are very, very important and constant repetition is good.1046

At a certain point, let us say we have a gradient vector that way. Let us say this is our a. If we want to know the rate of change of the function in the direction of a, this is ∇f, remember that upside down triangle? that is the symbol for the gradient, called del.1064

We go ahead and we actually end up doing the dot product, we end up essentially forming the projection. It is the rate of change in this direction.1082

That gives the direct... the vector gives the direction. This value right here gives the rate at which it is changing.1092

Okay, let me see here, I think we are actually ready for some examples. I hope that made sense, so let us go ahead and do an example here. Example 1.1104

Example 1. So, we will let f = x - e^zsin(y). This is a function of 3 variables, x, y, and z.1120

Now, f is a mapping from R3 to R.1135

Now we want, our task is to find the directional derivative, the direction of a at p, at the point p, which is -2pi and ln(3), in the direction of the vector (1,2,3).1143

This is a function from R3 to R, so it is a function of 3 variables.1172

Basically what you have is you are going to have... let me just draw this out.1177

This is where again you want to sort of get away from the geometric, you want to use the geometric -- geometry -- to help our intuition but we really want to concentrate on the algebra.1182

There is some point, (-2pi,ln(3)), wherever the heck that is, let us just say right there, in 3-space.1193

When I take the gradient of this function, and I put the point (-2pi,ln(3)) in for the values for that gradient, I am going to get some gradient vector.1201

Now it is telling me I want the directional derivative in the direction of (1,2,3).1213

So, (1,2,3) is a vector, let us say, going in that direction. That is what I am doing. I am using my gradient as a basis to find the derivative in another direction in space, that is it.1218

Now let us go ahead and just do it. We have the gradient of f is equal to... actually, you know what, let me go ahead and write my equation because I like doing that..1234

So, the directional derivative of a, and the directional derivative of f in the direction of a at p is equal to the gradient of f at p dotted with a.1248

Again, a has to be a unit vector, here a is not a unit vector. We have to convert it to a unit vector, and it is very important that we make sure that we are working with unit vectors.1261

So, the gradient of f is equal to... well, df/dx is going to be just 1, and if I take the df/dy, that is going to be -e^zcos(y), right?1270

Everything else is held constant so -e^z is just a constant.1293

Then the last one, derivative with respect to z, it is going to be -e^zsin(y), it looks like.1298

So, this is our gradient function, df/dx, df/dy, df/dz, okay?1313

When we put those values in, now the gradient of f evaluated at p which is (1,2,3), in other words when I put in (1,2,3,) for the (x,y,z) here, I end up with (1,3,0). I hope that you will confirm my arithmetic here because again I make a lot of arithmetic mistakes.1322

So that is it. We have this vector here (1,3,0). That is our gradient vector. Again I am not being precise, I am just specifying a direction, that is our gradient vector at p.1342

Let us go ahead and turn a into a unit vector. a unit, well we know how to do a unit vector, we take a and we divide by its norm, in other words we just shorten it and turn it into a unit vector.1352

That equals a/1+9+9, right?1368

a is... no, 1,3, oh wait, I am sorry, this was (1,3,3), not (1,2,3)... I was going to say it was going to be 1 + 4 + 9, but it was (1,3,3) that was the point.1377

So the norm of the vector is just this squared + this squared + this squared under the radical. 9 + 9 + 1, 9 and 9 is 18, so it equals a/sqrt(19) which equals 1/sqrt(19), 3/sqrt(19), and 3/sqrt(19).1392

Okay. Here we go. We have got it. Now we have our unit vector in the direction of a, we have our gradient, right?1419

Now, what we do is we just plug these into here, and then we actually take the dot product of this × that.1431

Let us go ahead and do that, let us do that in red. Let me just go ahead and write, so we will take (1,3,0) dotted with 1/sqrt(19), 3/sqrt(19), 3/sqrt(19), and what we end up getting is... so this times that is 1/sqrt(19) + 3 × 3 + 9/sqrt(19) + 0.1441

The answer is 10/sqrt(19). Now, let us talk about what this means. This was the direction of a.1475

That means that if I am at this point, if I start at that point and I move in the direction of a, my function is going to change by that much. That is what I am measuring. That is it. That is all that is happening.1487

This is the directional derivative. It is a measure of the rate of change of a function of several variables as you move in a direction other than the gradient.1502

if you happen to move in the direction of the gradient, perfect! You just take the norm of the gradient and that is the rate of change in the direction of the gradient.1513

In this case, this is how you do it, by using the gradient as the base. That is all that is happening here.1522

I hope that made sense. Now, let us go ahead and examine this idea of the gradient of f at p dotted with 'a' a little more closely.1529

Let me go ahead and go back to black here. So, let us examine the gradient of f at p · a, when a is a unit vector.1542

So, I am just going to subject what it is we have done to some analysis.1568

Well, we know from several lessons back, that the dot product of 2 vectors is equal to the norm, the product of the norm of those vectors × the cosine of the angle between those vectors, right?1573

If I have got some vector this way, there is some angle between them, θ.1593

Well, the norm of this vector × the norm of this vector × the cosine of the angle between them happens to equal the dot product of those vectors. It is a number.1599

The gradient of f at p dotted with a using this formula here, equals the norm of the gradient of f at p × the norm of the vector a × the cosine of the angle between them, cos(θ).1612

Again, if this is some gradient vector, ∇f, like we did and this was a, there is going to be some θ, you know, some angle that is in between them.1635

But, a is a unit vector, so its norm is 1. a is a unit vector, so the norm of a, this number, is equal to 1.1647

What we have, since this is 1... what we have is this thing which is the directional derivative, so we have... let me write it out, yeah, it is fine.1665

So the gradient of f at p dotted with a is just equal to the norm of the gradient of f at p × cosine of the angle between them.1681

This is of course, the directional derivative in the direction of a. That is it, that is all this is.1692

However, so now let us take a look at what we have got. We are saying that the directional derivative of the function at p is equal to this right here.1703

The norm of the gradient × the cosine of the angle theta between the gradient and the particular vector a that... in whose direction we are interested in.1714

So, we did this because a is a unit vector, and we use this formula for the dot product.1724

Well, now let us see what we have got. Well, cos(θ) takes on values from -1 to 1, in other words the maximum value of cos(θ) is 1, the smallest value is -1.1732

So what does that mean? Well, that means, so if cos(θ) takes on values from 1 to -1, the maximum value of the directional derivative, in other words the max that the directional derivative can actually take, is when...1748

Oh, let us actually write out a few so the cos(0) = 1, right? and the cos(pi/2) = 0, and the cos(pi), which is 180 degrees, equals -1. Let us just throw those out there for reference so that we remember in case we have forgotten our trigonometry.1765

So, this thing right here is going to take on its maximum value when the angle between them is going to be 0, when the cos(θ) = 1.1788

It is going to take on its minimum value when the angle between them is actually -1. What does that mean?1800

Well, that means, so if that is the gradient vector and that is a, if they are in the same direction, the directional derivative of f at p in the direction of a is at a maximum when you are actually moving in the direction of the gradient vector.1808

That is what this is saying. It is at a minimum when you are moving opposite to that. This is the gradient vector, that is what this says.1827

Again, we came up with this particular value, the directional derivative of f in the direction of a is equal to the norm of the gradient × the cosine of the angle between the 2 directions.1837

Well, as it turns out this value takes on its maximum value when the direction happens to be the direction of the gradient. It takes on its minimum value when it is the opposite direction of the gradient.1850

Okay. Let me write all of this out, so red for this one, so the directional derivative of f at p in the direction of a is a maximum when cos(θ) = 1.1863

That is, when θ = 0, which means when the angle between the gradient of f at p and a is 0.1889

This is the most important part right here. So, the gradient of f at p is the direction of maximal increase of the function, maximal increase of f.1915

Stop and think about this again. That means that if I am at a point p, if I move in the direction of the gradient, my function is going to be changing, it is going to be growing the fastest, and the most.1939

If I move in any other direction, it is not going to be changing as much as fast. That is what this says.1957

What this confirms is that the direction of the gradient, so now we have a geometric interpretation of what the gradient vector tells us.1964

It tells us the direction in which the function is changing the fastest. If I end up going in the direction opposite the gradient, that is the direction at which the function actually decreases the fastest.1973

Now the norm of the gradient, the actual value, the numerical value that gives us the rate of change.1989

So, the gradient is a vector. It gives us the direction of maximal increase of the function. The norm of the gradient gives us the rate at which it changes.1995

Let us write that down, the second part. The norm of the gradient of f at p is the rate of this increase.2004

Say it one more time. The gradient vector is the direction in which the function changes the greatest. If you are moving in the direction of the gradient vector, your function has the greatest change.2030

Your norm of that gradient is the rate of that change. It is the numerical value.2044

Now, let us look at this globally. A few variables... this vector... so, let us see... yeah, let us just take a quick little picture here, so in other words..2050

For a function of 2 variables, I have a bunch of... let us say I have that direction... well, I can also move it so let us call it ∇f.2064

I can move in that direction, that direction, that direction. These are all a bunch of different a's. a's, that is it.2073

The function is going to be changing, if I start at this point and move in a given direction. If I want the greatest change of that function, I am going to move in the direction of that gradient.2079

Now, let us go ahead and do an example. Example 2.2094

A temperature distribution, so now we can actually use a nice physical situation, a temperature distribution in space -- space is given by the formula... is given by, let us not use the word formula... is given by the function temperature x,y = 12 + 4cos(x)cos(y) + 9cos(2x).2107

Let me actually change this to... no, that is fine. So a temperature distribution in space is given by a function, this thing right here.2156

All that means is that wherever I am in terms of x and y, notice that this is a temperature distribution in space. The z can be any value, the z does not change, so what we are looking at is sort of a cylinder.2167

Z is just infinite values. We are still talking about 3-space. Notice we have used 2 variables, so it is x and y that is changing here.2181

Now, so at a given point x,y, the temperature at that point is going to be t(x,y). That is what this means, a temperature distribution, at different points in space you are going to have different temperatures.2190

Now, at the point... here is our problem... at the point (pi/3,pi/3) find the direction of the greatest increase in the temperature.2204

In other words, once I get to the point (pi/3,pi/3), and if I want to move in the direction that gives me the highest and fastest increase of the temperature, what direction do I move in?2237

Well, I already know the answer to that. The direction of maximal increase is the direction of the gradient vector. That is what is nice.2249

So all I have to do is I have to find the gradient vector, nice and easy.2254

So now, let us go ahead and find the gradient of t. That is going to equal, and I hope that you will confirm this for me... It is going to be -4sin(x)cos(y) - 18sin(2x), and the y coordinate is going to be -4cos(x)sin(y).2260

Okay. It looks complicated... it is not. It is just functions, that is it. We are going to be sticking numbers in here, pi/3, it is a 60 degree angle, we already know what all of these numbers are.2289

So now let us go ahead and evaluate this at the point (pi/3,pi/3). So, the gradient of t at (pi/3,pi/3) is going to equal...2299

I am going to write all of these out actually... -4 × sqrt(3)/2, and again I am hoping that you will confirm my arithmetic here, -18 × sqrt(3)/2, and then this one is going to be -4 × 1/2 × sqrt(3)/2.2312

Again, arithmetic is just arithmetic, it is not as important as the mathematics. That is what is important. There will always be someone there to check your arithmetic, there will not always be someone there to check your mathematics, the concepts.2333

Okay, that is going to equal, -sqrt(3), -9sqrt(3), and then -Sqrt(3). I am actually going to go onto the next page here because i am getting some stray lines here.2345

So our final answer is 10sqrt(3) - sqrt(3). That is it, this vector. If I move in the direction of the vector, -10sqrt(3) - sqrt(3), from (pi/3,pi/3) at that point, that is the direction that is going to give me the maximal increase in the temperature.2366

The gradient vector. The direction of the gradient vector.2389

Let us go ahead and write. At (pi/3,pi/3), the gradient... the greatest increase in t happens when we move in this direction.2394

Now, the rate of this change. The rate of this increase is just the norm of that vector, of this increase is just the gradient of t at (pi/3,pi/3), and that is equal to (sqrt(3),0,3). That is it.2435

So, as I move in the direction of the gradient, temperature is going to be changing by that much for every unit change in that direction.2464

So, that is it. That is the directional derivative, and that is the geometric interpretation of the gradient.2472

Thank you so much for joining us here at educator.com, we will see you next time. Take care, bye-bye.2478