# How is gradient the maximum rate of change of a function?

Recently I read a book which described about gradient. It says $${\rm d}T~=~ \nabla T \cdot {\rm d}{\bf r},$$ and suddenly they concluded that $\nabla T$ is the maximum rate of change of $f(T)$ where $T$ stands for Temperature. I did not understand. How gradient is the maximum rate of change of a function? Please explain it with pictures if possible.

-
You've just taken the englisch grammar out of the title. –  Nikolaj K. Mar 25 '13 at 16:50
@NickKidman: :-) –  Inquisitive Mar 25 '13 at 16:53

Formally, we have the symbolic equation $$\nabla T\cdot\mathrm d\mathbf r = \begin{pmatrix} \frac{\partial T}{\partial x^1} & \cdots & \frac{\partial T}{\partial x^n} \end{pmatrix} \begin{pmatrix} \mathrm dx^1 \\ \vdots \\ \mathrm dx^n \end{pmatrix} = \sum_i \frac{\partial T}{\partial x^i}\mathrm dx^i = \mathrm dT$$ However, that doesn't really help us with our intuition.

For a more precise as well as more intuitive definition, we need to evaluate differential and gradient and (using the chain rule for the 2nd equality) arrive at \begin{align*} \mathrm dT_{\mathbf r}(\mathbf v) = \frac{\mathrm d}{\mathrm dt}\Big|_{t=0} T(\mathbf r + t\mathbf v) = \nabla T(\mathbf r)\cdot\mathbf v && \forall\mathbf r,\mathbf v\in\mathbb R^n \end{align*} As we see here, the differential (or equivalently the scalar product with the gradient) yields the (infinitesimal) change of $T$ as we move along the curve $t\mapsto \mathbf r + t\mathbf v$.

Now, we can ask in which direction $\mathbf v$, $||\mathbf v||=1$ does $T$ change the most? The answer is obviously in the direction of the gradient, as the scalar product is maximal if $\measuredangle(\mathbf v,\nabla T)=0$. The length of $\nabla T$ of course is relevant as well: It measures the strength (or rate) of this change.

-

Gradient such as $\nabla T$ refers to vector derivative of functions of more than one variables. Physically, it explains rate of change of function under operation by Gradient operation.

$\nabla T$ is a vector which points in the direction of greatest increase of function. The direction is zero at local minimum and local maximum.

Physical meaning of equation ${\rm d}T~=~ \nabla T \cdot {\rm d}{\bf r}$:
${\rm d}T$ is the projection of $\nabla T$ in the direction of ${\rm d}{\bf r}$.

-
Just above you said " Its explaining physical meaning of dot product" . Same thing you have repeated here. You mentioned dot product in the sentence format. –  Inquisitive Dec 29 '12 at 13:38
@Inquisitive Main answer is above horizontal line which says what a gradient is. What you see below horizontal line can be applied for any vector, not just gradient. Maybe, you wanted to know about dot product too, but my answer is more complete. –  Sachin Shekhar Dec 29 '12 at 13:52
that's why I gave u vote up. –  Inquisitive Dec 29 '12 at 13:55

Consider an $n$-dimensional space (two dimension in the picture), and let $f(\vec x)$ be a non-constant scalar function, like a temperature distribution in your case. Let $\vec y(t)$ be any curve in the space such that the function $f(\vec y(t))=c$ is constant along that trajectory (the colored lines).

Now compute the scalar product $\left\langle ., .\right\rangle$ of the gradient vector $\vec \nabla f$ (the red arrows) evaluated at the point $\vec y(t)$, with the tangent vector $\vec Y(t):=\frac{\text{d}\vec y(t)}{\text{d}t}$ along that very same curve

$$\left\langle \vec Y(t), \left(\vec\nabla f\right)_{\vec y(t)}\right\rangle :=\sum_{i=1}^n\ Y^{\ i}(t)\left(\nabla_i f\right)_{\vec y(t)}=$$ $$=\sum_{i=1}^n \frac{\text{d}y^i(t)}{\text{d}t}\left(\frac{\partial f}{\partial y^i}\right)_{\vec y(t)} =\frac{\text{d}f(\vec y(t))}{\text{d}t} =\frac{\text{d}c}{\text{d}t} =0.$$

The result that the scalar product of these two vectors is zero means that the gradient $\vec \nabla f$ is always orthogonal to the direction in which the function doesn't change, i.e. $\vec Y(t)$. So it is therefore pointing in the direction of maximal change. You can clearly see the orthogonality in the picture.

And if the function changes fast with respect to spatial coordinates, then the components of the gradient $\nabla_i f\equiv\frac{\partial f}{\partial y^i}$ and therefore the whole gradient vector will be big.

Notice also how the dimenion $n$ wasn't used in any essential way to derive the statement.

-

From elementary calculus, the total diffrential of a function $f(x_1, x_2,...x_n)$ is given by \begin{align*}df &= \frac {\partial f}{\partial x_1}dx_1 + \frac {\partial f} {\partial x_2} dx_2~...+\frac{\partial f} {\partial x_n} dx_n\\&=\nabla f\cdot\vec {dx} \end{align*}

For $df = 0$, $\nabla f$ and the corresponding differential $\vec {dx}_{min}~~$ are therefore orthogonal to one another since their dot product is zero. The differential $\vec{dx}_{max}$ orthogonal to $\vec {dx}_{min}$ maximises $df$, which means $\vec{dx}_{max}$ is parallel to $\nabla f$. So we can write \begin{align*}df_{max} &= \nabla f\cdot\vec {dx}_{max}\\ &= |\nabla f||\vec {dx}_{max}|\\ |\nabla f| &= \frac {df_{max}} {|dx_{max}|}\end{align*}

Therefore $\nabla f$ is a vector with a magnitude equal to the maximum change in $df$ wrt $|dx|$, and a direction along $\vec {dx}_{max}$

-

The equation

${\rm d}T~=~ \nabla T \cdot {\rm d}{\bf r}$,

says that the change in T, namely ${\rm d}T$, is the scalar product of 2 vectors, $\nabla T$ and ${\rm d}{\bf r}$, which can also be written as the magnitude of the 1st vector times the magnitude of the 2nd vector times cosine the angle between them.

${\rm d}T~=~ |\nabla T| |{\rm d}{\bf r}|\cos\theta$.

Now assume that we are fixing the length of the infinitesimal displacement vector but we can move it around changing its direction, and hence changing $\theta$. You notice that $dT$ is maximum if $\theta$ is $0$.

$\theta=0$ means both vectors have the same direction, and since $d{\bf r}$ is the displacement vector then in this case you move along the same direction of $\nabla T$ that makes $dT$ maximum.

Hence you can interpret $\nabla T$ as the vector whose direction is the direction along which the change of the function $T$ is maximum.

-
Thanks for explaining it in simple words. –  Inquisitive Feb 5 '12 at 17:21
@Inquisitive Its not the correct answer. Its explaining physical meaning of dot product, not gradient. –  Sachin Shekhar Dec 29 '12 at 12:50
@SachinShekhar - Yes, Title and the content provided by me was not matching.That was the reason for accepting those answers. Anyways now I know the physical meaning of $Grad$ :) –  Inquisitive Dec 29 '12 at 12:57
Del, or $\nabla$ , is a generalisation of the gradient to more than one dimension. In one dimension $\nabla$ is the same as the gradient.