# Using differentials to optimize a function [closed]

I've read in a paper by Tevian Dray an alternative way to solve optimization problems manipulating "differentials". Here is an example of how it works (next I quote the paper).

Consider the problem of minimizing the length of a piecewise straight path connecting two fixed points with a given line, as shown in Figure 1. For instance, the line could represent a river along which a single pumping station is to be built to serve two towns. The distances $C$, $D$, and $S = a + b$ are specified; the goal is to determine $a$ and/or $b$ so that $ℓ = p + q$ is minimized.

The standard solution to this problem involves expressing $a, b, p, q$, and hence $ℓ$, in terms of a single variable, typically $a$, then minimizing $ℓ$ by computing $\frac {dℓ} {da}$ and setting it equal to zero. This computation is straightforward, but involves the derivatives of square roots and some messy algebra.

Consider instead the following solution, using differentials. First, write down what you know: $$a + b = S$$ $$a^2 + C^2 = p^2$$ $$b^2 + D^2 = q^2$$ $$p + q = ℓ$$

where $S, C, D$ are known constants. Next, take the differential of each equation: $$da + db = 0$$ $$2a\ da = 2p\ dp$$ $$2b\ db = 2q\ dq$$ $$dp + dq = dℓ$$

We are trying to minimize $ℓ$, so we set $dℓ = 0$ to obtain

$$0 = dℓ = dp + dq = \frac{a}{p} da + \frac{b}{q} db = \left ( \frac{a}{p} - \frac{b}{q} \right ) da$$

so that

$$\frac{b^2}{a^2}=\frac{q^2}{p^2}=\frac{b^2+D^2}{a^2+C^2}$$

which (since lengths must be positive) quickly yields

$$\frac{b}{a}=\frac{D}{C}$$

so that

$$a =\frac{CS}{C + D}$$ $$b =\frac{DS}{C + D}$$ and it is straightforward to verify that these values do in fact minimize $ℓ$.

I've tried to use this method to solve some problems, and sometimes it works and sometimes doesn't (when I have to optimize a function $f(x,y,z)$ with two constraint function $g_1(x,y,z)=c_1$ and $g_2(x,y,z)=c_2$, generally I have to use Lagrange multipliers because this manipulation of differentials doesn't lead to the correct answer). Also, I've seen this method been used in some thermodynamics textbooks.

Note: I've also asked this question on Mathematics Stack Exchange

• As much as I personally like this question I should tell you someone will probably comment here saying that cross-posting between sites is frowned upon. Commented Mar 4, 2015 at 20:26
• I'm voting to close this question as off-topic because it is a math question. (No migrate vote as it already exists on math) Commented Mar 5, 2015 at 0:33
• possible duplicate of How to treat differentials and infinitesimals? Commented Mar 5, 2015 at 6:21

So you know that if $du$ is small compared to $u$ then you can linearize about $u$. When you want to generalize to more variables, you have to do partial derivatives: for example $$f(x + dx, y + dy, z + dz) \approx f(x,y,z) + \left(\frac{\partial f}{\partial x}\right)_{y,z} dx + \left(\frac{\partial f}{\partial y}\right)_{x,z} dy + \left(\frac{\partial f}{\partial z} \right)_{x,y} dz.$$ There is a nice shorthand for this expression using the dot product of the vector $d\vec r = (dx, dy, dz)$ as $$f(\vec r + d\vec r) \approx f(\vec r) + \nabla f \cdot d\vec r.$$ If you calculate it out, the last term here is the "differential" approach that you get above. Your constraints will generate fixed conditions for $d\vec r$, $$\nabla g_1 \cdot d\vec r = \nabla g_2 \cdot d\vec r = 0.$$
Now we want to find the points $\vec r$ such that, if $d\vec r$ obeys the constraints, then $\nabla f \cdot d\vec r = 0$. There is no reason that I see that this should lead to a wrong answer. In fact, it leads to a matrix solution $$\left[\begin{array}{c}(\nabla f)^T\\(\nabla g_1)^T\\(\nabla g_2)^T\end{array}\right] \left[\begin{array}{c}dx \\ dy \\ dz\end{array}\right] = \left[\begin{array}{c} 0 \\ 0 \\ 0\end{array}\right]$$which amounts to looking for a nontrivial kernel of that matrix, which amounts to saying that the vectors are linearly dependent, which says that $\nabla f + \lambda_1 \nabla g_1 + \lambda_1 \nabla g_2 = \nabla (f + \lambda_1 g_1 + \lambda_2 g_2) = 0$, which gives you back exactly the method of Lagrange multipliers! So these are totally equivalent approaches if you do them correctly.