How to treat differentials and infinitesimals?

In my Calculus class, my math teacher said that differentials such as $dx$ are not numbers, and should not be treated as such.

In my physics class, it seems like we treat differentials exactly like numbers, and my physics teacher even said that they are in essence very small numbers.

Can someone give me an explanation which satisfies both classes, or do I just have to accept that the differentials are treated differently in different courses?

P.S. I took Calculus 2 so please try to keep the answers around that level.

P.S.S. Feel free to edit the tags if you think it is appropriate.

-
–  Qmechanic Jan 9 '14 at 0:53
@dmckee Depending on what you mean by "such shortcuts" I think they're perfectly rigorous either for reasons seen at anupam's link or because of the formalizations of nonstandard analysis or smooth infinitesimal analysis. (for a very brief demonstration of what calculus is like with the former, see math.stackexchange.com/a/623657/26369 ) –  Mark S. Jan 9 '14 at 3:14
As a non-expert (although educated in Physics), it seemed to be enough for me to realize that things like going from $\frac{dy}{dx} = x$ to $dy = x dx$ or using $\frac{dy}{dx} = \frac{dy}{dt} \frac{dt}{dx}$ are not algebraic operations as they might first appear to be. We can do that, but there is a great deal of complicated math required to justify it (as seen in these answers). So I didn't think of them as numbers. I just thought of what I was doing as a shortcut that allowed me to keep track of what I was doing, and I made sure I never tried to apply algebraic operations blindly. –  jpmc26 Jan 9 '14 at 8:42
When somebody tells you that something is not a number, it probably means that it is not a real number, or that it doesn't belong to the real numbers or any other number set that the person speaking has in mind. For example people may say that infinite is not a number, yet it is - for example in Riemann sphere - but it is not a real number. The same goes for infinitesimals. –  Theraot Jan 9 '14 at 9:55
There is no effort to incorporate physics into this question, and it needs to be migrated, despite what QM says. –  Larry Harson Jan 14 '14 at 23:26

There is an old tradition, going back all the way to Leibniz himself and carried on a lot in physics departments, to think of differentials intuitively as "infinitesimal numbers". Through the course of history, big minds have criticized Leibniz for this (for instance the otherwise great Bertrand Russell in Chapter XXXI of "A History of Western Philosophy" (1945)) as being informal and unscientific.

But then something profound happened: William Lawvere, one of the most profound thinkers of the foundations of mathematics and of physics, taught the world about topos theory and in there about "synthetic differential geometry". Among other things, this is a fully rigorous mathematical context in which the old intuition of Leibniz and the intuition of plenty of naive physicists finds a full formal justification. In Synthetic differential geometry those differentials explicitly ("synthetically") exist as infinitesimal elements of the real line.

A basic exposition of how this works is on the nLab at

Notice that this is not just a big machine to produce something you already know, as some will inevitably hasten to think. On the contrary, this leads the way to the more sophisticated places of modern physics. Namely the "derived" or "higher geometric" version of synthetic differential geometry includes modern D-geometry which is at the heart for instance of modern topics such as BV-BRST formalism (see e.g. Paugam's survey) for the quantization of gauge theories, or for instance geometric Langlands correspondence, hence S-duality in string theory.

-
+1 Not quite at the OP's asked for level, but most interesting indeed and a collection of this kind of posts of this kind could make this a killer question and set of answers. You've emboldened me to write up Robinson's nonstandard analysis when I have more time if someone doesn't beat me to it. –  WetSavannaAnimal aka Rod Vance Jan 9 '14 at 1:16
Is the infinitesimal interval $D$ used to cheat new generalized elements into our space $X$, which aren't there in the classical formulations? Some of the links speak of $D$ as a subset of a ring. Is there always an ordering on the infinitesimal interval and does one think of this as a collection of different small things, equipped with some notion of size for distinguishing them? It also seems like the Kock-Lawvere axiom (a functional "$ϵ^2=0$"?) seems suitable for extension to infinitesimal calculi like Ito calculus etc., is that right? Do we do super-stuff directly by adding this object? –  NikolajK Jan 10 '14 at 20:14
There are two complementary aspects to this. On the one hand the categorical logic of toposes allows to formally speak of the subset of the real line of elements that square to 0. This is just what people following Leibniz intuitively did anyway, but categorical logic shows that and how exactly this is consistent. This is then how notably Anders Kock (home.imf.au.dk/kock) wrote his two textbooks on synthetic differential geometry (home.imf.au.dk/kock/SGM-final.pdf): he speaks "synthetically" of the subset D of R on the elements that square to 0 and derives all of diff geometry. –  Urs Schreiber Jan 11 '14 at 0:17
On the other hand one can choose to build concrete models for the axioms in which notably the textbooks by Kock are written, hence for toposes that validate the Kock-Lawvere axioms. In the typical such models the category of smooth manifolds is enlarged somewhat by objects known as "smooth loci", which include for instance the space formally dual to the "ring of dual numbers", which is just the ring embodying the equation "epsilon^2 = 0". This more concrete incarnation of SDG can be phrased entirely in classical logic and hence shows which classical notions embody the idea of infinitesimals. –  Urs Schreiber Jan 11 '14 at 0:21

I think your math teacher is right. One way to see that differentials are not normal numbers is to look at their relation to so called 1-forms. I do not know if you already have had forms in calculus 2, but it is easy to look up on the internet.

Since you chose a tag "integrals" in your question, let me give you an example based on an integral. Let's say you have a function $f(x^2+y^2)$ and want to integrate it over some area $A$:

$$\int_A f(x^2+y^2) dx dy$$

The important thing to realize here is, that the $dxdy$ is actually just an abbreviation for $dx\wedge dy$. This $\wedge$ thingy is an operation (wedge product - much like multiplication, but with slightly different rules) that can combine forms (in this case it combines two 1-forms to a 2-form). One important rule for wedge products is anti-commutation:

$$dx\wedge dy=-dy\wedge dx$$

This makes sure that $dx\wedge dx=0$ (where a physicist could cheat by saying that he neglects everything of order $O(dx^2)$, but that is like mixing pears and apples, frankly misleading). Why would differentials in integrals behave like this and where is the physical meaning? Well, here you can think about the 'handedness' of a coordinate system. For instance the integration measure $dx\wedge dy\wedge dz$ is cartesian 'right-handed'. You can make it 'left-handed' by commuting the $dx$ with $dy$ to obtain $-dy\wedge dx\wedge dz$, but then the minus sign appears in front, which makes sure that your integration in a 'left-handed' coordinate system still gives you the same result as the initial 'right-handed' one.

In any case, to come back to the above integral example, let's say you like polar coordinates better to perform your integration. So you do the following substitution (assuming you already know how to take total differentials):

$$x = r \cos \phi~~~,~~~dx = dr \cos \phi - d\phi\, r \sin \phi$$ $$y = r \sin \phi~~~,~~~dy = dr \sin \phi + d\phi\, r \cos \phi$$

Multiplying out your $dx\wedge dy$ you find what you probably already know and expect:

$$dx\wedge dy = (dr \cos \phi - d\phi\, r \sin \phi)\wedge(dr \sin \phi + d\phi\, r \cos \phi)$$ $$= \underbrace{dr\wedge dr}_{=0} \sin \phi\cos \phi + dr\wedge d\phi\, r \cos^2 \phi - d\phi\wedge dr\, r \sin^2 \phi - \underbrace{d\phi\wedge d\phi}_{=0}\, r^2 \cos \phi \sin \phi$$ $$=r(dr\wedge d\phi \cos^2 \phi - d\phi\wedge dr \sin^2 \phi)$$ $$=r(dr\wedge d\phi \cos^2 \phi + dr\wedge d\phi \sin^2 \phi)$$ $$=r\, dr\wedge d\phi ( \cos^2 \phi + \sin^2 \phi)$$ $$=r\, dr\wedge d\phi$$

With this the integral above expressed in polar coordinates will correctly read:

$$\int_A f(r^2)r\, dr d\phi$$

Where we suppressed the wedge product here. It is important to realize, that if we would not have treated the differentials as 1-forms here, the transformation of the integration measure $dx dy$ into the one involving $dr$ and $d\phi$ would not have worked out properly!

Hope this example was down to earth enough and provides some feeling for how differentials are not entirely very small numbers.

-
Interesting interpretation of differentials. But it should be said that the interpretation where differentials are something like small numbers works well too in your example. In this traditional view, there is no need to require that $dxdy = rdrd\phi$, but instead we require that the latter expression gives standard hypervolume (that is the standard volume of the corresponding domain), up to first order in the differentials. This then leads to determinant of the Jacobi matrix and to the factor $r$ in $rdrd\phi$. –  Ján Lalinský Jan 9 '14 at 11:13
It is misleading to interpret $dx\wedge dx=0$ as neglecting terms of $O(dx^2)$. The former is a geometrical thing: the area of a parallelogram with degenerate sides is zero. The latter is ingrained in the theory. –  Emilio Pisanty Jan 9 '14 at 11:18
I think that this interpretation of the change of coordinates is wrong. The integral of a $n$ form over an $n$ chain $c:[0,1]^n \to \mathbb R ^n$ is defined as $$\intop _c \omega = \intop _{[0,1]^n} c^* \omega ,$$ because this gives exactly the result of a change of coordinates, if $\omega = f\,\text d x^1 \wedge \dots \wedge \text d x^n$. So if $c([0,1]^n)=A$, we may write $$\intop _A f = \intop _c \omega$$ as a consequence of the definition of the integral of forms. In other words, the change of coordinates through the pullback doesn't constitute a proof of the change of variable s formula. –  pppqqq Jan 9 '14 at 12:03
@Toby: If this was true, any integral $\int_{-\infty}^{\infty} dx f(x^2)$ would be trivially zero which is clearly not the case. (Just substitute $x\to -y$. If $dx$ in the integral is abbreviation for $|dx|$, then you get $\int_{\infty}^{-\infty} f(y^2)dy$, pull out a minus by reversing the integration direction and relabel y back to x. You would get that the integral must be equal to plus or minus itself and therefore zero. Clearly not a correct result generally.) –  Kagaratsch Jan 9 '14 at 19:17
Could you move this discussion to a chat room if it is to continue? You can use Physics Chat or just click on the prompt to move to chat which should appear when you try to post a comment at some point. –  David Z Jan 12 '14 at 0:57

(I'm addressing this from the point of view of standard analysis)

I don't think you will have a satisfactory understanding of this until you go to multivariable calculus, because in calculus 2 it's easy to think that $\frac{d}{dx}$ is all you need and that there's no need for $\frac{\partial}{\partial x}$ (This is false and it has to do with why in general derivatives do not always behave like fractions). So that's one reason why differentials are not like numbers. There are some ways that differentials are like numbers, however.

I think the most fundamental bit is that if you're told that $f dx=dy$, this means that $y$ can be approximated as $y(x)=y(x_0)+f\cdot(x-x_0)+O((x-x_0)^2)$ close to the point $x_0$ (this raises another issue*). Since this first order term is really all that matters after one applies the limiting procedures of calculus, this gives an argument for why such inappropriate treatment of differentials is allowable - higher order terms don't matter. This is a consequence of Taylor's theorem, and it is what allows your physics teacher to treat differentials as very small numbers, because $x-x_0$ is like your "dx" and it IS a real number. What allows you to do things you can't do with a single real number is that that formula for $y(x)$ holds for all $x$, not just some x. This lets you apply all the complicated tricks of analysis.

If I get particularly annoyed at improper treatment of differentials and I see someone working through an example where they write, "Now we take the differential of $x^2+x$ giving us $(2x+1)dx$", I may imagine $dx$ being a standard real number, and that there's a little $+O(dx^2)$ tacked off to the side.

Your math teacher might argue, "You don't know enough about those theorems to apply them properly, so that's why you can't think of differentials as similar to numbers", while your physics teacher might argue, "The intuition is the really important bit, and you'd have to learn complicated math to see it as $O(dx^2)$. Better to focus on the intuition."

I hope I cleared things up instead of making them seem more complicated.

*(The O notation is another can of worms and can also be used improperly. Using the linked notation I am saying "$y(x)-y(x_0)-f\cdot(x-x_0)=O((x-x_0)^2)$ as $x\to x_0$". Note that one could see this as working against my argument - It's meaningless to say "one value of $x$ satisfies this equation", so when written in this form (which your physics prof. might find more obtuse and your math prof. might find more meaningful) it's less of an equation and more of a logical statement.)

-

In mathematics the notation $\def\d{\mathrm d}\d x$ is actually a linear form, this means that $\d x$ is a linear function taking a vector a giving a scalar.

Let us take a differentiable function $f$ defined over $\def\R{\mathbf R}\R$ and consider it at point $a$. The tangent to the curve of $f$ at the point $a$ has a slope $f'(a)$. The point on this tangent of abscissa $b$ has ordinate $f_a(b)=f(a)+(b-a)f'(a)$. $f_a(b)$ is the linear approximation of $f(b)$ knowing $f$ at point $a$.

We define then $\d x(b-a)=b-a$. We have $$f_a(b)-f(a)=f'(a)\d x(b-a),\tag{1}$$ and we write $$\d f_a=f'(a)\d x$$ which is the formula (1) written for linear forms. Indeed the linear form $\d f_a$ is defined by $$\d f_a(\epsilon)=f'(a)\d x(\epsilon)=f'(a)\epsilon.$$

In physics one often makes the confusion between $\d x$ (the linear form) and $\epsilon$ (the argument of $\d x$). I hope you understand why when looking at the last equation.

NOTE. This may seem quite useless but in dimension $n>1$ this becomes more interesting. You have indeed $$\def\vec#1{\boldsymbol{#1}} \def\der#1#2{\frac{\partial #2}{\partial #1}} \d f_{\vec a}=\nabla f(\vec a)\cdot\d\vec r=\begin{pmatrix}\der {x_1}{f(\vec a)}\\\vdots\\\der {x_n}{f(\vec a)}\end{pmatrix}\cdot \begin{pmatrix}\d x_1\\\vdots\\\d x_n\end{pmatrix}$$ that translates into, for $\vec\epsilon=(\epsilon_1,\dots,\,\epsilon_k)\in\R^n$, $$\d f_{\vec a}(\vec\epsilon)=\sum_{k=1}^n \der{x_k}{f(\vec a)}\d x_k(\vec\epsilon)=\sum_{k=1}^n\der{x_k}{f(\vec a)}\epsilon_k,$$ because $\d x_k(\vec\epsilon)=\epsilon_k$ ($\d x_k$ is the $k^{\rm th}$ coordinate form).

-

As you see from the variety of answers there are many possibilities to interpret differentials mathematically exact.

One nice simple interpretation is as coordinates of tangential vectors.

Consider an equation $$z = f(x,y)$$ describing a curved surface in three-dimensional space ($z$ is the height).

Then the equation $$dz = \frac{\partial}{\partial x} f(x,y) \cdot dx + \frac{\partial}{\partial y} f(x,y) \cdot dy$$ describes the points $(\bar x,\bar y,\bar z)=(x+dx,y+dy,z+dz)$ of the tangential plane at the point $(x,y,z)$ on the surface. This equation is often named tangent equation.

If you have some specific point $(x,y,z)$ given by coordinate values as numbers and would like to have also a specific point on the tangent plane just put numbers in for $dx$, $dy$ and $dz$. Thus, the differentials can stand for numbers. Why not.

So far so good. Now, why should the numbers be small? We assume that the surface is smooth at the point $(x,y,z)$, meaning that $f$ should be continuously differentiable there. Then $$\frac{z+dz - f(x+dx,y+dy)}{|(dx,dy)|}\rightarrow 0 \quad\text{ for } |(dx,dy)|\rightarrow 0$$ where $dz$ fulfills the above tangent equation. Here $|(dx,dy)|=\sqrt{dx^2 + dy^2}$ denotes the Euclidian norm.

The division by $|(dx,dy)|$ lets us look at a scaled picture of the surface around the point $(x,y,z)$. To keep angles as they are we scale the picture evenly in all directions. The picture is always scaled such that the disturbance $(dx,dy)$ from the point $(x,y,z)$ is in the order of magnitude of 1. Even in this up-scaled picture the height $z+dz$ of the disturbed point $(x+dx,y+dy,z+dz)$ on the tangential plane fits better and better the corresponding height $f(x+dx,y+dy)$ on the curved surface.

$\sum$: The tangent plane with the local coordinates $dx$, $dy$ and $dz$ fits the better the curved surface the smaller the disturbations $dx,dy,dz$ are.

To clarify things let us consider an example. Let the curved surface be $$z=x^2-y.$$ We pick the specific point with $x=1$ and $y=2$ yielding $z=1^2-2 = -1$. The tangent equation is $$dz = 2x\cdot dx - dy,$$ and at our specific point $$dz = 2 dx - dy.$$ To have a specific point on the tangent plane let us consider the differentials $dx=\frac14$ and $dy=1$ yielding $$dz = 2\cdot\frac14 - 1 = -\frac12.$$

The location of this point on the tangent plane in 3d-space is $(x+dx,y+dy,z+dy)=\left(1+\frac14,2+1,-1-\frac12\right)=\left(\frac54,3,-\frac32\right)$.

At the same $x$- and $y$-coordinates we get on the curved surface the height $z'$ with $$z' = f(x+dx,y+dy) = f\left(\frac54,3\right) = \left(\frac54\right)^2 - 3 = -\frac{23}{16} = -1.4375.$$ It is a little bit off the height $z+dz=-1.5$ of the corresponding point on the tangent plane.

Even if I presented here a numerical example in practice the differentials are more often used as variables to determine relations between the differentials (with their interpretation as tangent coordinates).

In the context of tangent coordinates the differential quotient $\frac{dy}{dx}=f'(x)$ is the ratio of the coordinates $dx$ and $dy$ of the tangent on the graph of $f$ at $x$.

As long as you avoid division by zero you can divide through a differential $dx$ (as tangent coordinate).

-
Tobias: "Then the equation $dz = [...]$ describes the points $(\bar x,\bar y,\bar z)=(x+dx,y+dy,z+dz)$ of the tangential plane at the point $(x,y,z)$ on the surface." -- How should be distinguished whether the indicated equation describes points/elements of a plane, and not (for instance) elements of some other surface, such as an "Osculating sphere" ("Schmiegekugel") to the given surface, at point $(x,y,z)$; or indeed any other surface containing point $(x,y,z)$? Or does that not even matter? –  user12262 Jan 10 '14 at 6:30
I know what you mean. But no! Everything I wrote is taken to be literally. For reading the text forget what you have learned about tangent vectors on manifolds in higher semesters. The above version works whenever you consider smooth $m$-dimensional sub-manifolds of $\mathbb{R}^n$ with $m\leq n$. This approach is simple and helps with practical problems where you have some canonical coordinates (e.g., envelopes, many problems from numerics, multi-body mechanics in computer simulation and so on). –  Tobias Jan 10 '14 at 7:18
I have added an example to show how literal one can see it. –  Tobias Jan 10 '14 at 7:58
@user12262 A good way to grasp the "nonuniqueness" you speak of is to think of tangent vectors as equivalence classes of $C^1$ paths - because you can show that this kind of discussion is independent of the class member. –  WetSavannaAnimal aka Rod Vance Jan 10 '14 at 8:47
Tobias: "I know what you mean." -- Fabulous. (Btw., in my comment above I'd perhaps better have written of "Osculating ellipsoid" ("Schmiegeellipsoid") instead of spheres.) "[...] whenever you consider smooth $m$-dimensional sub-manifolds of ${\mathbb R}^n$" -- Well, I still need to digest your added example and WSA-(RV)'s comment. Whenever I read "curvature" I look at orbit.dtu.dk/en/publications/… ... –  user12262 Jan 10 '14 at 16:50

With the objective of keeping complexity to a minimum, the best "unifying" solution, is to think of differentials, infinitesimals, numbers, etc. as mathematical symbols to which certain characteristics, properties, and mathematical operations (rules), are applicable.

Since not all rules are applicable to all symbols, you need to learn which rules are applicable to a particular set of symbols.

Whether you are learning fractions, decimals, differentials, etc., just learn the symbols and their particular rules and operations and that will be sufficient for 99% of the time.

-
This seems to "answer" the question by saying "You should learn the rules you can apply to differential and infinitesimals". It is correct as far as it goes, but is no help whatsoever. –  dmckee Jan 15 '14 at 0:39

protected by Qmechanic♦Jan 14 '14 at 22:58

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.