Why does Taylor’s series “work”?

I am an undergraduate Physics student completing my first year shortly. The following question is based on the physical systems I’ve encountered so far. (We mostly did Newtonian mechanics.)

In all of our analyses of the physical systems (up till now) we recklessly exploited Taylor’s series, retaining terms up to the desired precision of our approximate model of reality.

But what is the justification of using Taylor’s series? It implicitly implies that the mathematical functions in our physical model are analytic. But how can we be sure about that?

Sure, the nature doesn’t seem to be discontinuous or have “kinks” (i.e. nonexistent derivatives) in its behaviour. That seems plausible. But still, there are non-analytic smooth functions. And there are “many” more of them than there are analytic functions. So even if the nature works smoothly in its endeavours, it is essentially zero probability that it should do so analytically.

So why do we use Taylor’s series at all?

• Possible duplicates: physics.stackexchange.com/q/1324/2451 and links therein. May 15, 2019 at 8:53
• @Steeven a function by definition is analytical when it is equal to its Taylor series. This is rare in the same sense that rational numbers are rare among the reals May 15, 2019 at 9:41
• @Steeven if every smooth function would be analytic, then you can reconstruct whole function from its knowledge on arbitrary small interval. Intuitively this is nosense. Famous example is function for which $f(x)=0$ if $x<0$ and $f(x)=e^{-1/x}$ if $x>0$ May 15, 2019 at 9:42
• "it is essentially zero probability that it should do so analytically" - this assumes a probability measure on the space of all functions which is somehow "natural". While there are several important measures with this property, choosing one of them is still a human prejudice. May 15, 2019 at 10:09
• Be careful with how you think of probability. If I throw a pencil on the ground, there is infinitesimally small probability that it lands pointing at any specific angle (since angles are continuous), and yet it does so every time. May 15, 2019 at 17:41

I had this same problem, too. The trick with it is realizing that there's an important difference between Taylor series and Taylor approximations or polynomials, whose behavior is described by Taylor's theorem. Yes, very often I suspect a common mistake is that you first see Taylor polynomials and theorem, and then you get Taylor series and that becomes the focus and suddenly you forget about the rest.

But here, what we're actually doing when we "truncate" a Taylor series is that we are going back to a Taylor polynomial, since that is what a truncated Taylor series is - or alternatively, a Taylor series is the natural extension of to infinite order. In that context, Taylor's theorem tells you exactly how it does or does not behave as an approximation and - surprise - it doesn't require anything about analyticity at all. Analyticity only comes into play when you consider the full series: in fact, what Taylor's theorem tells you is that a finite Taylor polynomial will still work as an approximation for even a non-analytic function, so long as you get suitably close to the point at which you're taking the polynomial and the function is differentiable enough to be able to make the polynomial of the given degree possible to take.

Specifically, Taylor's theorem tells you that, analytic or not, if you cut the Taylor series so that the highest term has degree $$N$$, to form the Taylor polynomial (or truncated Taylor series) $$T_N(a, x)$$, where $$a$$ is the expansion point, you have

$$f(x) = T_N(a, x) + o(|x - a|^N),\ \ \ \ \ x \rightarrow a$$

where the last part defines the behavior of the remainder term: this is the "little-o notation" and means that the error pales in comparison to the bound $$|x - a|^N$$.

As an example in elementary mathematical physics, consider the analysis of the "pathological" potential in Newtonian mechanics given by

$$U(x) := \begin{cases} e^{-\frac{1}{x^2}},\ x \ne 0\\ 0,\ \mbox{otherwise} \end{cases}$$

which is smooth everywhere, but not analytic when $$x = 0$$. In particular, it is so bad that not only is it not analytic, the Taylor series exists and even converges ... just to the wrong thing!:

$$U(x)\ "="\ 0 + 0x + 0x^2 + 0x^3 + 0x^4 + \cdots,\ \ \ \ \mbox{near x = 0}$$

... and yes, that is literally 0s on every term, so the right-hand expression equals $$0$$!

(ADD - see comments: no... not THAT 0! ... uh ... Ooops... uhhh ... 😁)

Nonetheless, while that is technically "wrong", the usual analysis methods you have for this system will still tell you the "right thing", provided you're careful: in particular, we note that $$x = 0$$ looks like some kind of "equilibrium" since $$U'$$ is zero there, but we also note that we are told - correctly! - that we should not apply the harmonic oscillator approximation because we also have that the coefficient out in front of $$x^2$$ is 0 as well.

We are justified in both conclusions because while this Taylor series is "bad", it is still A-OK by Taylor's theorem to write the truncated series, and thus Taylor polynomial,

$$U(x) \approx 0 + 0x + 0x^2,\ \ \ \ \mbox{near x = 0}$$

even though it "equals $$0$$", because this $$U(x)$$ is "so exquisitely approximated by the constant function $$U^{*}(x) := 0$$" that it is $$o(|x|^N)$$ for every order $$N > 0$$ and thus, in particular, also $$N = 2$$! Hence, the harmonic analysis and conclusion of failure thereof are still 100% justified!

ADD (IE+1936.6817 Ms - 2018-05-16): Per a comment added below, there is an additional wrinkle in this story which had been thinking of mentioning but didn't, yet for which, in light of that, I thought maybe I now should.

There are actually two different kinds of ways in which the Taylor series can fail when a function is not analytic at a point and it is taken at that point. One of these is the way I showed above - where the Taylor series converges, but it converges to the "wrong" thing in that it does not equal the function in any non-trivial interval around that point (you might be able to have it equal it on some weird dusty/broken-up set, but not on any interval), i.e. no interval $$[a - \epsilon, a + \epsilon]$$ with $$\epsilon \ne 0$$. Such a point is called a Cauchy point, or C-point.

The other way is for the Taylor series to have actually radius of convergence 0, i.e. it does not converge in any non-trivial interval of the same form with $$\epsilon \ne 0$$. This kind of point is called a Pringsheim point, or P-point. This case was not demonstrated, but even in such a case, the Taylor series is still an asymptotic series in the sense that it will at least try to start to converge if you're close enough and, moreover, the closer you are to the expansion point $$a$$, the more terms you can take before it stops converging and starts to diverge again. Since in physics, we are usually interested - and esp. for the harmonic oscillator - in only a few low-order terms, the ultimate behavior of the series is not important and we can still take it to get, say, the harmonic approximation near a point of equilibrium even if the function is not analytic there - e.g. consider the potential $$U_3(x) := U(x) + \frac{1}{2} kx^2$$ with $$k > 0$$, where we used the first potential we just gave above. This is not analytic at $$x = 0$$ either, but nonetheless, the harmonic approximation will not only work, but work exquisitely well, and with the frequency $$\omega := \sqrt{\frac{k}{m}}$$ as usual.

See:

https://math.stackexchange.com/questions/620290/is-it-possible-for-a-function-to-be-smooth-everywhere-analytic-nowhere-yet-tay

• "so the right-hand expression equals 0!" No, it does not equal 0! = 1. :P May 15, 2019 at 11:51
• @The_Sympathizer /r/UnexpectedFactorial May 15, 2019 at 12:30
• @Atom : I didn't say you couldn't. In fact, the answer was to show that you could, even in a nonanalytic case, because you're always truncating at a finite number of terms in such analyses. Effectively, the Taylor series always gives you the "local" behavior of any smooth function, in the "best possible" way. It's just that for certain functions - i.e. analytic functions - you get the boon that it also gives you more! May 15, 2019 at 12:53
• Just one small caveat (I'm a mathematician, so allow me to be pedantic): one of the problem with non-analytic functions is that the interval in which the Taylor approximation is valid typically decreases very quickly as $N$ increases, so some care still needs to be taken to check that the approximation we're making is valid in the range of values of $x$ we are interested in. May 16, 2019 at 6:44
• @LSpice : The error term decreases for a while in the beginning, then starts increasing again, never to come back. May 17, 2019 at 22:24

The Stone-Weierstrass theorem says that any continuous function on a compact interval is arbitrarily well approximated by polynomials. Thus, as long as we're only interested in explaining experimental results (and not in the exact solutions of theoretical models), series expansions are plenty good enough. That is, for whatever we'd like to describe, there is a model (at least in the statistics sense) which describes it to any good enough accuracy and which is analytic. So it doesn't seem obvious that we'll ever be able to tell whether the world is analytic or just $$C^\infty$$, or even $$C^0$$!

Of course in practice, our theories make infinitely many predictions for the values of such functions, for instance mechanics predicts them as solutions to differential equations, field theory by some integrals. Typically we cannot evaluate our theoretical predictions exactly and so we use numerics or asymptotic series methods. Things that come out of our models tend not to be analytic, so I think we're a bit spoiled in our physics education with all these analytic and solvable models.

The question of why so many (but not all!) exact solutions to theoretical models are real or even complex analytic is a whole different discussion, and much more mysterious, although causality does have some bear on it. For instance, response functions in time always have an extension to complex time in the upper-half-plane.

But more mysteriously, there are things like the KdV equation, the first equation to describe solitons, whose integrability turned out to be closely related to elliptic curves. So integrability seems not just related to analyticity, but even to algebraicity! But it is a rather hidden connection, because the solutions to KdV themselves are transcendental. Anyway, I do recommend the book I linked. It's written for undergraduates and it's a lot of fun.

• but this polynamial will not in general be the same as taylor expansion around some point. So it doesn't explain why taylor expansion is so broadly used. May 15, 2019 at 9:49
• @Umaxo It doesn't really matter, so long as we keep track of our errors. Taylor's theorem says that as you take more terms, the approximation gets better. One could in principle use some other sort of approximating polynomial, such as the Bernstein polynomials: en.wikipedia.org/wiki/Bernstein_polynomial . The point is because the function has a good analytic (even polynomial!) approximation, its Taylor series will be well-behaved. May 15, 2019 at 9:52
• @Atom I suppose there's two questions to ask, 1. why are the mechanical and other laws so simple? For instance, the equations of motion in rigid-body mechanics are all second order differential equations. If this is an approximation to reality, it is certainly a very good one. And 2. why are exactly-solvable models so useful? The solution to most mechanical problems will be chaotic, and not analytic, but still so many things we ecounter have an exactly solvable version that we can learn from. (There are exceptions though, like turbulence.) For these models we really do write the exact solution May 15, 2019 at 10:03
• @Ryan Thorngren that is not true. The approximation gets better only for analytic functions, not nonanalytic ones. F.e. take the famous $f(x)=0;x\le 0$ and $f(x)=e^{-1/x};x >0$ on the domain $x \in [-2,2]$. The taylor series around, for example, point $x=1$ will get closer and closer to $f(x)=e^{-1/x}$ on the whole compact domain and will fail to be zero for all points $x<0$. The polynom from SW theorem will be different and taylor expansion cannot be used May 15, 2019 at 10:27
• @Atom Whether the solution is analytic or not is whatever comes out of the equations. They typically have unique solutions once the boundary conditions are specified, and that solution is often analytic. May 15, 2019 at 11:34

If we know the value of $$f$$ at $$t$$, and we want to know the value of $$f(t+\Delta t)$$ for small $$\Delta t$$, then the most basic thing to do is to just assume that $$f(t+\Delta t) = f(t)$$. This is known as the "zeroth order approximation". In calculus, you learned about tangent lines. With a tangent line, instead of approximating the function with a fixed value, you approximate it with a line, and the slope of the line is the derivative: $$f(t+\Delta t) = f(t)+(\Delta t) f'(t)$$. This is the "first order approximation". So this is treating the derivative as a constant, i.e. a first order approximation of the function is given in term of a zeroth order approximation of the derivative.

We could instead calculate a first order approximation of the derivative, and use that to approximate the function. This would then be a second order approximation of the function. We have $$f'(t+\Delta t) = f'(t) +(\Delta t)f''(t)$$, and integrating that we get $$f(t+\Delta t) = f(t)+(\Delta t)f'(t)+\frac {(\Delta t)^2}{2}f''(t)$$. We can continue this process, and the nth order approximation will then simply be the first n (with zero indexing) terms of the Taylor Series. This isn't assuming that the function is analytic; it's simply applying an intuitive strategy to approximate the function.

So is it valid? Well, if we have a bound on the nth derivative of $$f$$ over the interval, then we can use that to put a bound on the (n-1)th derivative, which can then give a bound on the (n-2)th, and so. So even without knowing the $$f$$ is analytic, having a bound on the nth derivative gives a bound on the error for the nth order approximation.

So even if the nature works smoothly in its endeavours, it is essentially zero probability that it should do so analytically

Where does this implication come from? Most equations that we use in physics are solved by functions that are indeed analytic: this is because most equations are differential up to some order and you can use Cauchy conditions to prove it. In those cases using expansions in power series is justified and you commit an error on the predictions proportional to whichever order you choose to stop at.

In some other cases functions are not analytic and in fact one does not apply such expansions blindly: there is a whole area of classical electrodynamics dealing with singularities in Green's functions and residual's theorem (just to mention one) or investigation of poles of the Lagrangians in QFT (to mention another one).

Something additional you should be aware of:

In order to talk about analycity, you have to talk about derivatives, and in order to talk about derivatives, you need limits. In fact, even to talk about continuity, you need limits. And to talk about limits, you have to have things defined to infinite precision.

But the whole concepts by which physical values are defined break down when you push the accuracy too far. If you want to talk about the velocity of an object, you have to be able to define its position. But that requires an exact definition of what the object is. Real physical objects shed atoms to the environment and gain other atoms from it. At any one time, how do you decide exactly which atoms are part of your object and which are not? Most of your object is open space between those atoms. How do you define exactly where the boundary is between the open space that contributes to the volume of your object and the open space that is outside?

Beyond that, it is believed that there is a fundamental lower limit to measurable size: the Planck length. Nothing smaller can be measured (and currently, nothing a whole lot larger can be either). But limits cannot be defined with such a restriction. The definition of a limit admits no lower bound on how close you can get to the target. Thus it makes no sense to talk about limits involving physical measurements. And if you cannot talk about limits, you cannot talk about continuity or derivatives or analycity or Taylor series.

If you were to take the most comprehensive measurements of an object as it moves that are even theoretically possible, you would not end up with a well-defined mathematical function. Instead, you would end up with data having error ranges into which an infinite number of functions would fit. Among those functions will be some that are quite badly behaved - not continuous at any point. But also among those functions will be some that are analytic.

Any of these functions would provide an equally accurate description of your object's motion. The discontinuous functions are difficult to work with, but the analytic functions have excellent behavior. It's your choice. Which will you prefer to use?

I see the core of your question being the statement:

So even if the nature works smoothly in its endeavours, it is essentially zero probability that it should do so analytically

This is besides the point. We:

• Observe phenomena
• Build models of these using analytic functions
• Mostly because it is easier
• Use maths to analyse those and create predictions
• Test those predictions
• If those predictions are borne out, then we can consider the model useful.

No-where does this assume knowledge of how nature works, or that the underlying structure of the universe is something logically equivalent to being analytic

So, what you are really observing is that models based on analytic functions are useful.

Adding to Sympathiser's answer -- one can see why the existence of functions like $$e^{-1/x}$$ is not surprising by rephrasing them as "functions that approach zero near zero faster than any polynomial". This is not fundamentally more surprising than e.g. functions that grow faster than every polynomial -- in fact, for any function $$f(x)$$ that grows faster than every polynomial, the function $$\frac1{f(1/x)}$$ approaches zero near zero faster than any polynomial.

So for rapidly growing $$f(x)=e^x$$, one gets the corresponding smooth non-analytic $$e^{-1/x}$$. For $$x^x$$, one gets $$x^{1/x}$$. For $$x!$$, one gets $$\frac{1}{(1/x)!}$$, and so on.

See my post What's with e^(-1/x)? On smooth non-analytic functions: part I for a fuller explanation.