# WHY does the "order" of a differential equation = number of "energy storage" elements in a system?

OK. in all engineering courses there comes a point when they introduce you to systems theory and modeling of systems (for eg. via the impulse response) and then the Laplace transform.

The modern approach to this includes the single order "state space" representation of systems, where as a strategy for finding the "state variables", it is recommended that the energy storage elements be targeted as the state variables $x$ and $\dot x$.

Magically, all these courses ONLY allude to the fact that "the order" of a differential equation describing the system of interest becomes "the order" of the polynomial in the Laplace s-plane, which all turn out (the magic bit) to match THE NUMBER OF ENERGY STORAGE ELEMENTS in the system of interest.

Question: what is the deep phenomenon whereby this holds? [aside: I looked a bit at the Lagrangian framework of physics w Noether's theorem etc (which to be frank I never learned at all about in school) but that doesn't seem to cover this curious symmetry or fact regarding energy storage].

The conclusion that seems to filter up from engineering systems courses is that anytime a derivative is involved in a physical system's physics, that is indicative of an "energy storage" going on! [my reaction: how bizarre...!]

Can anyone shed light please?

• aside, on the Lagrangian physics aspect: would the operation of "differentiation" have an implication as a "symmetry" in physics? could differentiation imply energy element present? Aug 5 '14 at 6:38

I'd love to see more elaborate answers to this, but generally the idea is intuitive. Resistors are linear and only dissipate energy. Inductors and capacitors are nonlinear and store magnetic and electric energy respectively. Their constitutive relations are the following:

Inductor: $v = L\frac{di}{dt}$ Capacitor: $i = C\frac{dv}{dt}$

It's only when you have these last two elements that you have derivatives in your equations because of their constitutive relations. For this reason, it makes sense that (derivatives) => (energy storage elements).

The reason why the order determines the number of energy storage elements is more mathematical. Imagine you have a series RLC circuit (two energy storage elements L and C), and you write the loop equation for the voltage drops in terms of the loop current.

$0 = Ri + \frac{1}{C}\int i dt + L\frac{di}{dt}$

In order to turn this integro-differential equation into a differential equation in the current, you'd normally use Laplace theory or simply take the derivative with respect to time of both sides. Then,

$0 = R\frac{di}{dt} + \frac{i}{C} + L\frac{d^2i}{dt^2}$

and you see, you had to take as many derivatives as you needed to eliminate the integrals. In the end, your order is 2 for this particular example, which is exactly the number of energy storage elements. But this is a more general idea -- the number of derivatives you take will eventually set things such that the order of your equation is the number of energy storage elements.

EDIT: Additionally, if you take the Laplace transform of the above equation, you see that the highest power will be $s^2$, and so your order is 2. More generally, the order of your algebraic equation after doing the Laplace transform will also be equal to the order of your time-domain differential equation, and hence the number of energy storage elements.

• thank you for showing a vivid example of the question. systems theory also generalizes the concepts of "resistance" and "capacitance" to other fields like liquid levels (eg C :: mass of liquid for a given head), pressure systems, thermal systems (where C :: specific heat) and so on. This is of course part of the magic built in into the Laplace transform: viewing analog-computer type operations on the signals independent of the physical signal so thus the same characteristic equation can have analogues to completely different equipments (thermal, hydraulic..etc) Aug 5 '14 at 6:30
• I remember thinking why systems theory randomly drew analogies between different systems out of the blue. It's all about the similar differential equations. don't forget the green check mark on the answer ;)
– eqb
Aug 5 '14 at 7:05

This question has been marinating in my own mind for some time since asking here, and seeing how the respected community of this site has not yet answered this head on in the awesome manner I have seen other answers, I will share my "answer" as far as i have been able to find it on my own. [EDIT: this answer has been edited to more directly answer the question correctly.]

First, the question is reasonable. I have looked at many different engineering texts and they all literally imply that Energy Storage and State variables are indeed "related" or the same thing, only no one has stopped to wonder "why" or "how" this is magically connected to the governing differential equation. Here is a direct quote from Prof. BC Kuo's text Automatic Control Systems 1982 edition: "The reason for this choice [of state variables] is because the state variables are directly related to the energy storage elements of a system". See also Prof Erik Cheever page (http://lpsa.swarthmore.edu/Representations/SysRepAll.html). Also the order of differential equation describing a system and number of energy storage "devices" in a system is THE SAME.

Above, where i said "anytime a derivative is involved in a physical system's physics (ie. it's mathematical model), that is indicative of an "energy storage" going on" is literally what is happening! For eg. systems texts recommend the direct method of decomposition of transfer functions into state equations (cf below). In this method, in the s-plane, the numerator and denominator of the transfer function are multiplied by X(s), a dummy state variable. Then, numerator & denominator are divided by $s^n$ (where n is the highest order derivative). with simple algebraic manipulation, now you have expressed the transfer function in terms of integrators ${s^-}^1$, ie. state variables ie. energy storage elements. (From differential equation to energy storage elements.)

The reason the highest order of the derivatives of differential equations describing a system equals the number of energy storage elements is because systems with "energy storage" have "memory", ie. their responses to an input depend on not only the current value of the input, but also on the past history of inputs. Thus, their behavior is not simple, but "dynamical", or, they have a state dependent output. The same input does not lead to the same output every time b/c of this "memory" effect.

It turns out this "memory effect" is the most abstract, distilled, and surprisingly correct, mathematical, high-level definition of the property physicists call "energy" which is taken as a "given" in physics. Energy is state.

We will see that systems engineers model state (or energy) as merely the output of integrators eg $\int y dx$. It makes sense..., the simplest form of preserving the past of any variable in a compact fashion, is to SUM it, ie. integrate wrt time, like how accountants summarize past cash flows into an account with a sum ("balance"), like how card counters at blackjack don't have to memorize the entire past sequence of play of cards but can use a simple summation via the "hi-lo method" of one Prof Thorp (using values of only +1, 0, -1) to arrive at the same results!. Integration = state = energy storage. These latter two are examples of information-theoretic "energy".

(Aside: and just like a physicists are familiar with the fact that the change in potential energy around a closed loop = 0; likewise Prof Thorp demonstrated that the "state"/energy of the hi-lo card counting method also returns to the same value as you keep summing through the remaining deck of unused cards and back to present hand through the used cards.)

Given that systems with internal energy storage are dynamical and their output/response is not merely a function of present input but also it's state/energy, this is most directly modelled by differential equations because at it's most elementary definition, a derivative, $\frac {dy}{dx}$ depends not only on the present y(t) and x(t) but also their value at an EARLIER infinitesimal moment, x(t-dt) and y(t-dt) as everyone knows. Eg. if ${dy \over dt} = x(t)$, then knowing x(t=present) is NOT enough to tell you the value of y(t).

(In the earlier version of my answer I was pursuing a totally WRONG line of thought whereby I was under impression that a new (the fourth) law of thermodynamics was waiting to be discovered, where the order of the rate of change of work applied (input) to an isolated system must equal the number of modes of lumped energy storage within the system). This is INCORRECT b/c the act of differentiation does NOT affect the input variable at all, but the highest order derivative appears as an input only to the first of the daisy chained/catenated energy storage elements).

The only hyper generalization one can make (as in from a "thermodynamic law" viewpoint (lol) ), is that the natural response of a system will be a sum of a number of exponentials which matches the number of energy storage elements.

1. The term, "Systems" are an engineers view of physics, strictly modelled in terms of such driving input (cause) leads to such and such output (effect). The system is studied in isolation to other environmental elements by means of lumped Linear Time Invariant elements. S.J. Mason (of the Mason signal flow graph) introduced linked cause and effect diagrams in his Sept 1953 paper in Proceedings of IRE which are the bedrock of this engrg field today.
• For math, we have MathJax to imitate LaTeX. For a more thorough guide as to what you can do with it, see here. As for the low activity on this question, I suspect it has to do with different ways of communicating between physics and engineering. Your question is intriguing (and I'm digesting your answer), but I must say I never deal with Laplace transforms, nor do I think in terms of energy storage units.
– user10851
Aug 27 '14 at 2:20
• thanks for this link. this tutorial link should be on top of every stackexchange page! Sep 4 '14 at 1:57

Completely unexpectedly, the rigorous answer (less "hand-waving" answer) lies in Quantum Mechanics!

It is never explained in undergraduate engineering classes and texts as to WHY the statement regarding Energy Storage is so casually slipped in to a discussion of Laplace Transforms applied to the descriptive/governing differential equations of time that define a "system" (eg. block and spring, motor arm etc are classic cases in undergraduate texts).

[Specifically, every time the governing equation of a system has a derivative of time, this Laplace-Transforms into a state variable "s", with higher orders of "s" representing transforms of higher order derivatives of time. The highest exponent of "s" in the transformed equation is called the "order of a system" also corresponds to the number of "energy storage elements".]

Professor Susskind's Lecture on "Time Evolution of States" lays it out! (http://www.lecture-notes.co.uk/susskind/quantum-entanglements/lecture-8/time-evolution-of-states/)

It is fascinating to read. And so directly connected to the application of Laplace transform theory and assigning of an "order, N" to a system:

1) All states must evolve linearly over time. This is "The Linearity Postulate" 2) The evolution of a system can be represented by an Evolution Operator, U, which has properties:

• Linearity
• Unitary
• Hermitian

This Evolution operator, under the above conditions, is represented by a Hamiltonian operator, H.

Susskind says, "H [the Evolution Operator] is called the Hamiltonian observable of the system. It corresponds to energy and its eigenvalues are the discrete energy bands associated with the system."

(I am not an expert in Quantum Mechanics, but I wanted to post this here in my excitement at finding it. It will take me a long time to digest it properly, however).