What is an intuitive way of understanding Ostrogradsky instability? Is there a way of explaining why no differential equations in physics exceed order two without delving into Lagrangian and Hamiltonian mechanics - i.e. from Newtonian mechanics? Moreover, is there such an alternative proof, which focuses more on physical reasoning than abstract mathematics?
 A: Without the framework of Hamiltonian mechanics, it would be difficult or impossible to establish the Ostragradski instability in full generality. But there is a simple example we can look at that has an Ostragradski instability that can be understood from Newtonian mechanics.
Let's start with
\begin{equation}
S = \int {\rm d}t \left(-\frac{\alpha}{2} (\ddot{x})^2 + \frac{Z}{2} \dot{x}^2 - \frac{\omega_0^2}{2} x^2\right)
\end{equation}
The equations of motion are fourth order if $\alpha$ is not zero
\begin{equation}
\alpha \frac{{\rm d}^4x}{{\rm d}t^4} + Z \frac{{\rm d}^2 x}{{\rm d}t^2} + \omega_0^2 x = 0
\end{equation}
To solve this, we can make an ansatz $x=Ae^{Bt}$, leading to an equation for B
\begin{equation}
\alpha B^4 + Z B^2 + \omega_0^2 = 0
\end{equation}
Now, if $\alpha$ were zero, we would have two roots, $B=\pm i \omega_0/\sqrt{Z}$. Crucially, the roots are purely imaginary. Therefore the solution oscillates, rather than growing exponentially.
On the other hand, if $\alpha$ were not zero, there are more roots. (This is sort of the main physics point, is that there are additional degrees of freedom embedded in the higher order equations of motion). Let's set $Z=0$ for simplicity. In this case we can write the roots as $B= (\pm 1 \pm i) \sqrt{\omega_0}/(4\alpha)^{1/4}$.
Now you can see that $B$ has a real part. This means that the general solution will have an exponentially decaying mode $\sim e^{-\sqrt{\omega_0}/(4\alpha)^{1/4} t}$ as well an exponentially growing mode $\sim e^{+\sqrt{\omega_0}/(4\alpha)^{1/4} t}$. The exponentially growing mode is the instability -- for general initial conditions, you will excite this mode.
Here is another way you can see the instability.
Imagine we drive the system with a sinusoidal driving force with frequency $\omega$. For simplicity set $Z=0$ and $\alpha=-1/\omega_0^2$, so the equations of motion are
\begin{equation}
-\frac{1}{\omega_0^2}\frac{{\rm d}^4x}{{\rm d}t^4}  + \omega_0^2 x = J e^{i \omega t}
\end{equation}
Then making an ansatz $x=A e^{i \omega t}$ we would find
\begin{equation}
A(\omega) = \frac{J\omega_0^2}{\omega^4  - \omega_0^4} = \frac{J}{2} \left[\frac{1}{\omega^2-\omega_0^2} - \frac{1}{\omega^2+\omega_0^2}\right]
\end{equation}
The first term, with the denominator $\omega^2-\omega_0^2$, is what you would expect from a driven harmonic oscillator. The second denominator $\omega^2+\omega_0^2$ has a zero with a positive imaginary part, which means there will be an exponentially growing solution excited by the driving force.
