Forces affect the second derivative of positions. Waves are described by second derivatives. Etc. Why do much of physics operates on the second derivative?
What physical quantities, if any, operate on the third derivative or higher? Is there a fundamental reason why second derivatives are so fundamental to physics?
More generally saying, OP is asking why does the operator $$\Delta \equiv \partial^2_x+\partial^2_y+\partial^2_z$$ appear all over in Physics. The above operator is the so-called Laplacian operator which reduces to ordinary second derivative in the one-dimensional case.
You can find in this answer, Why does the Nature favor Laplacian.
While I am not aware of a reason for the frequent occurrence of second derivatives with respect to spatial coordinates, there is a reason for why very few, if any, physical laws depend on higher derivatives with respect to time: Ostrogradsky's instability.
In short, when one considers physical models with high time derivatives, one often will run into solutions that allow particles with negative kinetic energy—yes, this is weird, but the kinetic energy of these modes is given by expressions similar to $K = - \frac{m \dot{q}^2}{2}$. Due to this, one can obtain runaway solutions with a positive-energy particle and a negative-energy particle. More specifically, one can accelerate the positive-energy particle as much as desired by accelerating the negative-energy particle in the same way. This way, the two energies cancel and conservation of energy is ensured, but one still gets arbitrarily fast particles. This is often ruled out as unphysical.
A recent review of the Ostrogradsky instability can be found, for example, at arXiv: 2007.01063 [hep-th], which givens many examples on the context of Classical Mechanics, but also discusses applications to theories of modified gravity, such as galileons and degenerate higher-order scalar-tensor theories.
This explains, at least partially, why it is so rare for us to encounter physical laws with, for example, a third-order time derivative. While it is uncommon to find higher spatial derivatives, it does happen on a bunch of physical situations. For example, the Korteweg–De Vries equation, which occurs in Fluid Mechanics, reads $$\frac{\partial \phi}{\partial t} + \frac{\partial^3 \phi}{\partial x^3} - 6 \phi \frac{\partial \phi}{\partial x} = 0,$$ which includes a third-order spatial derivative. The biharmonic equation, $$\Delta^2 \phi = 0,$$ where $\Delta$ is the Laplacian, also occurs in some problems of elasticity (I believe Landau's book discusses this in much more detail).
In summary, high-order time derivatives are rare to occur due to Ostrogradsky's instability. High-order spatial derivatives do occur in some phenomena.
The reason why most physical theories contain at most up to second-order derivatives has to do with something called Ostrogradsky's instability. If you have a non-degenerate Lagrangian of the form $L(q,\dot{q},\ddot{q},...,q^{(N)})$ where $q^{(N)}$ is the N-th derivative of q with respect to time for $N>3$, the associated Hamiltonian is unbounded from below, which means that the theory is unstable. Here are some good links to read about this:
http://www.scholarpedia.org/article/Ostrogradsky's_theorem_on_Hamiltonian_instability
Although his work suggests that no physical theories can have time-derivatives of order greater than 2, it does not imply that no stable solutions exist for higher-order theories.
