I'm aware this question was submitted in 2014, so it's quite likely that in the intervening years, you have found the answer to it.
But just in case I'm submitting this answer anyway.
The background of the answer I'm submitting here is the exposition of Hamilton's stationary action that I submitted on physics.SE in October 2021.
In this answer, I will discuss in which cases the true trajectory corresponds to a minimum of Hamilton's action, and in which cases the true trajectory corresponds to a maximum of Hamilton's action. I will also discuss which case is the critical case that is at the cusp of the flip from minimum to maximum
I will discuss the simplified case of motion in one spatial dimension; generalization to 3 spatial dimensions is straightforward.
I will in sequence discuss the following three cases:
- the motion is subject to a uniform force, hence the potential increases linear with displacement
- the motion is subject to a force that increases linear with displacement, hence the potential increases with the square of the displacement
- the motion is subject to a force that increases quadratic with displacement, hence the potential increases with the cube of displacement
The variation of the trial trajectory is variation of the position coordinate.
As variation of the trial trajectory is applied the evaluation compares the response of the kinetic energy to the response of the potential energy.
The rate of change of the trial trajectory propagates to the velocity along the trial trajectory. As we know: the expression for kinetic energy is proportional to the square of the velocity. Therefore the response of the kinetic energy to variation is in all cases a quadratic function.
Potential increases linear with displacement
When the potential energy increases linearly with displacement we have that the response of the potential energy to variation of the trial trajectory is linear.
Because of that: when the potential energy increases linearly with displacement the true trajectory corresponds to a minimum of Hamilton's action.
Potential increases quadratic with displacement
As we know: in the idealized case of a force that increases in exact proportion to the displacement (perfect Hooke's law) the solution to the equation of motion is harmonic oscillation.
As we know: idealized harmonic oscillation has the following property: the period of oscillation is independent of the amplitude. Phrased differently: given a particular quadratic potential every amplitude of oscillation has the same period of oscillation.
Hamilton's stationary action reproduces the above amplitude property.
Evaluate a quadratic potential over a time interval that is equal to half a period of a full oscillation. So: if the period of oscillation is $2\pi$ seconds, then evaluate from $t=0$ to $t=\pi$ In that case, with the time interval equal to half a period, setting the starting position coordinate to zero means the ending position coordinate will be zero.
The evaluation then comes out as follows: for every amplitude of oscillation, Hamilton's action evaluates to zero.
Hamilton's action evaluates to zero because in the case of harmonic oscillation both the kinetic energy and the potential energy respond quadratically to variation of the trial trajectory; the responses are equal.
Summarizing:
In the case of Hooke's law, evaluated for a time interval that is half a period of oscillation (or any integer multiple of that) we have that Hamilton's action evaluates to zero for every amplitude of oscillation.
It is a fundamental property of idealized harmonic oscillation that period of oscillation is independent of the amplitude. Hamilton's action complies with that: for every amplitude of oscillation Hamilton's action evaluates to zero
Therefore:
In order to calculate an actual amplitude of oscillation for some specific case an additional initial condition must be supplied. With just the boundary conditions the problem is underdetermined.
Potential increases in proportion to the cube of displacement
Let me make a comparison. In the physics of damping there is a natural subdivision in underdamping, critical damping, and overdamping. The cases of linear potential, quadratic potential, and potential-proportional-to-cube-of-displacement fall in an analogous subdivision.
Critical case:
Both the kinetic energy and the potential energy respond quadratically to variation of the trial trajectory
Under-critical:
Whenever the response of the potential energy to variation of the trial trajectory is of lower order than quadratic the true trajectory corresponds to a minimum of Hamilton's action.
Over-critical:
Whenever the response of the potential energy to variation of the trial trajectory is of higher order than quadratic the true trajectory corresponds to a maximum of Hamilton's action.
For emphasis let me repeat that:
There are classes of cases where the true trajectory corresponds to a maximum of Hamilton's action.
Note for instance that the Euler-Lagrange equation is agnostic as to whether the trajectory that it identifies corresponds to a minimum or a maximum of Hamilton's action. Whether Hamilton's action is a minimum or a maximum is not relevant. The only relevant property is that you are identifying the point where Hamilton's action is stationary.
There is a twist, however, in the over-critical case.
If the potential energy as a function of position is proportional to the cube (or higher) the evaluation of Hamilton's action must be performed over a sufficiently long time interval.
To see why, draw in the same diagram a quadratic function $f(x)=x^2$ and a cubic function $g(x)=x^3$. Given enough distance along the horizontal axis the cubic function will always outrun the quadratic function, but the cubic function is at the start slower.
So there is a scaling issue. Below some scale the quadratic function is steeper than the cubic function; for the cubic function to win the day the evaluation must extend over a sufficiently long scale.
Because of that scaling issue: when narrowed down to a sufficiently narrow specific interval of time Hamilton's action will be a minimum, even with a higher-than-quadratic potential.
My assessment is that Landau noticed this behavior in mathematics, and he subsequently decided to assert it as a property, be it without supplying an explanation. I assume that if Landau would have known the explanation he would have given the explanation.
'Stationary action' versus 'least action'
As we know, in a large majority of practical cases the response of potential energy to variation of the trial trajectory is of lower order than quadratic. The two biggies, gravity and the Coulomb force are inverse square force laws.
The critical case, harmonic oscillation, is ubiquitous too, of course. (Then again, it is the idealized case that is critical, and I'm not sure whether in classical mechanics there are instances where an oscillation is physically the idealized harmonic oscillation.)
Cases where the potential increases with the cube of displacement (or higher order) are rare, but they do exist.
The flip from minimum to maximum as you move from under-critical to over-critical demonstrates that fundamentally the criterium is to identify the point of stationary action.