The most geometric formulation of classical mechanics is in terms of symplectic geometry. However in terms of the question asked about the principle of least action, the issue is that this formulation
Objects seek a path that minimizes total action.
is mathematically vague, and that the phrase (and definition of action) can have different variational forms. Some of these are more intuitive and some are more general than others.
The most general formulation is Hamilton's Principle:
The motion of the system from time $t_1$ to time $t_2$ is such that the line integral
$I = \int_{t1}^{t2} L dt$
where $L= T - V$ is a stationary point for the path of motion.
The elegance of this is that it identifies one path from amongst all those between initial position at time $t_1$ and final position at time $t_2$ which extremizes the path. The points and paths here belong to the multidimensional configuration space of the system. Not only that but the remaining equations of motion can now be derived from this for all classical systems, with an extension into quantum mechanics as well.
However there are other (older) formulations of similar variational principles which can also refer to "action" and which have use in classical mechanics too. The classic text Goldstein introduces a "Principle of Least Action" which is defined a little differently. If we introduce the Hamiltonian of a system H and assume that it is conserved:
$\Delta \int_{t1}^{t2} \Sigma p_i q_{i}^{dot} dt = 0$
The term inside the integral is also known as the Action and the variation involved (which is minimized) is denoted by $\Delta$.
To understand this formulation let us consider only the simplest case of it: assume non-relativistic mechanics with no time dependence and no external forces. Then this expression reduces to:
$\Delta(t_1 - t_2) =0$
This says that of all paths possible between two points, consistent with conservation of energy, the system moves along that particular path for which the time of transit is the least.
Other variations of this formulation can have the particle moving along geodesics within configuration space. Some of these formulations are similar to Fermat's principle of Geometric Optics.