I do know the principle of least action, but is it possible to formulate classical mechanics based on the principle of least time? That is, if we know the initial state $(x_i,p_i)$ of the particle and the external force field, then some variational problem would imply the classical path as a minimal time problem from the initial point $x_i$ to the final point $x_f$.
1 Answer
It's a fact that the action for a free relativistic massive point particle happens to be (proportional to) proper time $\tau$, which superficially sounds like the Fermat's principle of stationary time.
However, there are important differences and shortcomings:
On one hand, the massive point particle action stops being proper time if we try to include forces/interactions/potentials.
On the other hand, the Fermat's principle uses laboratory time $t$, and it is for massless particles, namely light.
A more fruitful strategy is to turn OP's question on its head, and instead try to derive Fermat's principle from the action for a relativistic massless particle in a curved spacetime. This is done in my Phys.SE answer here.