Local inertial reference frames are defined as follows:
Pick a set coordinates for the manifold (assuming the manifold can be described by global coordinates) $\{x^i\}$ such that $g(p) = \eta$ is the metric at a point $p$ at the manifold and the first derivative of the metric coefficients will vanish at the point $p$.
But the problem that I have is not this formal definition, which is mathematically clear, but the physics is difficult. Since in special relativity we defined inertial frames as frames that have no acceleration (which has to be absolute). But if we look for example how the equivalence principle is described, then people always tend to use accelerating elevators and then conclude that locally the system should be inertial, but I do not get this statement. How can an object be accelerated and describe an inertial frame locally at the same time? The statement "we can apply SR" locally does not make quite sense. Mathematically it does, since we showed that the metric is the Minkowski metric at a point $p$ so that in the tangent space (isomorphic to Minkowski space, only at $p$) we can define 4-vectors and so on, but physically it is not obvious for me.
Could somebody elaborate on this notion?