In special relativity one assume that spacetime can be locally described by 4 coordinates, so it makes sense to model it as a 4-dimensional manifold.
I had the impression that it is assumed that there is an atlas of special charts called inertial reference frames. Their transition maps are restrictions of affine maps on $\mathbb{R}^4 \rightarrow \mathbb{R}^4$. The subgroup of invertable affine maps which are allowed as transition maps is called the kinematical group.
Einstein shows in his paper, I guess, that one can choose the kinematical group to be the Poincare group and for example [Bacry Levy-Leblond] showed that there would only be a few possible choices for these kinematical groups.
In the books, I am aware of, comes now a huge step and one immediately assume one is in Minkowski space.
So my questions are the following: Given a manifold $M^4$ with an atlas such that the transition maps are all restrictions of the affine maps of the Poincare group.
How can one construct a Lorentz metric (a (3,1)-semi-Riemannian metric) on $M$.
Does anyone know a reference for the fact that a manifold $M^4$ with affine transition maps is covered by an affine space (i.e. A simple connected one is topologically $\mathbb{R}^4$)?
One might be tempted to consider the above as something belonging to general relativity, as one uses charts etc. But note affine transition maps are a huge restriction and at the end (maybe after taking a covering map) one talking about Minkowski space anyways.