The justification of the compactification to $S^3$ and $S^4$ is different.
In the first case (compactification of space), the compactification can be explained as follows: (This is a plausible physical explanation, not a complete mathematical proof).
We believe that the Skyrme model explains the low energy behavior of QCD. There are plenty of experimental results supporting this assumption. In particular, this model is able to predict certain properties even for heavy Baryons within 10% of the experimental values.
The Baryon number in this model is given by:
$$B = \frac{1}{24\pi^2} \int_{\mathrm{space}}\mathrm{Tr} \left ( U^{-1}dU \wedge U^{-1}dU \wedge U^{-1}dU \right ) $$
If space is flat, the baryon number of any finite mass baryon vanishes. Thus flat space does not support baryons. Please, observe that this consequence is very strong physically because it tells us that space-time which is a solution of Einstein's field equations of gravitation must be compact on the spatial slices.
In the instanton case, we can still be in a physical Minkowskian space-time. The solutions in the Euclidean signature just correspond to tunneling events in the physical space time. This is the basic trick of the usage of the Euclidean signature. It happens that these solutions if restricted to finite energy must vanish at the Euclidean infinity, thus effectively describing solutions on a compactified Euclidean space-time, but the amplitudes of these solutions correspond to true tunneling amplitudes in the physical Minkowskian space time.
Remarks:
Mathematicians(1): Mathematicians do not have interest in a physical explanation of why space-time is compactified. They choose whatever space-time suiting the mathematical result they require. So I don't think that you can find this type of explanations in mathematically rigorous works.
Mathematicians(2): Mathematicians engaged in quantum field theory research use functorial qft machinery (especially in tqft). According to this way of thinking a quantum field theory is just a black box which accepts a manifold as an input and returns a Hilbert space in the output, i.e., the same theory is not defined on a single space-time manifold, and can be simultaneously used on compact and non-compact manifolds with Minkowskian or Euclidean signatures.
Large gauge: You cannot apply large gauge transformations on instantons, because if you do so you get another inequivalent configuration with a different instanton number. Large gauge transformations are not redundancies in the description of the field theory as are small gauge transformations. Large gauge transformations are symmetries connecting inequivalent configurations. They are singular at infinity therefor unacceptable gauge transformations. This subject was discussed here on PSE in several occasions.