I am currently doing some Lorentz invariance exercises considering infinitesimal Lorentz transformations, and have been told to neglect second order differentials.
It's not the first time I have come across seeing something like this, and I wonder which is the justification behind this, I mean, is this a justifiable exact procedure or just an approximation?
is this a justifiable exact procedure or just an approximation?
Well it all depends on precisely what you are doing. Examples:
If you are attempting to determine what the Lie algebra $\mathfrak{so}(3,1)$ of the Lorentz group $\mathrm{SO}(3,1)$ is, then you are doing something exact because the Lie algebra of the Lorentz group is obtained precisely by considering a general one-parameter family $\Lambda(\epsilon)$ of Lorentz transformations that "begin" at the identity, $\Lambda(0) = I$, and then determining the derivative of such a family at $\epsilon = 0$. The result $X$ is an element of the Lie algebra (up to a factor of $i$ depending on your conventions): \begin{align} \Lambda'(0) = X \end{align} How is this related to neglecting second-order terms and higher? Well recall that we can Taylor expand the family $\Lambda(\epsilon)$ as follows: \begin{align} \Lambda(\epsilon) = I + \epsilon\Lambda_1 + O(\epsilon^2) \end{align} And now, if we take the derivative with respect to $\epsilon$ and then set $\epsilon$ to zero, we obtain precisely the coefficient of the term fist order in $\epsilon$; $\Lambda'(0) = \Lambda_1$, which is therefore an element of the Lie algebra of the Lorentz group.
If you are attempting to determine what happens if you perform a "small" Lorentz transformation, then considering a family $\Lambda(\epsilon)$ as above and only keeping the terms up to first order is an approximation, but it can be viewed as a more precise definition of "small."
If you are attempting to show that some object, like a term in a Lagrangian, or an action etc., is Lorentz-invariant, then often you only care to show that this invariance holds "infinitesimally," namely to first order. One reason for this is that such infinitesimal invariance is sufficient for the conclusions of Noether's theorem to hold, so in this context you don't so much care if the object being considered is invariant under a full Lorentz transformation.
Another important observation (as essentially indicated by Danu in his comment above) is that every element of the proper, orthochronous Loentz group $\mathrm{SO}^+(3,1)$ can be obtained by performing successive infinitesimal Lorentz transformations. The more precise statement of this fact is that
The exponential map $\exp:\mathfrak{so}(3,1)\to\mathrm{SO}^+(3,1)$ is surjective.
Concretely, this means that for every proper, orthochronous Lorentz transformation $\Lambda$, there exists an element $X$ of the Lie algebra $\mathfrak{so}(3,1)$ for which \begin{align} \exp X = \Lambda. \end{align} One way of thinking about this result is that it shows that the Lie algebra contains all of the information about the group in this case; every group element can be reconstructed by exponentiating a Lie algebra element. In this sense, you lose no information about proper-orthochronous Lorentz transformations by considering only first-order approximations to them.
You may also be interested in the following related physics.SE posts:
Rigorous underpinnings of infinitesimals in physics
Commutator of Lorentz boost generators : visual interpretation
