After reading all the very good answers carefully I think I understood what is going on and decide to post a summary of it.
First, let $G$ be an arbitrary Lie group with Lie algebra $\mathfrak{g}$. Let further $\mathscr{D} : G\to GL(V)$ be a representation of $G$. Then, $\mathscr{D}$ gives rise to a representation of the Lie algebra $\mathfrak{g}$ by differentiation at the origin.
In fact, the exponential map $\exp : \mathfrak{g}\to G$ is surjective onto a neighbhorhood $U\subset G$ of the origin. Thus we are able to write for any $g\in U$
\begin{equation}g=\exp \lambda X,\quad X\in \mathfrak{g}.\end{equation}
This allows us to define $D : \mathfrak{g}\to \operatorname{End}(V)$ by
\begin{equation}D(X)v=\dfrac{d}{d\lambda}\bigg|_{\lambda=0} \mathscr{D}(\exp \lambda X)v\end{equation}
One would question if all representations of $\mathfrak{g}$ arise in this way. In fact there is a result saying that this is true only when $G$ is simply connected. So the first point of importance is the following:
If $G$ is a Lie group with Lie algebra $\mathfrak{g}$ the representations of $G$ descend to representations of $\mathfrak{g}$. On the other hand, in general not all representations of $\mathfrak{g}$ arise in this form. In the special case that $G$ is simply connected, then it is true and all representations of $\mathfrak{g}$ arise as derivatives of those of $G$.
When $G$ is not simply connected, then its universal cover $\tilde{G}$ is, and they both share the same Lie algebra. So in that case, we might say that the representations of $\mathfrak{g}$ which do not arise from derivatives of representations of $G$ instead arise from derivatives of representations of the universal cover.
Second, let now a representation $D : \mathfrak{g}\to \operatorname{End}(V)$ be given and suppose that it is one of these representations that arise from a representation $\mathscr{D}:G\to GL(V)$ of $G$. The question is: can $\mathscr{D}$ be determined from $D$?
The answer is that we know how to do it in a neigbhorhood of the identity. Again, $\exp : \mathfrak{g}\to G$ is surjective onto a neigbhorhood of the identity $U = \exp(\mathfrak{g})$. In that case, for $g\in U$ there's $\lambda$ and $X$ such that $g = \exp (\lambda X)$.
We thus define $\mathscr{D}$ by $$\mathscr{D}(\exp \lambda X)=\exp \lambda D(X).$$
This recovers $\mathscr{D}$ out of $D$ inside the open set $U$ only. So the second important result is the following one:
Let $G$ be a Lie group with Lie algebra $\mathfrak{g}$. Let $\mathscr{D}:G\to GL(V)$ be a representation of $G$ that descends to $D : \mathfrak{g}\to \operatorname{End}(V)$. Then we can recover $\mathscr{D}$ out of $D$ in the neighborhood $U = \exp(\mathfrak{g})$ of the identity reconstructed by the exponential.
Third, we ask when this is enough to understand $\mathscr{D}$ completely. The answer lies in the following theorem: Let $G$ be a connected topological group, then any neighborhood of the identity generates $G$.
To prove this fact recall that in a connected topological space the only sets which are open and closed at the same time are $\emptyset$ and the whole space, thus define $S$ the set generated by a neighborhood of the identity and show $S$ is both open and closed. In other words: in a connected topological group any group element is a finite product of elements in a neighborhood of the identity.
So now suppose $G$ is connected. The theorem applies in particular to the exponential neigbhorhood $U = \exp(\mathfrak{g})$. In that case, knowing the representation as an exponential inside $U$ is enough to determine it everywhere since a general group element is just a product of such exponentials! If $G$ is not connected, this applies to the connected component of $G$ containing the identity. In fact, this leads to the third point of importance:
Let $G$ be a Lie group with Lie algebra $\mathfrak{g}$. Let $\mathscr{D}: G\to GL(V)$ be a representation of $G$ that descends to $D : \mathfrak{g}\to \operatorname{End}(V)$. Since $U = \exp(\mathfrak{g})$ is a neighborhood of the identity, it generates the connected component of $G$ containing the identity $G_e$. Thus, we recover $\mathscr{D}$ inside $G_e$ by noticing that for a general $g\in G_e$ there are $g_1,\dots,g_k\in U$ with $ g= g_1\cdots g_k$ and that $g_i = \exp \lambda_i X_i$. Hence we get \begin{equation}\mathscr{D}(g)=\exp \lambda_1 D(X_1)\cdots \exp \lambda_k D(X_k).\end{equation}
This is the general story that enables us to look at the Lorentz group. So let $O(1,3)$ be given, we wish to understand its representations. Now by the above methods, we are able to get the representations of $O(1,3)$ in the connected component with the identity by exponentiating its Lie algebra $\mathfrak{so}(1,3)$ representations. Since $O(1,3) = SO_e^+(1,3)\rtimes D_4$ where $D_4$ is generated by parity and time reversal, understanding the representations of $SO_e^+(1,3)$ is already enough.
Finally, in general the elements of $SO_e^+(1,3)$ would be given by finite products in the image of the exponential so that the representation of a general $g\in SO_e^+(1,3)$ would be a finite product of exponentials and this would be fine. It so happens, however, that the neighborhood $U = \exp(\mathfrak{so}(1,3))$ recreated by the exponential is in fact $SO_e^+(1,3)$, the consensus in the literature being that this is already a non-trivial result.
Being more precise: for $SO(1,3)$ the exponential is surjective onto the connected component with the identity. So in fact, this simplifies matters and any $g\in SO_e^+(1,3)$ takes just a single exponential to be represented. This is not an essential point though, because even if it weren't the case, knowledge of the representation on the exponential neighborhood would still give the understanding of the representation as a whole, it is just one very nice state of affairs which makes matters much simpler.