Preliminary remarks.
As Danu writes in his comment, the physics of the other four generators has to do with spacetime translations, one for each spatial direction, and one for time. But how do we see this explicitly in the math behind the somewhat odd-looking presentation of the Poincare group and its Lie algebra that Hall discusses.
First, recall that any $d+1$-dimensional Lorentz transformation is a Linear transformation on $\mathbb R^{d+1}$, so it can be representation by multiplication by a $(d+1)\times(d+1)$ matrix $\Lambda$.
Second, and most crucially, recall that translations of $\mathbb R^{d+1}$ are not linear transformations; there is no way to write spacetime translation as multiplication by a matrix.
However, here's the really cool thing. If we embed a copy of $(d+1)$-dimensional spacetime into the vector space $\mathbb R^{d+2}$, namely into a space with one higher dimension, then we can implement translations as linear transformations. Here's how it works.
The main construction.
For each $x\in \mathbb R^{d+1}$, we associate an element of $\mathbb R^{d+2}$ as follows:
\begin{align}
x \mapsto \begin{pmatrix}
x \\
1 \\
\end{pmatrix}
\end{align}
Now, for each Lorentz transformation $\Lambda\in \mathrm O(d,1)$, and for each spacetime translation characterized by a vector $a\in\mathbb R^{d+1}$, we form the matrix
\begin{align}
\begin{pmatrix}
\Lambda & a \\
0 & 1 \\
\end{pmatrix} \tag{$\star$}
\end{align}
and we notice what this matrix does to the embedded copies of points in $\mathbb R^{d+1}$;
\begin{align}
\begin{pmatrix}
\Lambda & a \\
0 & 1 \\
\end{pmatrix}
\begin{pmatrix}
x \\
1 \\
\end{pmatrix}
= \begin{pmatrix}
\Lambda x+a \\
1 \\
\end{pmatrix}
\end{align}
Whoah! That's really cool! What has happened here is that when we augment the dimension of spacetime by one with the embedding give above, and when we correspondingly embed Lorentz transformations and translations appropriately into square matrices of dimension $d+2$, then we actually do get a way of representing both Lorentz transformations and translations as linear transformations on $\mathbb R^{d+2}$ that act in precisely the correct way on the copy of Minkowski space embedded in $\mathbb R^{d+2}$!
In other words, the Poincare group in $d+1$ dimensions can be thought of as the set of all $(d+2)\times(d+2)$ matrices of the form $(\star)$ where $\Lambda\in \mathrm O(d,1)$ and $a\in\mathbb R^{d+1}$.
What about the Lie algebra?
A natural question then arises: what do the Lie algebra elements look like as matrices when we represent the Lie group elements this way? Well, I quick standard computation will show you that the Lie algebra of the Poincare group can, in this representation, be regarded as all matrices of the form
\begin{align}
\begin{pmatrix}
X & \epsilon \\
0 & 0 \\
\end{pmatrix} \tag{$\star$}
\end{align}
where $X\in\mathfrak{so}(d,1)$ and $\epsilon\in \mathbb R^{d+1}$, precisely as Hall indicates. But from the remarks above, we see clearly that the parameter $\epsilon$ precisely corresponds to the generators that span the subspace of the Poincare algebra that yield spacetime translations.
