Let us suppose that the 4-momentum of the vector boson points in the z-direction ($p^\mu=(E,0,0,p_z)$).
A basis for the polarization vectors satisfying $p\cdot \epsilon,~\epsilon^{*}\cdot \epsilon=-1$ is
$$
\epsilon_1^\mu=(0,1,0,0),~\epsilon_2^\mu=(0,0,1,0), ~\epsilon_L^\mu=(\frac{p_z}{m},0,0,\frac{E}{m})$$.
In the infinite momentum limit $E=p_z+\mathcal{O}\left(\frac{m^2}{p_z}\right)$.
With this in mind the longitudinal polarization vector takes the form
$$\epsilon_L^\mu=\frac{p^\mu}{m}+\mathcal{O}\left(\frac{m}{E}\right)$$.
As you mentioned and as we see above the longitudinal polarization vector blows up in the high-energy//massless limit. This problem is resolved if we consder how the photon couples. The photon generally couples to some current $J^\mu$, via $A_\mu J^\mu$. Thus if we want a smooth massless limit we have to ensure that $p_\mu J^\mu=0$ or in position space $\partial_\mu J^\mu=0$, i.e the current is conserved.
We can also see this from the fact that a massless vector-boson has 2-degrees of freedom (helicity +1,-1) while a massive has 3(helicity +1,0,-1). The current conservation condition then ensures that the 0 helicity states are not emitted in the limit of vanishing mass.