Consider an idealized dipole $\vec{p}$ moving along a trajectory $\vec{w}(t)$. The We assume that this dipole has a constant magnitude and direction in an inertial frame. This is somewhat artificial, particularly for motions along the direction of $\vec{p}$ (one would expect the dipole moment to Lorentz-contract); but the calculations below are complicated enough as it is, and so long as the speed of the dipole is non-relativistic, this assumptions should be valid.
The charge distribution for this dipole will be $$ \rho(\vec{r},t) = \vec{p} \cdot \vec{\nabla} \delta^{(3)}(\vec{r} - \vec{w}(t)) $$ and the current distribution will be $$ \vec{J}(\vec{r},t) = \dot{\vec{w}} \rho(\vec{r},t) = \dot{\vec{w}} \left[ \vec{p} \cdot \vec{\nabla} \delta^{(3)}(\vec{r} - \vec{w}(t)) \right]. $$
Thus, the overall radiation field is $$ \vec{E}(\vec{r}, t) = (\vec{p} \cdot \vec{\nabla}) \vec{E}_1 \\ \sim \frac{\mathcal{R}}{4 \pi \epsilon_0} \left\{ \frac{ \vec{\mathcal{R}} \times [ ((\vec{p} \cdot \vec{\nabla})\vec{u} ) \times \vec{a}]}{(\vec{\mathcal{R}} \cdot \vec{u})^3} + \frac{ \vec{\mathcal{R}} \times [ \vec{u} \times ((\vec{p} \cdot \vec{\nabla}) \vec{a})]}{(\vec{\mathcal{R}} \cdot \vec{u})^3} - 3 \frac{\vec{\mathcal{R}} \times [ \vec{u} \times \vec{a}]}{(\vec{\mathcal{R}} \cdot \vec{u})^4} \vec{\mathcal{R}} \cdot [ (\vec{p} \cdot \vec{\nabla}) \vec{u} ] \right\} $$ The first term vanishes, since $(\vec{p} \cdot \vec{\nabla}) \vec{u}$ is parallel to $\vec{a}$; and the result is that $$ \vec{E}(\vec{r}, t) \sim - \frac{\vec{p} \cdot \vec{\mathcal{R}}}{4 \pi \epsilon_0 c} \left\{ c \frac{\vec{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{\jmath}]}{(c \mathcal{R})^3} + 3 c \frac{\vec{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{a}]}{(c \mathcal{R})^4} \vec{\mathcal{R}} \cdot \vec{a} \right\} $$$$ \vec{E}(\vec{r}, t) \sim - \frac{\vec{p} \cdot \vec{\mathcal{R}}}{4 \pi \epsilon_0 c} \left\{ c \frac{\vec{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{\jmath}]}{(c \mathcal{R})^3} + 3 c \frac{\vec{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{a}]}{(c \mathcal{R})^4} \vec{\mathcal{R}} \cdot \vec{a} \right\} \\ = - \frac{\vec{p} \cdot \hat{\mathcal{R}}}{4 \pi \epsilon_0 c^3 \mathcal{R}} \left\{ \hat{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{\jmath}] + \frac{3}{c} (\hat{\mathcal{R}} \cdot \vec{a}) \hat{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{a}] \right\} $$ It is easy to see that we can construct trajectories $\vec{w}(t)$ for which this quantity does not vanish.
Thankfully, we don't have to go through this all again with the magnetic field. We will have $$ \vec{B}(\vec{r},t) = (\vec{p} \cdot \vec{\nabla}) \vec{B}_1 (\vec{r},t) = (\vec{p} \cdot \vec{\nabla}) \left[ \frac{1}{c} \hat{\mathcal{R}} \times \vec{E}_1 \right] \sim \frac{1}{c} \hat{\mathcal{R}} \times \left[ (\vec{p} \cdot \vec{\nabla}) \vec{E}_1 \right] = \frac{1}{c} \hat{\mathcal{R}} \times \vec{E}(\vec{r},t). $$ In the third step, we have again used the fact that any derivatives acting on $\hat{\mathcal{R}}$ will lead to terms that do not contribute to the radiation fields. The Poynting vector will then be $$ \vec{S} \propto \vec{E} \times \vec{B} \propto \hat{\mathcal{R}} E^2, $$ since $\vec{E}$ is perpendicular to $\hat{\mathcal{R}}$ and $\vec{B}$ is perpendicular to both of these. Thus, for a general motion of the dipole, there will be a finite amount of power radiated to infinity.