Summary: A dipole moving through space radiates. Specifically, the power radiated depends both on the dipole's acceleration and its jerk.
Finding the potentials:
Consider an idealized dipole $\vec{p}$ moving along a trajectory $\vec{w}(t)$. We assume that this dipole has a constant magnitude and direction in an inertial frame. This is somewhat artificial, particularly for motions along the direction of $\vec{p}$ (one would expect the dipole moment to Lorentz-contract); but the calculations below are complicated enough as it is, and so long as the speed of the dipole is non-relativistic, this assumptions should be valid.
The charge distribution for this dipole will be
$$
\rho(\vec{r},t) = \vec{p} \cdot \vec{\nabla} \delta^{(3)}(\vec{r} - \vec{w}(t))
$$
and the current distribution will be
$$
\vec{J}(\vec{r},t) = \dot{\vec{w}} \rho(\vec{r},t) = \dot{\vec{w}} \left[ \vec{p} \cdot \vec{\nabla} \delta^{(3)}(\vec{r} - \vec{w}(t)) \right].
$$
To find the potentials, we use the retarded Green's functions for the wave operator. We have
$$
V(\vec{r}, t) = - \frac{1}{\epsilon_0} \iiiint G(\vec{r} - \vec{r}',t - t') \rho(\vec{r}',t') \, d\tau' dt'
$$
$$
\vec{A}(\vec{r}, t) = - \mu_0 \iiiint G(\vec{r} - \vec{r}',t - t') \vec{J}(\vec{r}',t') \, d\tau' dt',
$$
where
$$G(\vec{r} - \vec{r}',t - t') = - \frac{c}{4\pi} \begin{cases} \delta(\mathcal{R} - ct)/\mathcal{R} & t > t' \\
0 & t < t' \end{cases}
$$
with $\vec{\mathcal{R}} \equiv \vec{r} - \vec{r}'$.
Let's consider the integral for $V$ first. We have
$$
V(\vec{r}, t) = \frac{c}{4 \pi \epsilon_0} \iiiint \frac{\delta( \mathcal{R} - (t - t'))}{\mathcal{R}} \vec{p} \cdot \vec{\nabla}' \delta^{(3)}(\vec{r}' - \vec{w}(t')) \, d\tau' dt'
$$
where $\vec{\nabla}'$ denotes the gradient with respect to $\vec{r}'$.
Integrating by parts, the boundary term vanishes, and this becomes
$$
V(\vec{r}, t) = - \frac{c}{4 \pi \epsilon_0} \iiiint \vec{p} \cdot \vec{\nabla}' \left[ \frac{\delta( \mathcal{R} - (t - t'))}{\mathcal{R}} \right] \delta^{(3)}(\vec{r}' - \vec{w}(t')) \, d\tau' dt'
$$
The term being acted on by the gradient only depends on $\vec{r}'$ through $\mathcal{R}$, and for any such function, $\vec{\nabla} f(\mathcal{R}) = - \vec{\nabla}' f(\mathcal{R})$ (where $\vec{\nabla}$ is the "normal" gradient.) Thus,
$$
V(\vec{r}, t) = \frac{c}{4 \pi \epsilon_0} \iiiint \vec{p} \cdot \vec{\nabla} \left[ \frac{\delta( \mathcal{R} - (t - t'))}{\mathcal{R}} \right] \delta^{(3)}(\vec{r}' - \vec{w}(t')) \, d\tau' dt'
$$
Since the final delta function is independent of $\vec{r}$, we can pull the gradient operator out of the integral to obtain
$$
V(\vec{r}, t) = \vec{p} \cdot \vec{\nabla} \left[ \frac{c}{4 \pi \epsilon_0} \iiiint \frac{\delta( \mathcal{R} - (t - t'))}{\mathcal{R}} \delta^{(3)}(\vec{r}' - \vec{w}(t')) \, d\tau' dt'\right]
$$
But the quantity in square brackets is precisely what we would get for a point charge (with unit magnitude) moving along the trajectory $\vec{w}(t)$; in other words, this is the standard Liénard-Wiechert potential! If we denote the Liénard-Wiechert potential for such a point charge as $V_1 (\vec{r},t)$, we can conclude that
$$
V(\vec{r}, t) = \vec{p} \cdot \vec{\nabla} \left[ V_1 (\vec{r},t) \right].
$$
A similar line of argument involving the vector potential results in a very similar-looking result:
$$
\vec{A}(\vec{r}, t) = \vec{p} \cdot \vec{\nabla} \left[ \vec{A}_1 (\vec{r},t) \right] = \vec{p} \cdot \vec{\nabla} \left[ \frac{\dot{\vec{w}}}{c^2} V_1 (\vec{r},t) \right].
$$
Note that $\dot{\vec{w}}$ (as well as all the other distances and positions involved in the Liénard-Wiechert potentials) are evaluated at the retarded time.
Finding the radiation fields:
To obtain the electric and magnetic fields, we would now want to take the time derivatives, gradients, and curls of these expressions. But since the potentials of an accelerated dipole are related to those of an accelerated unit point charge by taking the directional derivative $\vec{p} \cdot \vec{\nabla}$, and since this operator commutes with all spatial and time derivatives, it follows that the electric and magnetic fields of an accelerated dipole are similarly related to those of an accelerated unit point charge:
$$
\vec{E}(\vec{r},t) = (\vec{p} \cdot \vec{\nabla}) \vec{E}_1(\vec{r},t) \qquad \vec{B}(\vec{r},t) = (\vec{p} \cdot \vec{\nabla}) \vec{B}_1(\vec{r},t)
$$
For a unit point charge in arbitrary motion, the piece of the electric field responsible for radiation is the acceleration field:
$$
\vec{E}_1 \sim \frac{\mathcal{R}}{4 \pi \epsilon_0} \frac{ \vec{\mathcal{R}} \times (\vec{u} \times \vec{a})}{(\vec{\mathcal{R}} \cdot \vec{u})^3},
$$
where $\vec{u} = c \hat{\mathcal{R}} - \vec{v}$, $\vec{v} = \dot{\vec{w}}$, $\vec{a} = \ddot{\vec{w}}$, and all quantities are evaluated at the retarded time. Note that this quantity scales as $\mathcal{R}^{-1}$ at large distances; we use the symbol $\sim$ to denote "equality up to highest order in $\mathcal{R}$."
We now want to take the directional derivative of this quantity. This quantity depends on position in two ways: first, via the explicit dependence on $\vec{\mathcal{R}}$; and second, via the implicit dependence of $\vec{w}$, $\vec{v}$ and $\vec{a}$ on the retarded time $t_r$. (Note that $\vec{\mathcal{R}} = \vec{r} - \vec{w}(t_r)$ also depends implicitly on $t_r$.) Taking this derivative in the most general case is left (out of self-preservation) to the reader; instead, I will focus on the case where the charge is instantaneously at rest at time $t_r$ (cf. the usual simplification made in the calculation of the Larmor formula.) In such a case, we have
$$
(\vec{p} \cdot \vec{\nabla}) \vec{\mathcal{R}} = (\vec{p} \cdot \vec{\nabla}) (\vec{r} - \vec{w}(t_r)) = \vec{p} - \dot{\vec{w}} (\vec{p} \cdot \vec{\nabla} t_r) = \vec{p}.
$$
This means that any time the operator $\vec{p} \cdot \vec{\nabla}$ acts on a function of $\vec{\mathcal{R}}$, the resulting expression will scale by one power of $\mathcal{R}$ less than the original function. Since the radiation fields are those which scale as $\mathcal{R}^{-1}$, and $\vec{E}_1$ already scales as $\mathcal{R}^{-1}$, we can effectively treat $\vec{\mathcal{R}}$ as constant when taking the derivative of $\vec{E}_1$: all the terms arising from $\vec{p} \cdot \vec{\nabla}$ acting on $\vec{\mathcal{R}}$ will fall off faster than $\mathcal{R}^{-1}$.
We will still need to take the directional derivatives of $\vec{u}$ and $\vec{a}$, though. The first one works out to be
$$
(\vec{p} \cdot \vec{\nabla}) \vec{u} = (\vec{p} \cdot \vec{\nabla}) (c \hat{\mathcal{R}} - \vec{v}(t_r)) \sim - (\vec{p} \cdot \vec{\nabla})\vec{v}(t_r) = - \dot{\vec{v}} (\vec{p} \cdot \vec{\nabla} t_r) = \vec{a} \left(\frac{\vec{p} \cdot \hat{\mathcal{R}}}{c} \right),
$$
where we have used the fact that for a charge at rest at time $t_r$, $\vec{\nabla} t_r = - \hat{\mathcal{R}}/c$. Similarly, the second one becomes
$$
(\vec{p} \cdot \vec{\nabla}) \vec{a} = - \vec{\jmath} \left(\frac{\vec{p} \cdot \hat{\mathcal{R}}}{c} \right),
$$
where $\vec{\jmath}$ is the jerk of the dipole. Finally, if the dipole is at rest at time $t_r$, then $\vec{u} = c \hat{\mathcal{R}}$.
Thus, the overall radiation field is
$$
\vec{E}(\vec{r}, t) = (\vec{p} \cdot \vec{\nabla}) \vec{E}_1 \\
\sim \frac{\mathcal{R}}{4 \pi \epsilon_0} \left\{ \frac{ \vec{\mathcal{R}} \times [ ((\vec{p} \cdot \vec{\nabla})\vec{u} ) \times \vec{a}]}{(\vec{\mathcal{R}} \cdot \vec{u})^3}
+ \frac{ \vec{\mathcal{R}} \times [ \vec{u} \times ((\vec{p} \cdot \vec{\nabla}) \vec{a})]}{(\vec{\mathcal{R}} \cdot \vec{u})^3} - 3 \frac{\vec{\mathcal{R}} \times [ \vec{u} \times \vec{a}]}{(\vec{\mathcal{R}} \cdot \vec{u})^4} \vec{\mathcal{R}} \cdot [ (\vec{p} \cdot \vec{\nabla}) \vec{u} ] \right\}
$$
The first term vanishes, since $(\vec{p} \cdot \vec{\nabla}) \vec{u}$ is parallel to $\vec{a}$; and the result is that
$$
\vec{E}(\vec{r}, t) \sim - \frac{\vec{p} \cdot \vec{\mathcal{R}}}{4 \pi \epsilon_0 c} \left\{ c \frac{\vec{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{\jmath}]}{(c \mathcal{R})^3} + 3 c \frac{\vec{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{a}]}{(c \mathcal{R})^4} \vec{\mathcal{R}} \cdot \vec{a} \right\} \\
= - \frac{\vec{p} \cdot \hat{\mathcal{R}}}{4 \pi \epsilon_0 c^3 \mathcal{R}} \left\{ \hat{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{\jmath}] + \frac{3}{c} (\hat{\mathcal{R}} \cdot \vec{a}) \hat{\mathcal{R}} \times [ \hat{\mathcal{R}} \times \vec{a}] \right\}
$$
It is easy to see that we can construct trajectories $\vec{w}(t)$ for which this quantity does not vanish.
Thankfully, we don't have to go through this all again with the magnetic field. We will have
$$
\vec{B}(\vec{r},t) = (\vec{p} \cdot \vec{\nabla}) \vec{B}_1 (\vec{r},t) = (\vec{p} \cdot \vec{\nabla}) \left[ \frac{1}{c} \hat{\mathcal{R}} \times \vec{E}_1 \right] \sim \frac{1}{c} \hat{\mathcal{R}} \times \left[ (\vec{p} \cdot \vec{\nabla}) \vec{E}_1 \right] = \frac{1}{c} \hat{\mathcal{R}} \times \vec{E}(\vec{r},t).
$$
In the third step, we have again used the fact that any derivatives acting on $\hat{\mathcal{R}}$ will lead to terms that do not contribute to the radiation fields. The Poynting vector will then be
$$
\vec{S} \propto \vec{E} \times \vec{B} \propto \hat{\mathcal{R}} E^2,
$$
since $\vec{E}$ is perpendicular to $\hat{\mathcal{R}}$ and $\vec{B}$ is perpendicular to both of these. Thus, for a general motion of the dipole, there will be a finite amount of power radiated to infinity.