Your formulas are essentially correct.
The current density and charge density
$$\begin{align}
\vec{J}&=I_0 \cos(\omega t)\delta(x)\delta(y)\delta(z)\hat{z} \\
\rho&=-\frac{I_0}{\omega}\sin(\omega t)\delta(x)\delta(y)\frac{\partial\delta(z)}{\partial z}
\end{align}$$
together satisfy charge conservation
$\vec{\nabla}\vec{J}=-\frac{\partial\rho}{\partial t}$.
In the $\vec{J}$ and $\rho$ above you may well use the $\delta$ representation
$$\delta(z)=\lim_{a\to 0}\frac{1}{|a|\sqrt{\pi}}e^{-z^2/a^2}$$
and its derivative
$$\frac{\partial\delta(z)}{\partial z}=
  \lim_{a\to 0}\frac{-2z}{|a|a^2\sqrt{\pi}}e^{-z^2/a^2}$$

The fact that $\rho$ near the center becomes infinite for $a\to 0$ was to
be expected, because an idealized "point-like" dipole is made up by two
opposite infinitely big charges separated by an infinitesimal small distance.
In your case you have two charges $\pm\frac{I_0}{\omega a}\sin(\omega t)$
located at $(x,y,z)=(0,0\pm a)$ (I have deliberately neglected any
factors of $2$ or $\sqrt 2$),
thus giving a dipole moment of $\frac{I_0}{\omega}\sin(\omega t)$.

You can also verify that the total charge
$Q=\iiint\rho\ d^3r$ is zero as it should be.