Your formulas are essentially correct. The current density and charge density $$\begin{align} \vec{J}&=I_0 \cos(\omega t)\delta(x)\delta(y)\delta(z)\hat{z} \\ \rho&=-\frac{I_0}{\omega}\sin(\omega t)\delta(x)\delta(y)\frac{\partial\delta(z)}{\partial z} \end{align}$$ together satisfy charge conservation $\vec{\nabla}\vec{J}=-\frac{\partial\rho}{\partial t}$. In the $\vec{J}$ and $\rho$ above you may well use the $\delta$ representation $$\delta(z)=\lim_{a\to 0}\frac{1}{|a|\sqrt{\pi}}e^{-z^2/a^2}$$ and its derivative $$\frac{\partial\delta(z)}{\partial z}= \lim_{a\to 0}\frac{-2z}{|a|a^2\sqrt{\pi}}e^{-z^2/a^2}$$
The fact that $\rho$ near the center becomes infinite for $a\to 0$ was to be expected, because an idealized "point-like" dipole is made up by two opposite infinitely big charges separated by an infinitesimal small distance. In your case you have two charges $\mp\frac{I_0}{\omega a}\sin(\omega t)$$\pm\frac{I_0}{\omega a}\sin(\omega t)$ located at $(x,y,z)=(0,0\pm a)$ (I have deliberately neglected any factors of $2$ or $\sqrt 2$), thus giving a dipole moment of $-\frac{I_0}{\omega}\sin(\omega t)$$\frac{I_0}{\omega}\sin(\omega t)$.
You can also verify that the total charge $Q=\iiint\rho\ d^3r$ is zero as it should be.