Magnetic decay versus rotation braking of neutron stars A canonical pulsar can be described as a ball of mass $M \approx 1,44 \, M_{\odot}$ and radius $R \approx 10 \, \mathrm{km}$, rotating with a period of about $P \approx 5 \, \mathrm{ms}$.  It also have a typical magnetic field of around $B_{\text{pole}} \sim 10^{6} \, \mathrm{tesla} = 10^{10} \, \mathrm{gauss}$ (roughly).  The field can be approximated as a dipolar magnetic field.  Because of the emission of dipolar electromagnetic radiation, the pulsar loses some energy, thus reducing its angular velocity $\omega \equiv 2 \pi / P$ (and maybe its polar magnetic field) :
\begin{equation}\tag{1}
\frac{dE_{\text{rad}}}{dt} = -\, \frac{\mu_0 \, \mu^2 \, \omega^4}{6 \pi c^3} \, \sin^2 {\alpha},
\end{equation}
where $\mu$ is the magnetic moment of the star, and $\alpha$ is the tilt angle relative to the rotation axis.  The magnetic filed at the poles has this intensity :
\begin{equation}\tag{2}
B_{\text{pole}} = \frac{\mu_0 \, \mu}{2 \pi R^3},
\end{equation}
where $R$ is the radius (assumed to be a constant) of the star.  The rotation kinetic energy and the magnetic energy stored into the dipolar magnetic field (assuming that the internal field is uniform) can be added together :
\begin{equation}\tag{3}
K_{\text{rot}} + U_{\text{magn}} = \frac{1}{2} \, I \, \omega^2 + \frac{\mu_0 \, \mu^2}{4 \pi R^3},
\end{equation}
where $I \approx \frac{2}{5} \, M R^2$ is the moment of inertia of the star.
The time derivative of (3) should be equal to the power lost (1) :
\begin{equation}\tag{4}
\frac{dE}{dt} = I \, \omega \, \dot{\omega} + \frac{\mu_0 \, \mu \, \dot{\mu}}{2 \pi R^3} = -\, \frac{\mu_0 \, \mu^2 \, \omega^4}{6 \pi c^3} \, \sin^2 {\alpha}.
\end{equation}
Now the problem is the following.  It is usually assumed that the star will slow down by the electromagnetic emission, so $\dot{\omega} \ne 0$.  In all textbooks and lectures I have seen, the magnetic energy is not added in (3)-(4).  But yet it is known that the magnetic field intensity may also be evolving (i.e decaying) with time.  If I neglect the rotation frequency decreasing (i.e consider $\omega = \text{constant}$), I get this from (4) :
\begin{equation}\tag{5}
\dot{\mu} = -\, \Big( \frac{\omega^4 \, R^3}{3 c^3} \, \sin^2 {\alpha} \Big) \mu \equiv -\, \lambda \, \mu.
\end{equation}
This is a linear differential equation, of solution $\mu(t) = \mu(0) \, e^{-  \lambda \, t}$.  Thus, the polar magnetic field (2) is exponentially decaying with time.  For our canonical pulsar, this gives an half-life of about $22.5 \, \mathrm{s}$ for the field decays, if $\alpha = 90^{\circ}$.

How can we justify that this decay mode is negligible relative to the
  rotation decay ?  I.e how can we justify that $\dot{\mu} \approx 0$
  while $\dot{\omega} \ne 0$ ?


EDIT 1 : If we assume $\dot{\mu} = 0$, equation (4) gives another differential equation for $\omega(t)$. It gives this solution, which is not exponential :
\begin{equation}\tag{6}
P(t) = P_0 \sqrt{1 + \kappa \, t},
\end{equation}
where $\kappa$ is a complicated constant :
\begin{equation}\tag{7}
\kappa = \frac{4 \pi \mu_0 \, \mu^2 \sin^2 {\alpha}}{3 I c^3 \, P_0^2} = \frac{5 (2 \pi)^3}{3 \mu_0 \, c^3} \, \frac{B_{\text{pole}}^2 \, R^4}{M P_0^2} \sin^2 {\alpha}.
\end{equation}
According to (6), the constant $\tau = \kappa^{-1}$ is the caracteristic time of period evolution.  For our canonical pulsar defined at the beginning, with $\alpha = 90^{\circ}$ and $B_{\text{pole}} \approx 10^6 \, \mathrm{tesla}$, (7) gives this time lenght :
\begin{equation}
\tau \approx 5.87 \times 10^{14} \, \mathrm{s} \sim \text{18.6 millions years}.
\end{equation}
This is the model usually considered for a breaking pulsar.  But I think that the scenario (5) is also valid in its own right and should be considered as a possibility.
 A: The time dependent magnetic dipole moment is driven by the rotation of the star, so it is natural that rotation would provide the energy that goes into radiation. (You can check this by computing the torque.) Indeed, the energy in the magnetic field is much too small to power the emission.
The B-field, on the other hand, cannot just disappear (magnetic field lines in ideal MHD cannot just disappear). The B field decays by ohmic diffusion
$$
\frac{\partial B}{\partial t} = \frac{c^2}{4\pi\sigma}\nabla^2 B
$$
This gives a decay time 
$$
\tau = \frac{4\pi\sigma}{c^2}\frac{R^2}{\pi^2}
$$
Using $R=10$ km and $\sigma=6\cdot 10^{22}$ $s^{-1}$ (this is the conductivity cgs units) G. Baym, C. Pethick, and D. Pines, Nature, 224, 673, (1969) 
get $\tau=4\cdot 10^6$ yr, several million years.  
Postscript: A useful review is Petri, https://arxiv.org/abs/1608.04895v1 .
Among many other things the author provides estimates of the energies involved. For a mili-second pulsar the  gravitational energy is $2.6 \cdot 10^{46}$ J, the rotational energy is $3.2 \cdot 10^{45}$ J, the magnetic energy is $1.6 \cdot 10^{28}$ J, and the thermal energy is $3.4 \cdot 10^{40}$ J.
A: I may have found a proper solution/interpretation of my query.  When we consider a rotating magnetic dipole, it is demonstrated that the emission of radiation implies these two equations (these can be found in many textbooks) :
\begin{align}
\frac{dE}{dt} &= -\, \frac{\mu_0 \, \mu^2 \omega^4}{6 \pi c^3} \, \sin^2{\alpha}, \tag{1} \\[12pt]
\frac{d\vec{\boldsymbol{\mathrm{L}}}}{dt} &= -\, \frac{\mu_0 \, \mu^2 \omega^3}{6\pi c^3} \, \sin^2 \alpha \; \vec{\boldsymbol{\mathrm{n}}}, \tag{2}
\end{align}
where the unit vector $\vec{\boldsymbol{\mathrm{n}}}$ is the fixed rotation axis.  Now, it is very important to remember that $E$ and $\vec{\boldsymbol{\mathrm{L}}}$ are NOT the mechanical energy and mechanical angular momentum of the rotating dipole.  They are the total energy and angular momentum of the system dipole $+$ electromagnetic field :
\begin{align}
E &= E_{\text{mec}} + E_{\text{field}}, \tag{3} \\[12pt]
\vec{\boldsymbol{\mathrm{L}}} &= \vec{\boldsymbol{\mathrm{L}}}_{\text{mec}} + \vec{\boldsymbol{\mathrm{L}}}_{\text{field}}. \tag{4}
\end{align}
In our case :
\begin{align}
E &= \frac{1}{2} \, I \, \omega^2 + \frac{\mu_0 \, \mu^2}{4 \pi R^3}, \tag{5} \\[12pt]
\vec{\boldsymbol{\mathrm{L}}} &= I \, \omega \: \vec{\boldsymbol{\mathrm{n}}} + \vec{\boldsymbol{\mathrm{L}}}_{\text{field}}. \tag{6}
\end{align}
According to equation (2), the radiation emission produces a torque on the system, so the total angular momentum is varying with time.  This doesn't mean that the dipole must slow down !  We may very well have a dipole constantly rotating at a constant $\omega$ : $\dot{\omega} = 0$, but then the field angular momentum must decreases, which implies a time variation of the magnetic moment $\mu$ : $\dot{\mu} \ne 0$.  We could also have a constant $\mu$ : $\dot{\mu} = 0$, but then the dipole must slow down : $\dot{\omega} \ne 0$.  We could have a combination of both.
The "internal" magnetic moment $\mu$ may change for some internal reason, and the function $\mu(t)$ may be anything (depending on what is happening inside the dipole).  Assuming an exponential decay seems reasonable : $\mu(t) = \mu(0) \, e^{-\, \lambda \, t}$ (or $\dot{\mu} = -\, \lambda \, \mu$), where the constant $\lambda$ is arbitrary.  So equation (4) of my question is perfectly valid and may be solved analytically to give the angular velocity $\omega(t)$ (the solution is a bit complicated and I'm not showing it here).
Depending on the value of $\lambda$, we may have all sorts of behaviors for the dipole rotation.  It may even accelerates if $\mu$ decreases fast enough, as soon as the initial angular velocity isn't 0 : $\omega(0) \ne 0$ !
