Does an accelerating proton also lose mass? A proton accelerated with electric field gives off E.M. radiation and therefore should lose mass. Larmor's formula gives us a value for the power emitted (varies as acceleration squared).  However, as the proton picks up speed, it also gains mass. Now, say I set up an immense electric field which provides an immense acceleration to the proton. In the initial moments of motion, even though its acceleration is extremely high, its velocity is low. In those moments, does the proton lose mass faster than it gains?
 A: There are two different senses of the word "mass" that will be useful to disentangle here.
First, we have rest mass, which is the mass of a particle as seen in a frame where the particle is at rest (not moving).  We might write this mass as $m_0$.  This never changes; it's just a property of the particle.  So when you ask "does the proton lose mass faster than it gains?", you might be asking if it loses rest mass faster than it gains.  No, because the rest mass is constant.  
Second, we have the mass-energy of the particle, which is the time component of the four-dimensional mass-energy-momentum vector.  The mass-energy just depends on the velocity of the particle relative to the observer.  In particular, if the particle's speed is $v$, then the mass-energy is given by
\begin{equation}
E = \frac{m_0\, c^2}{\sqrt{1-v^2/c^2}}~.
\end{equation}
In particular, when $v=0$, we have $E = m_0\, c^2$, which you'll recognize.  And the faster the particle is going, the closer $v$ is to $c$, and the larger $E$ becomes.  Note that the energy only depends on a couple of constants, and the speed $v$.  It doesn't matter where the energy is coming from, or where it might be going; it just depends on the speed at each instant.
So presumably you're asking if the particle loses mass-energy faster than it gains.  And the answer is: if it move faster, it is gaining mass-energy, no matter what the reason for it moving faster.  You say it's gaining speed, so it's gaining mass-energy.  Period.
You don't have to even think about the radiation, because that doesn't come into it directly.  It may be losing energy due to radiation, but the electric field you've provided is evidently providing more power than that.

If you bring in bound states (like an atomic nucleus or an electron around a nucleus), you can think of a third type of "mass", and that is the binding energy, which adds a negative amount to the rest mass of the bound state (but not to the constituent particles).  For example, if $m_{0,p}$ is the rest mass of a proton, $m_{0,n}$ is the rest mass of a neutron, and $m_{0,d}$ is the rest mass of their bound state (a deuteron), then we know that
\begin{equation}
m_{0,d} < m_{0,p} + m_{0,n}~.
\end{equation}
The "missing" mass is energy given off in the fusion of the proton and the neutron into the deuteron, which we count as binding energy, which adds a negative amount to the rest mass of the deuteron.  But the rest masses of the two constituents are still exactly what they were before they came together.  You can't really say that the rest mass of the deuteron has changed because it didn't exist previously.  So no rest mass is lost, though some mass-energy that the proton and neutron had before fusion was given off in the form of radiation.
I'll emphasize the point that, even in nuclear reactions, you can't really say that rest mass is lost, because you're talking about a combined particle that never existed before.
