About mass defect Here's how my book explains mass defect:

Particles inside the nucleus interact with each other - they feel attraction. The potential energy $U$ of such attraction is negative, because in absence of these forces we consider the potential energy to be zero. So we can write the total energy as:
  $$E=E_{rest}+U$$
  Dividing $E$ by $c^2$ we obtain the mass, and because $U<0$ the mass of the nucleus is less than the sum of individual nucleons.

Now, I have problem with the $U$ term. We know that we can choose the zero level for PE arbitrarily. Thus, $E$ can't be defined well (up to constant). However, real measurements "obey" the standard convention of zero PE at infinity. So how can I solve the contradiction? (Obviously, I'm wrong, but I fail to understand why).
This question leads me to a more general question regarding the $E=mc^2$ relation. It follows that $m$ has no certain value when we're dealing with potential energies. Only the change in mass matters, because only the change in potential energy has physical meaning (and can be defined precisely). But mass is a quantity which we measure everyday very precisely, and there's no ambiguity in its value, despite the fact that the systems we measure include quite often some potential energy.
 A: It isn't possible to measure potential energy because it has a (global) gauge symmetry. It's like trying to measure the height of a mountain - this could be the height above sea level, the height relative to the deepest sea trench, the height relative to the centre of the earth and so on. Any measurement can only measure the change in potential energy, and of course a change can be positive or negative i.e. the potential energy can increase or decrease.
This is my preferred way to understand the mass defect: start with the separate particles of combined mass $M$ at large distance. As the particles come together they accelerate due to the attractive force between them, so at the moment all the particles meet they have a high kinetic energy, $T$. To get the particles to form a bound state we need to remove this kinetic energy $T$, and that means taking out a mass of $m = T/c^2$ - hence the mass defect.
The total kinetic energy $T$ is just the negative of the potential energy change $\Delta U$ as the particles are brought from infinity into the bound state. It makes no difference what value you assign to $U(\infty)$ because all that matters is the change $\Delta U$.
Response to user52153's comment:
Suppose we start with two particles that attract each other. This could be an electron and proton, or two nucleons, though for convenience I'll assume they have same mass $m_0$. Start with the particles stationary and far enough apart that any interaction is negligable. Then the total energy is just:
$$ E_0 = 2m_0c^2 $$
Now let the particles move together under their mutual attraction. The attractive force will accelerate them, so some time later they will have a kinetic energy $T$. Since we haven't put any energy in or taken any energy out, the total energy must be unchanged so there must be a (negative) potential energy $U$ such that:
$$ 2m_0c^2 + T + U = 2m_0c^2 = E_0 $$
In other words $T + U = 0$ or $T = -U$, and remember that $U$ is a negative number.
At this point there is no mass deficit because the total energy is unchanged. Now user52153 proposes we use some mechanism to take the kinetic energy out of the system. It doesn't matter exactly how we do this - we use the kinetic energy to do work $W$ external to our system of two particles and we continue doing this work until $W = T$ so at this point the particles have been brought to rest. Because we have taken work $W = T$ out of the system the total energy is now:
$$ E_1 = 2m_0c^2 + U $$
The mass deficit $\Delta m = (E_0 - E_1)/c^2$ so:
$$ \Delta_m = \frac{E_0 - E_1}{c^2} = \frac{(2m_0c^2) - (2m_0c^2 + U)}{c^2} = -\frac{U}{c^2} $$
and because $U$ is a negative number that means $\Delta m$ is a positive number i.e. the total mass of the system has decreased.
A: If you use relativity (which the use of $E = mc^2$ implies), we cannot choose the potential arbitrarily, because the relation between energy and mass makes absolute values of the energy measurable through the gravitational forces exerted by stored energy.
EDIT:
Since you asked, I will explain it in somewhat more detail: 
Let $V : \mathbb{R}^4 \rightarrow \mathbb{R}$ be a potential that is zero at spatial infinity, i.e $\lim_{\vec{x}\rightarrow\infty}V(x) = 0$. Then, the class of potentials equivalent (meaning they produce the same physics) to this without gravity is $\{V_k(x) | V_k(x) = V(x) + k, c\in \mathbb{R}\}$. With gravity, the energy-mass relation $E = mc^2$ tells us that only one of these potentials is realized, since mass is a measurable quantity. How do we tell which one? It's simple:
Let $V_k$ be the potential energy realized in nature. The energy of a particle at spatial infinty at rest must be $E_0 = m_0c^2$, where $m_0$ is the rest mass. Its energy is also $lim_{\vec{x}\rightarrow\infty}V_k(x) + m_0c^2 = k + m_0c^2$. Thus, $k = 0$. If I didn't make a glaring mistake somewhere, this shows that potential energy of a particle must vanish at spatial infinity in gravitating theories.
If you wonder why particles resting at spatial infinity must have $E_0$ as energy, it is because they are indistinguishable from free particles, and those come from a theory where there is no potential at all.
A: Tabulated values of nuclear mass defects do in fact include an arbitrary offset. Usually the standard is that one mole of carbon-12 weighs exactly twelve grams, so that a bare proton or neutron has a positive mass defect, while a tightly-bound nucleus like iron or nickel has a negative mass defect. In computing the $Q$-values of decays it is only the difference in the mass defect that matters.
Be warned that, in nuclei, there is a subtle difference between the mass defect and the binding energy. The rest mass of a (simple or composite) particle is always well-defined. If you compare the masses of a composite system and its simple constituents, you find that the bound composite system is less massive than the constituents in isolation: for example, a carbon-12 atom in its ground state is less massive than six protons, six neutrons, and six electrons. That mass difference is the binding energy, your $U$, and it doesn't suffer from the arbitrary-zero problem of potential energies because it is already a difference between two energies.
In tables of mass defects, like the one I linked above, the "defect" is the difference between the measured mass of a neutral atom with $Z+N=A$ nucleons and the naïve mass of $A$ atomic mass units. The mass defect would be the same as the binding energy if the neutron and the proton+electron both had a rest mass of one amu, but they don't: the proton+electron together are about 7 MeV heavier than 1 amu, and the neutron is about 8 MeV heavier than 1 amu. However, since nuclear reactions conserve baryon number $A$ and electric charge, you can always compute the energy available in a reaction by comparing the mass defects of the initial and final states. This is actually more robust than comparing binding energies, since it automatically accounts for the difference in the rest masses of the neutron and proton.
A: Let us take Newton's law for a particle in special relativity
$$F^\mu = \frac{d p^\mu}{d \tau} = \frac{d m_0}{d\tau} u^\mu + m_0\frac{du^\mu}{d\tau}$$
where $F^\mu = - \partial^\mu V$, $p^\mu = m_0 u^\mu = m_0 \gamma (c,\vec{v})$ and in our lab frame $V(x,t) = V(x)$. Taking $m=m_0 \gamma$ and a transformation from the proper time of the particle to our lab time $t$ ($\frac{d}{d\tau} = \gamma \frac{d}{dt}$) we thus get a statement of mass (energy) conservation
$$F^0 =0 = c \gamma \frac{d m}{dt}$$
Now we project the whole Newton's law in the direction of the four-velocity $u^\nu$ (sum with $u_\mu$). We use that $u^\mu u_\mu =-c^2 \,$ ($-+++$ signature of the metric), $u^\mu a_\mu = 0$, transform into lab-frame time and get
$$ - \gamma \vec{v} \cdot \nabla V = -c^2 \gamma \frac{dm_0}{d t}$$
This can be simplified and rewritten as
$$\frac{d V(x(t))}{dt} = \frac{d (m_0 c^2)}{dt}$$
Or
$$\Delta V = \Delta m_0 c^2$$
Notice however that in this process we have always $\Delta m = 0$ due to $F^0 = 0$. There thus must be a particle (say a photon) emitted in a separate process to carry away all the momentum and $T = mc^2 - m_0 c^2$. 
Considering a particle at rest at infinity we can write for it's mass after being bound
$$m^{bound}c^2 = m_0^{bound}c^2 + (m_0^{bound} - m^{bound})c^2  = m_0^{\infty}c^2 + \Delta V + \Delta T \; \; \; (*)$$
where we identify kinetic energy $\Delta T=(m_0^{bound} - m^{bound})c^2$ because it is the only term dependent on velocity, and the $\Delta$ stands for the fact the same term was zero before getting bound. Before emitting a photon we will have $\Delta T = -\Delta V$ so that $\Delta m=0$ applies.
Naturally we call $\Delta E \equiv \Delta T + \Delta V$. It is easy to generalize that
$$(mc^2)^{after} = (mc^2)^{before} + \Delta E,\, \to \Delta E = \Delta mc^2$$
So there is no need to invoke an absolute value of $E$ or $V$ at all, which means they can indeed be defined up to arbitrary constants. Thus, it is a more accurate statement to say that
$$E = mc^2 + C$$
And there is no $V$! How come? In your text book, by $E_{rest}$ they actually mean rest energy of the particle transported from the potential to infinity, i.e. $m_0^{\infty}c^2$. So by using equation $(*)$ and getting rid of kinetic energy $\Delta T = 0$, we get
$$E = mc^2 + C = m_0^{\infty}c^2 + \Delta V + C$$ 
So in your textbook $U(\infty)=C$, and the only justification of setting it to zero is to force the relation $E=mc^2$ to apply. On the other hand, mass does not have this kind of freedom because it's absolute value defines the local dynamical response of the particle. So you can repeat the whole argument of the textbook just by substituting $E\to mc^2$ and your problem is solved.

EDIT: This post has been completely revised two times since creation. First a flawed argument was removed and second a mathematical derivation of the stated facts was added. (Note that in the previous version of the post I have sloppily used $m_0$ as $m_0^{\infty}$.)
