Zero-level of combination of $1/r$ and $r^2$ potential I am solving a problem which involves a central big mass $M$ and around it a spherically symmetrically distributed mass of constant density $\rho$. 
The force on a mass a distance $r$ from the centre can be shown to be:
$$ F = \frac{-GMm}{r^2} - \frac{4\pi\rho Gmr}{3}. $$
Hence the potential:
$$ U = - \int F \cdot dr = Gm\left( \frac{2\pi\rho r^2}{3} - \frac{M}{r} + C \right)$$
where $C$ is an integration constant. My question is: what should $C$ be? For the $1/r$ type of potential it is customary to have $U(\infty) = 0$, whereas for the $r^2$ type we commonly have $U(0) = 0$. Is there any smart choice here? Maybe where the forces sum to zero?
Leaving it in the form above I have the feeling that the two terms have different zero points, is that okay?
 A: The choice of the zero point for the potential energy is entirely arbitrary. In practice we choose it in a way that makes out calculation simple.
In this case your test mass is going to oscillate around the centre of the mass distribution so it will start at rest at some distance $r_\text{max}$, fall through the centre and out to the same distance $r_\text{max}$ on the other side where it will come to rest again. I would set the potential to be zero at the distance $r_\text{max}$. This makes the total energy zero so as the particle falls in the sum of the potential and kinetic energy remains zero.
A: 
I am solving a problem which involves a central big mass $M$ and around it a spherically symmetrically distributed mass of constant density $\rho$. 
  The force on a mass a distance $r$ from the centre can be shown to be: $ F = \frac{-GMm}{r^2} - \frac{4\pi\rho Gmr}{3}. $

I do so dislike the way this is taught. If I had my way it wouldn't be. Yes it's Newtonian mechanics and it works well enough when you're dropping a brick. But IMHO it delivers the wrong message about mass, and creates an impression of action-at-a-distance forces that Newton was dead against. See his 1692 letter to Richard Bentley:
"That gravity should be innate inherent & {essential} to matter so that one body may act upon another at a distance through a vacuum without the mediation of any thing else by & through which their action or force {may} be conveyed from one to another is to me so great an absurdity that I believe no man who has in philosophical matters any competent faculty of thinking can ever fall into it".
I would recommend that you go back to a simpler situation where you have only your central mass M and your smaller mass m. Get that clear, then address the spherical shell, then combine them. 

Hence the potential: $ U = - \int F \cdot dr = Gm( \frac{2\pi\rho}{3} r - \frac{M}{r} + C )$ where $C$ is an integration constant. My question is: what should $C$ be? For the 1/r type of potential it is customary to have U($\infty$) = 0, whereas for the r^2 type we commonly have U(0) = 0. Is there any smart choice here? Maybe where the forces sum to zero?

I don't like this either. The gravitational potential is what it is, and the force is only there when there's a gradient in potential. So saying potential is the integral of force is getting things back to front. If you were midway between two stars, you'd be subject to gravitational time dilation that was related to the "depth" of potential, but the would be no force. Ditto if you were midway between two more massive stars. The lack of force tells you nothing about potential.  

Leaving it in the form above I have the feeling that the two terms have different zero points... Is that okay?

I don't think so. IMHO it's isn't even OK to set potential to zero at infinity, and then say potential energy is negative. I know that this is the convention, see the gravitational potential Wikipedia article. But there are no things that consist of negative energy. Binding energy is said to be negative, but there's nothing there that consists of negative energy. There is less mass-energy, that's all, see the Wikipedia binding energy article which talks about the mass deficit: "For example, if two objects are attracting each other in space through their gravitational field, the attraction force accelerates the objects, increasing their velocity, which converts their potential energy (gravity) into kinetic energy". It also says "This missing mass may be lost during the process of binding as energy in the form of heat or light, with the removed energy corresponding to the removed mass through Einstein's equation E = mc²".  
You have missing mass, because as the body falls, potential energy in the guise of its mass-energy is converted into kinetic energy. If this didn't happen, your body wouldn't fall down. It affects the smaller mass more than the bigger mass, because the bigger mass has no discernible motion. So your m isn't constant. So both the expressions in your question are "Newtonian approximations". In other words, they're wrong. So as to what C should be, I find it hard to say.    
