Why is the neutron magnetic moment negative? I understand that the magnetic moment is due to the quarks, but specifically why is it negative? Is it due to the two down quarks or something?
 A: Because the "large" pieces in the baryon octet wavefunctions usually win.
Bég,Lee,&Pais, PRL 13 (1964) 514, a historically significant demonstration  that the naive constituent quark model with a symmetric wavefunction was there to stay, one way or another-- and thus led to the inference of color to antisymmetrize it consistently with Fermi statistics (but that is another story...).
Pick your spin axis in the z direction. Take your proton and the neutron as bland symmetrized assemblies of just three inert constituent quarks, and their magnetic moments as the naive sum of the magnetic moments of each of their quark constituents. So, you only simply read off 
$$
\mu_p=\langle p\uparrow|\tfrac{e}{2m_q}\sum_i Q_i \sigma_i^3   |p\uparrow \rangle ,
$$
and likewise for the neutron, where Q indicates the (fractional) value of the quark's charge in terms of the elementary (proton) charge, and $m_q$ its common mass, but we'll only consider baryon moment ratios, so it washes off. 
But since the sum over the three quark constituents is symmetric among quarks, we need not write the full messy wavefunctions of the baryons: we'll simply imply symmetrization (and color antisymmetrization, today).
The calculation is then trivial, running on simplified collapsed wavefunctions,
$$
|p\uparrow \rangle \sim \frac{1}{\sqrt 6} (2 u\uparrow u\uparrow d\downarrow  -u\uparrow d\uparrow u\downarrow - d\uparrow u\uparrow u\downarrow ), \\
|n\uparrow \rangle \sim \frac{-1}{\sqrt 6} (2 d\uparrow d\uparrow u\downarrow  -d\uparrow u\uparrow d\downarrow - u\uparrow d\uparrow d\downarrow ).
$$
It is then evident by inspection that 
$$
\frac{\mu_p}{\mu_n}= \frac{\langle p\uparrow| \sum_i 3Q_i \sigma_i^3   |p\uparrow \rangle }{\langle n\uparrow| \sum_i 3Q_i \sigma_i^3   |n\uparrow \rangle}\\  =\frac{4(2+2+1)+(2-1-2)+(-1+2-2)}{4(-1-1-2)+(-1+2+1)+(2-1+1)}\\ =\frac{18}{-12}=-3/2.
$$
This is the classic "cannot be a coincidence" moment. The experimental value is -1.45989806(34), from Wikipedia.  (You can all but hear the palms slapping on foreheads all over the planet, at the time...).
