Part of your confusion here probably comes from your notation. Usually we reserve the index $\mu$ for spacetime. The generators of $SU(N)$ are more commonly labelled with Latin indices $t^a$. See for example [here](http://www.lpthe.jussieu.fr/~salam/teaching/M2-2009/FeynmanRules.pdf).

We can split the amplitude into two parts, according to whether they concern color or kinematics. You are just interested in the color part. Each quark comes with a color "polarization", i.e. a normalized basis vector $c$ in the fundamental representation of $SU(N)$. The quark-gluon vertices just give color factors of $t^a$.

Now the cross section is given by the modulus squared of the amplitude so we have

$$ \sigma \propto {1\over N^2} \sum_{initial} \sum_{final} [c^\dagger _{\overline d} ~t^a ~c_d][c^\dagger _{t} ~t^a ~c_{\overline t}][c^\dagger _{\overline d} ~t^b ~c_d]^*[c^\dagger _{t} ~t^b ~c_{\overline t}]^* $$

Now using that the color polarizations are normalized we can reinterpret the sum as a double trace term

$$ \sigma \propto {1\over N^2}Tr(t^a t^b)Tr(t^a t^b) $$

Now we can use the color algebra relation

$$Tr(t^a t^b) = \frac{1}{2}\delta^{ab}$$

to conclude that 

$$\sigma \propto \frac{(N^2 - 1)}{N^2}$$

As for a reference, you could do worse than read Part III of [Peskin and Schroeder's book](http://www.amazon.co.uk/Introduction-Quantum-Theory-Frontiers-Physics/dp/0201503972). It's quite a big step up from Griffiths though. But if you want to understand what's really going on then you'll need to do QFT at some stage! 

**Where Does the Trace Come From?**

Observe that all the square bracket factors in my first expression for $\sigma$ are scalars, so we may commute them. We can thus write 

$$ \sigma \propto {1\over N^2} \sum_{initial} \sum_{final} [c^\dagger _{\overline d} ~t^a ~c_d][c^\dagger _{\overline d} ~t^b ~c_d]^*[c^\dagger _{t} ~t^b ~c_{\overline t}]^*[c^\dagger _{t} ~t^a ~c_{\overline t}] $$

Now expanding out the complex conjugation and using that the $t^a$ are Hermitian we find

$$ \sigma \propto {1\over N^2} \sum_{initial} \sum_{final} [c^\dagger _{\overline d} ~t^a ~c_d~c_d^\dagger ~t^b  c_{\overline d} ][~c^\dagger_{\overline t}~t^b  c _{t} c^\dagger _{t} ~t^a ~c_{\overline t}] $$

Now the color polarizations are normalized, so the central terms in each factor just give $1$. This leaves

$$ \sigma \propto {1\over N^2} \sum_{initial} \sum_{final} [c^\dagger _{\overline d} ~t^a ~t^b  c_{\overline d} ][~c^\dagger_{\overline t}~t^b ~t^a ~c_{\overline t}] $$

But now the sum over initial colors gives exactly the definition of $Tr(t^at^b)$. Similarly for the final colors.