Why do we raise and lower indices of tensors of various groups with the invariants of that group? If $T_{ij}$ is tensor that transforms under $SO(N)$ then apparently (according to what I have been told) it does not matter whether we put the indices up or down.
If we instead have a tensor that transforms under $SU(N)$ then it matters.
I understand that it has to do with the invariants of the groups, but I haven't been able to understand it mathematically. Any help is much appreciated.
 A: Let's break this down in parts.  This is almost certainly overkill, but hopefully it will be clear, and you can skip the bits you don't need.
A rank-(0,2) tensor $\mathbf{T}$ is a multilinear map which takes two vectors and returns a real (or complex) number.  If we write it out component wise, we write its indices down.
If our vector space is equipped with a symmetric, nondegenerate rank-(0,2) metric tensor $\mathbf{g}$, then we can define an inner product between two vectors $\mathbf{X}$ and $\mathbf{Y}$ as follows:
$$\langle \mathbf{X},\mathbf{Y}\rangle = \mathbf{g}(\mathbf{X,Y})=\mathbf{g}(X^\alpha e_\alpha ,Y^\beta e_\beta) =X^\alpha Y^\beta \mathbf{g}(e_\alpha,e_\beta) \equiv X^\alpha Y^\beta g_{\alpha \beta}$$
(that's how we extract the components of a tensor - by letting it operate on the basis of the vector space)
Given one vector space, we can define a dual vector space of the same dimension whose elements are called dual vectors, or covectors.  If the basis of the vector space is given by $\{e_\alpha \}$, then the canonical basis of the covector space is given by $\{\epsilon^\alpha\}$ and is defined by the fact that the covector bases act on the vector basis like this:
$$ \epsilon^\alpha (e_{\beta}) =\delta^\alpha_\beta \equiv \cases{1 & $\alpha = \beta$ \\ 0 & $\alpha \neq \beta$}$$
So a covector $\mathbf{\omega}$ can act on a vector $\mathbf{X}$ like this:
$$ \mathbf{\omega(X)} = \omega_\alpha \epsilon^\alpha\left(X^\beta e_\beta\right)= \omega_\alpha X^\beta \epsilon^\alpha(e_\beta) = \omega_\alpha X^\beta \delta^\alpha_\beta = \omega_\alpha X^\alpha$$

Using the metric, we can define an isomorphism between a vector space and the covector space, which maps a vector $\mathbf{X}$ to a covector $\mathbf{\tilde X}$ which acts on a vector $\mathbf{Y}$ in the following way:
$$ \mathbf{\tilde X(Y)} = \mathbf{g}\left(\mathbf{X,Y}\right) $$
or in component form,
$$ \mathbf{\tilde X(Y)} = \tilde X_\alpha \epsilon^\alpha (Y^\beta e_\beta) = \tilde X_\alpha Y^\alpha = g_{\alpha \beta} X^\beta Y^\alpha$$
so
$$ \tilde X_\alpha = g_{\alpha \beta} X^\beta $$
Obviously $\mathbf{\tilde X}$ and $\mathbf{X}$ are completely different objects and inhabit completely different spaces, but they are in one-to-one correspondence and so people tend to loosely think of them as different "versions" of the same thing.  It is in this sense that we talk about "raising" and "lowering" the indices.

The aforementioned isomorphism can also be applied to tensors.  We can first define an "inverse metric" $\mathbf{\tilde g}$ which is a symmetric rank-(2,0) tensor, and which acts on dual vectors while respecting the isomorphism in the following way:
$$ \mathbf{\tilde g(\tilde X,\tilde Y)} = \mathbf{g(X,Y)}$$
It's not hard to see how this is defined component-wise:
$$ \mathbf{\tilde g}(\tilde X_\alpha \epsilon^\alpha,\tilde Y_\beta \epsilon^\beta) = \tilde X_\alpha \tilde Y_\beta \tilde g^{\alpha \beta} = g_{\alpha \rho} X^\rho g_{\beta \sigma} Y^\sigma \tilde g^{\alpha \beta} = X^\rho Y^\sigma g_{\rho \sigma}$$
so
$$ \tilde g^{\alpha \beta} g_{\alpha \rho} g_{\beta \sigma}  = g_{\rho \sigma}$$
which implies that
$$ \tilde g^{\alpha \beta} g_{\alpha \rho} = \delta^{\beta}_\rho$$
From there, we can define a general isomorphism between tensors like this:
$$ \mathbf{\tilde T(\tilde X,\tilde Y)}= \mathbf{\tilde T}(\tilde X_\alpha \epsilon^\alpha,\tilde Y_\beta \epsilon^\beta)=\tilde X_\alpha \tilde Y_\beta \tilde T^{\alpha \beta}$$
which we demand to be equal to
$$ \mathbf{T(X,Y)} = \mathbf{T}(X^\alpha e_\alpha,Y^\beta e_\beta) = X^\alpha Y^\beta T_{\alpha \beta}$$
comparing the two, it's clear that
$$ \tilde T^{\alpha \beta} = \tilde g^{\alpha \rho} \tilde g^{\beta \sigma} T_{\rho  \sigma}$$
Again, $\mathbf{\tilde T}$ and $\mathbf{T}$ are different objects which inhabit different spaces, but they are in one-to-one correspondence and their components are related as above.

Okay. So now we ask what it means for a tensor to transform according to a particular transformation rule.  Let $\mathbf{R}$ be a linear endomorphism on the vector space which maps a vector $\mathbf{X}$ to another vector $\mathbf{X'=R(X)}$:
$$\mathbf{X'}=X'^\alpha e_\alpha = \mathbf{R}(X^\beta e_\beta)= \left(R^\alpha_{\ \ \beta} X^\beta\right) e_\alpha$$
and so
$$ X'^\alpha = R^\alpha_{\ \ \beta} X^\beta$$
An orthogonal transformation is one which respects the inner product:
$$ \mathbf{g(X',Y')}=\mathbf{g}(R^\alpha_{\ \ \rho} X^\rho e_\alpha,R^\beta_{\ \ \sigma} Y^\sigma e_\beta)= R^\alpha_{\ \ \rho} R^\beta_{\ \ \sigma} X^\rho Y^\sigma \mathbf{g}(e_\alpha,e_\beta)$$
$$ = R^\alpha_{\ \ \rho} R^\beta_{\ \ \sigma} X^\rho Y^\sigma g_{\alpha \beta}$$
$$ = \mathbf{g(X,Y)}= g_{\rho \sigma} X^\rho Y^\sigma$$
so
$$ R^\alpha_{\ \ \rho} R^\beta_{\ \ \sigma} g_{\alpha \beta} = g_{\rho \sigma}$$
If we write this in matrix form, then $R^\alpha_{\ \ \rho} = (R^T)^{\ \ \alpha}_\rho$ and this becomes
$$ \mathbf{R^T \circ g \circ R = g}$$
Equivalently, we can use the apply the inverse metric, which yields
$$ \mathbf{\tilde g \circ R^T \circ g \circ R = \tilde g \circ g} = \mathbb{1}$$
which implies that
$$ \mathbf{\tilde g \circ R^T \circ g} = \mathbf{R^{-1}} \iff \mathbf{R^T} = \mathbf{ g \circ R^{-1} \circ \tilde g} \iff \mathbf{R}=\mathbf{\tilde g \circ \left(R^{-1}\right)^T \circ g}$$
and therefore that
$$ \mathbf{R^{-1}= R^T}$$
where we have used that $\mathbf{g}$ is symmetric.

What happens if we use a unitary transformation $\mathbf{U}$ rather than an orthogonal one?  Well, not much changes, but now the transformation respects the sesquilinear inner product
$$ \langle\mathbf{X,Y}\rangle = \mathbf{g(\bar{X},Y)} $$
where $\mathbf{\bar X} = \bar X^\alpha e_\alpha$ and $\bar X^\alpha$ is the complex conjugate of $X^\alpha$.  Working through the same process, we find that 
$$ \bar U^\alpha_{\ \ \rho} U^\beta_{\ \ \sigma} g_{\alpha \beta} = g_{\rho \sigma}$$
or, in matrix form (where $\dagger$ denotes the conjugate transpose),
$$ \mathbf{U^\dagger \circ g \circ U} = \mathbf{g}$$
and so
$$ \mathbf{\tilde g \circ U^\dagger \circ g} = \mathbf{U^{-1}} \iff \mathbf{U^\dagger} = \mathbf{g \circ U^{-1} \circ \tilde g} \iff \mathbf{U} = \mathbf{\tilde g \circ \left(U^{-1}\right)^\dagger \circ g}$$
and therefore that
$$ \mathbf{U^{-1} = U^\dagger}$$
where we've used that $\mathbf{g}$ is symmetric and that its components are real.

We can also ask how a general tensor $\mathbf{T}\rightarrow \mathbf{T'}$ must transform under a group action $\mathbf{Q}$ if we are to leave $\mathbf{T(X,Y)}$ invariant.  We do basically the same thing as before (without demanding that the $\mathbf{T'=T}$), and we arrive at
$$ Q^\alpha_{\ \ \rho} Q^\beta_{\ \ \sigma} T'_{\alpha \beta} = T_{\rho \sigma}$$
Because the group action is invertible, we can write
$$ T'_{\alpha \beta} = \left(Q^{-1}\right)^\rho_{\ \ \alpha} \left(Q^{-1}\right)^\sigma_{ \ \ \beta}T_{\rho \sigma}$$
or, in matrix form,
$$ \mathbf{T'} = \left(\mathbf{Q^{-1}}\right)^T \circ \mathbf{T} \circ \mathbf{Q^{-1}}$$

Now to the final piece.  Given all of the machinery we have constructed, how do the dual tensors transform?  Well, 
$$\tilde T^{\alpha \beta} = \tilde g^{\alpha \rho} \tilde g^{\beta \sigma}T_{\rho \sigma}$$
so (under a linear transformation $\mathbf Q$)
$$\tilde T'^{\alpha \beta} = \tilde g^{\alpha \rho} \tilde g^{\beta \sigma} T'_{\rho \sigma} = \tilde g^{\alpha \rho} \tilde g^{\beta \sigma} \left(Q^{-1}\right)^\mu_{\ \ \rho} \left(Q^{-1}\right)^\nu_{ \ \ \sigma}T_{\mu \nu}$$
$$ =\tilde g^{\alpha \rho} \tilde g^{\beta \sigma} \left(Q^{-1}\right)^\mu_{\ \ \rho} \left(Q^{-1}\right)^\nu_{ \ \ \sigma}g_{\mu \eta} g_{\nu \tau} \tilde T_{\eta \tau} $$
With a little elbow grease, this can be put in matrix form:
$$ \mathbf{\tilde T'} = \left[\mathbf{\tilde g} \circ \left(\mathbf{Q^{-1}}\right)^T \circ \mathbf{g}\right] \circ \mathbf{\tilde T} \circ \left[\mathbf{g} \circ \mathbf{Q^{-1}} \circ \mathbf{\tilde g}\right]$$

A bit icky, but meh.  What happens if this transformation is orthogonal?  We can answer right away based on our hard work from before.  We found that
$$\mathbf{\tilde g} \circ \left(\mathbf{R^{-1}}\right)^T \circ \mathbf{g} = \mathbf{R} $$
and that
$$\mathbf{g} \circ \mathbf{R^{-1}} \circ \mathbf{\tilde g} = \mathbf{R^T}$$
so in matrix form,
$$\mathbf{\tilde T'} = \mathbf{R \circ \tilde T \circ R^T}$$
whereas
$$\mathbf{T'} = \mathbf{\left(R^{-1}\right)^T \circ T \circ R^{-1}}$$
because orthogonal transformations are such that $\mathbf{R^T=R^{-1}}$, the tensor $\mathbf{T}$ and its dual $\mathbf{\tilde T}$ transform precisely the same way.

On the other hand, what if the transformation is unitary?  The tensor $\mathbf{T}$ transforms just as before:
$$ \mathbf{T'} =\mathbf{U \circ T \circ U^T}$$
but now the inverse and the transpose are not the same thing, and so the duel tensor transforms like this
$$ \mathbf{\tilde T'} = \mathbf{\bar U \circ \tilde T \circ \bar U^T} $$
where  $\mathbf{\bar U}$ is the complex conjugate of $\mathbf{U}$.
