Complex conjugate and transpose "with respect to a basis" In my quantum mechanics notes, my teacher described the complex conjugate and transpose of a linear operator X as "with respect to an orthogonal basis." What does it mean to take a transpose or complex conjugate "with respect to a basis?" I also remember him saying that if you take the complex conjugate or transpose with respect to another basis, the result would be different. Why would that be the case, and can you provide an example in which it is true?
 A: Consider a linear operator $A$ on a complex Hilbert space, which we can take as two-dimensional for simplicity, and start off by considering its hermitian conjugate $B=A^\dagger$, which we define as the complex conjugate of the transpose on a given orthonormal basis $\beta$. Thus, if the operator $A$ has a matrix representation
$$[A]_\beta=\begin{pmatrix} a & b \\ c & d\end{pmatrix}$$
on the basis $\beta$, we're defining $B$ to have the matrix representation
$$[B]_\beta=\begin{pmatrix} a^* & c^* \\ b^* & d^*\end{pmatrix}$$
on that same basis.
Now consider what happens if we look at the matrix representations of these two operators on some different basis $\gamma$: then we know that the transformed representation of $B$ is given by a matrix-equivalence relationship of the form
$$ [B]_\gamma = U^{-1} [B]_\beta U,$$
where $U$ is the basis-change matrix, and we also know that if $\gamma$ is also orthonormal then $U^{-1} = U^\dagger$, so that
$$ [B]_\gamma = U^\dagger [B]_\beta U = U^\dagger [A]_\beta^\dagger U.$$
since we know that $[B]_\beta = [A]_\beta^\dagger$, by definition. Now consider what happens when we take the matrix representation of $A$ on the basis $\gamma$, $[A]_\gamma = U^\dagger [A]_\beta U$, and we take its matrix hermitian conjugate, using the known behaviour of the matrix transpose for matrix products and the trivial behaviour of the complex conjugate:
$$
[A]_\gamma^\dagger = \left(U^\dagger [A]_\beta U\right)^\dagger = U^\dagger [A]_\beta^\dagger U = [B]_\gamma.
$$
See what happened? The definition of $\boldsymbol{B = A^\dagger}$, initially made in a given orthonormal basis, holds in any orthonormal basis we care to take, or in other words the relationship $B = A^\dagger$ is basis independent: it holds at an operator level, and we don't need to specify what basis should be taken for the conjugate-transpose to be calculated.
(Moreover: the fact that this operation, which we defined in terms of a matrix representation in a given basis, does not actually depend on the basis chosen, tells us that there is likely to be a nicer, basis-independent way to define the hermitian conjugate of any abstract linear operator. This is indeed the case $-$ it's the inner-product-based definition which uses the relationship $\langle A^\dagger u, v \rangle = \langle u, Av\rangle$ for arbitrary $u,v$.)

The point that your instructor is making about the matrix transpose and the complex conjugate is that this basis independence property is not true for either of them. 
To show that this is the case, a single counter-example is sufficient, so take 
$$[A]_\beta=\begin{pmatrix} 0 & i \\ -i & 0\end{pmatrix}$$
as the matrix representation of your operator, and 
$$
U = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 & i \\ 1 & -i\end{pmatrix}
$$
as your basis-change matrix. Then it is trivial to verify that
$$
[A]_\gamma = U^\dagger [A]_\beta U = \begin{pmatrix} -1 & 0 \\ 0 & 1\end{pmatrix}
$$
is the matrix representation of $A$ in the new basis, and while both the transpose and the complex conjugate leave the matrix representation $[A]_\gamma$ untouched in the new basis,
$$
[A]_\gamma ^T = [A]_\gamma ^* = [A]_\gamma,
$$
they both change its sign in the old matrix,
$$
[A]_\beta^T = [A]_\beta^* = -[A]_\beta,
$$
so neither operation can be performed without specifying what basis it should be done in.
(Also, this bears emphasizing: the example above is completely generic, and there is only a thin sliver of examples of operators $A$ and basis transformations $U$ which preserve the matrix transpose. Basically any example you take, so long as you're careful not to make it too simple, will show the behaviour above.)
A: In my mind, what it means is that, to take the complex conjugate or transpose of a linear operator, it first has to be expressed as a matrix. Expressing it as a matrix means that you have to choose a particular orthogonal basis.
It seems a bit confusing to me, because any matrix operator (apart from the identity matrix) expressed in (or translated to) a different basis is going to be different anyway, regardless of whether you then choose to take the transpose or complex conjugate.
So, the transpose/complex conjugate part to me seems irrelevant. Any matrix operator is defined with respect to a particular orthogonal basis, and if it is changed to another basis, the matrix (as in, the grid array of numbers) will be different.
A: Here's a simple example, to supplement the good answers that have already been posted.
Consider a $2$-dimensional Hilbert space $\mathcal{H}$ over the complex numbers. Let $|0\rangle$ and $|1\rangle$ be two orthonormal elements (vectors) in $\mathcal{H}$, and consider the linear operator $Z$ defined by
$$
 Z|0\rangle = |0\rangle
\hskip2cm
 Z|1\rangle = -|1\rangle.
\tag{1}
$$
In the orthonormal basis
$$
 |a\rangle := \frac{|0\rangle+i|1\rangle}{\sqrt{2}}
\hskip2cm
 |b\rangle := \frac{i|0\rangle+|1\rangle}{\sqrt{2}},
\tag{2}
$$
the same linear operator $Z$ is given by
$$
 Z|a\rangle = -i|b\rangle
\hskip2cm
 Z|b\rangle = i|a\rangle.
\tag{3}
$$
Both bases are perfectly legitimate, and indeed there would have been no reason to prefer one over the other before seeing the definition of $Z$. So... what is the complex conjugate of $Z$?
To define the complex conjugate $Z^*$ of $Z$, we need to specify a basis. In the basis (1), we have $Z^*=Z$. In the basis (3), we have $Z^*=-Z$.
The adjoint $Z^\dagger$ of a linear operator $Z$ is defined so that
$$
 \langle A|Z^\dagger|B\rangle = \big(\langle B|Z|A\rangle\big)^*
\tag{4}
$$
for all $|A\rangle,|B\rangle\in\mathcal{H}$. The adjoint is defined independently of any specific basis. The adjoint of $Z$ is $Z^\dagger = Z$, no matter what basis we use. 
Beware that the adjoint $Z^\dagger$ is usually denoted $Z^*$ by mathematicians. That's because the adjoint is the natural (basis-independent) generalization of the concept of complex conjugation. The complex conjugate of a matrix is not natural, because it depends on which matrix representation is used, as illustrated above.
A: In a vector space over the field of complex numbers the notion of complex conjugation is basis dependent. You might say a vector is "real" if its components in some basis are real numbers, but if you change to another basis and the matrix expressing the new basis in terms of the old has complex entries then the "real" vector  will have complex components in the new basis.  Similarly, the notion of complex conjugation will change. To define "complex conjugation" in a basis independent fashion you need to select a set of basis vectors that  you declare to be real.  The technical way this is done is by introducing a real structure on the space. This is an antilinear map $R:V\to V$ such  that $R^2=$ Identity. Antilinear means that $R(\lambda {\bf x})= \lambda^* R({\bf x})$. A "real" vector is then one that is left unchanged by the map $R$.
More important is the concept of "hermitian conjugation", that, when the operator $M$  is represented by a matrix 
in an orthonormal basis
$$
\langle {\bf e}_i, {\bf e}_j\rangle= \delta_{ij}
$$ 
by
$$
M ({\bf e}_i) = {\bf e}_j {M^j}_i,
$$
then the matrix representing  $M^\dagger$ is  given by transposing and complex conjugating the entries in ${M^i}_j$.  In a general basis, where
$$
\langle {\bf e}_i, {\bf e}_j\rangle= g_{ij}, \quad g_{ij}= g_{ji}^*
$$
however, 
you need the full definition
$$
\langle M^\dagger {\bf x},{\bf y}\rangle= \langle{\bf x},M{\bf y}\rangle
$$
so
$$
{(M^\dagger)^i}_j = (g_{jk}{M^k}_l g^{li})^*
 $$
where $g^{ij}g_{jk}= \delta^i_j$. It is important to realize that Hermitian conjugation depends on the choice of inner product on your vector space.
A: Given a matrix $M$ (=written in some basis). Now imagine you want to rewrite it in a different basis.  The basis transformation is given by a unitary matrix $U$.
In the new basis, the matrix reads $M'=UMU^\dagger$. The transpose of $M'$ is then
$$
(M')^T = (UMU^\dagger)^T = U^* M^T U^T\ .
$$
On the other hand, if you first transpose the matrix and then change the basis, you obtain
$$
UM^T U^\dagger\ .
$$
In general, these will be two different matrices (in fact, they differ by a basis change $U^*U^{-1}$).
The situation is different if you for instance consider the hermitian conjugate $M^\dagger$, as you can easily check.
