Shankar's Active/Passive Change of Basis I'm working my way through Shankar's Quantum Mechanics (7th printing, and I'm doing it alone, so I apologize if I have core concepts completely wrong).
He has a section on Active and Passive Transformations (which seems to be slightly different from what I know as Active & Passive Transformations, but that would be a different question :) ), and it reads like this:

Suppose we subject all the vectors $|V\rangle$ in a space to a unitary transformation
  $$ |V\rangle \rightarrow U|V\rangle $$
  Under this transformation, the matrix elements of any operator $\Omega$ are modified as follows:
  $$ \langle V'|\Omega|V\rangle \rightarrow \langle UV'|\Omega|UV\rangle = \langle V'|U^\dagger\Omega U|V\rangle $$
  It is clear that the same change would be effected if we left the vectors alone and subjected all operators to the change
  $$ \Omega\rightarrow U^\dagger\Omega U$$

I was trying to wrap my head around this, so I was working through some examples I made up and realized that the formula $\Omega\rightarrow U^\dagger\Omega U$ only works when $U$ is unitary. In my own examples I worked it out for an arbitrary (viz. non-unitary) change of basis T to be $$\Omega\rightarrow T^{-1}\Omega T$$ Like I said, I worked this out myself, so it may be wrong, but it checks out for the unitary case because when $U$ is unitary, $U^{-1} = U^\dagger$.
My question is: what part of Shankar's logic above makes use of the fact that U is unitary? It seems like his logic holds fine even without that constraint, but then it gives us a formula that isn't true for non-unitary U.
 A: If we rewrite the bra-ket expression in terms of matrix multiplication you can say
$$\langle V | \Omega | W \rangle \equiv V^{\dagger} \cdot \Omega \cdot W $$
where $\cdot$ means matrix multiplcation. Now suppose we transform $V$ and $W$ by a transformation $T$ (avoiding $U$ because I'm not saying anything about unitary-ness). Then we find
\begin{eqnarray}
\langle TV | \Omega | TW \rangle & \equiv & (T \cdot V)^{\dagger} \cdot \Omega \cdot (T \cdot W) \\
&=& V^{\dagger} \cdot T^{\dagger}\cdot \Omega \cdot T \cdot W \\
&=& V^{\dagger} \cdot (T^{\dagger}\cdot\Omega\cdot T )\cdot W \\
&\equiv& \langle V | T^{\dagger}\Omega T | W \rangle.
\end{eqnarray}
So actually, the daggers are the general case and it's only in the case where $T$ is unitary that you can write the transformation of $\Omega$ as $\Omega \rightarrow T^{-1}\Omega T$.
A: The unitary condition for the transformation is not about the math here. A meaningful transformation for a quantum system must be unitary. Actually I think probably your derivation is right. You just need to assume this unitary condition.  
A: By definition the matrix elements of an operator $\Omega$ are the expressions $\langle V|\Omega|V'\rangle$, and they transform as DanielSank described. However, their interpretation as actual coefficients of the matrix of $\Omega$ in the basis $V,V',\ldots$ only is correct when the $V,V',\ldots$ form an orthonormal basis. If $A$ is the matrix of $\Omega$ in any basis, and $T$ is the matrix mapping it to a different basis, the matrix of $\Omega$ in the new basis becomes $T^{-1}AT$. 
In other words, the inner products $\langle TV|\Omega|TV'\rangle$, called matrix elements, are equal to $\langle V|T^\dagger\Omega T|V'\rangle$, but the elements of the matrix of $\Omega$ in the basis consisting of the columns of $T$ are equal to $\langle V|T^{-1}\Omega T|V'\rangle$, if the $V,V',\ldots$ form an orthonormal basis. These are equal only if the $TV, TV',\ldots$ still form an orthonormal basis, which is equivalent to $T$ being unitary.
