Shankar's definition of adjoint I think Shankar's definition of adjoint operator (in his QM book) differs from many other sources.
On page 26, he made the definition
$$\langle \Omega V|=\langle V|\Omega^\dagger \quad .$$
Now $\langle V|$ belongs to the dual space (mathematicians use the notation $H'$) of $H$, i.e. the space of all complex linear maps $H\to {\mathbb C}$. In math text books, the action of $\Omega^\dagger$ on $\langle V|$ is, $\langle V|$ compose with $\Omega$, i.e. $\langle V|\Omega^\dagger$ should be the linear map which maps $|W\rangle $ to $\langle V|\Omega W\rangle$. Definition in some physics books, e.g. Griffiths, Peter Woit, seems to be equivalent to this. However, if you use Shankar's definition, $\langle V|\Omega^\dagger$ would have been the linear map which maps $|W\rangle $ to $\langle \Omega V|W\rangle$. If $\Omega$ is not self-adjoint, this is different from the math (and Peter Woit) definition.
Am I understanding this matter correctly?
 A: Generally speaking, mathematical notations can fail. This is one place where Dirac notation fails, and it fails partially by being ambiguous.  Here is how I understand how this works.  Note first that I don't like pulling operators into kets.  I think of the symbols inside a ket as labels for the ket, and you can't act with operators on labels.  Thus, here's my way of being careful with matrix elements of non-Hermitian operators.
First, always assume that operators act to the right. That is,
$$
\langle \psi \lvert \hat{\Omega} \rvert \varphi \rangle \to \langle v_\psi,\Omega v_\varphi\rangle\,,
$$
where the second expression is the typical$^1$ mathematical notation for the inner product on a Hilbert space:

*

*$v_{\psi}$ is an alternative notation for $\lvert \psi \rangle$,

*$\Omega v_\varphi$ is the operation of the operator $\Omega$ on the vector $v_{\varphi}$, represented in Dirac notation as $\hat{\Omega}\lvert \phi \rangle$, and

*$\langle v_\psi,\Omega v_\varphi\rangle$ is the inner product of $\Omega v_\varphi$ with $v_{\psi}$.

Thus, I always interpret $\langle \psi \lvert \hat{\Omega} \rvert \varphi \rangle$ as $\langle \psi \lvert \left(\hat{\Omega} \rvert \varphi \rangle\right)$, even if there is an adjoint, i.e., it's also true that
$$
\langle \psi \lvert \hat{\Omega}^{\dagger} \rvert \varphi \rangle
= \langle \psi \lvert \left(\hat{\Omega}^{\dagger} \rvert \varphi \rangle\right)\,.
$$
Now, what about if we wanted to take the inner product in the other direction? Let's trace this through:
$$
\langle \Omega v_\psi, v_\varphi\rangle
=
\langle  v_\psi, \Omega^{\dagger}v_\varphi\rangle
"="
\langle \psi \lvert \left(\hat{\Omega}^{\dagger} \rvert \varphi \rangle\right)
=
\langle \psi \lvert \hat{\Omega}^{\dagger} \rvert \varphi \rangle\,.
$$
This here is what is really meant by the notation $\langle \Omega \psi \lvert = \langle  \psi \lvert\hat{\Omega}^{\dagger}$: it's that the $\hat{\Omega}^{\dagger}$ actually acts to the right on whatever kets are going to be put in that spot.  You do not think of this as $\Omega^{\dagger}$ acting to the left. This is the answer to your following complaint:

In math text books, the action of $\Omega^\dagger$ on $\langle V|$ is, $\langle V|$ compose with $\Omega$, i.e. $\langle V|\Omega^\dagger$ should be the linear map which maps $|W\rangle $ to $\langle V|\Omega W\rangle$.

Well, $\langle  V \lvert\hat{\Omega}^{\dagger}$ does not represent the action of $\hat{\Omega}^{\dagger}$ on $\langle V$. It represents the action of $\hat{\Omega}$.  That is, again,
$$
\langle  V \lvert\hat{\Omega}^{\dagger} \rvert U \rangle
"="
\langle V, \Omega^{\dagger} U\rangle
=
\langle \Omega V, U\rangle\,,
$$
with a slight abuse of (my) notation where I am identifying $V$ (math notation) with $\lvert V \rangle$ (Dirac notation).
We could take this one step further and try to represent this more explicitly in the Dirac notation as
$$
\left(\hat{\Omega}\lvert V \rangle\right)^{\dagger}\lvert U \rangle
=
\langle V \lvert \hat{\Omega}^{\dagger} \rvert U \rangle\,,
$$
where the second expression is understood to be equivalent to $\langle V \lvert \left(\hat{\Omega}^{\dagger} \rvert U \rangle\right)^{\dagger}$,.

Note that this is consistent with the mathematical notation as well:
\begin{align}
\langle \psi \lvert \hat{\Omega} \rvert \varphi \rangle
"="
\langle v_\psi,\Omega v_\varphi\rangle
=
\langle \Omega^{\dagger}v_\psi, v_\varphi\rangle
=
\langle v_\varphi, \Omega^{\dagger} v_\psi\rangle^*
"="
\left(\langle \varphi \lvert \left(\hat{\Omega}^{\dagger} \rvert \psi \rangle\right)\right)^*
=
\langle \varphi \lvert \hat{\Omega}^{\dagger} \rvert \psi \rangle^*\,.
\end{align}

$^1$ Up to the order of the vectors. I've often seen this where the inner product is linear in the first argument rather than the second and anti-linear in the second rather than the frst, but I'll do it the other way that matches the Dirac notation better.
