Generally speaking, mathematical notations can fail. This is one place where Dirac notation fails, and it fails partially by being ambiguous. Here is how I understand how this works. Note first that I don't like pulling operators into kets. I think of the symbols inside a ket as labels for the ket, and you can't act with operators on labels. Thus, here's my way of being careful with matrix elements of non-Hermitian operators.
First, always assume that operators act to the right. That is,
$$
\langle \psi \lvert \hat{\Omega} \rvert \varphi \rangle \to \langle v_\psi,\Omega v_\varphi\rangle\,,
$$
where the second expression is the typical$^1$ mathematical notation for the inner product on a Hilbert space:
- $v_{\psi}$ is an alternative notation for $\lvert \psi \rangle$,
- $\Omega v_\varphi$ is the operation of the operator $\Omega$ on the vector $v_{\varphi}$, represented in Dirac notation as $\hat{\Omega}\lvert \phi \rangle$, and
- $\langle v_\psi,\Omega v_\varphi\rangle$ is the inner product of $\Omega v_\varphi$ with $v_{\psi}$.
Thus, I always interpret $\langle \psi \lvert \hat{\Omega} \rvert \varphi \rangle$ as $\langle \psi \lvert \left(\hat{\Omega} \rvert \varphi \rangle\right)$, even if there is an adjoint, i.e., it's also true that
$$
\langle \psi \lvert \hat{\Omega}^{\dagger} \rvert \varphi \rangle
= \langle \psi \lvert \left(\hat{\Omega}^{\dagger} \rvert \varphi \rangle\right)\,.
$$
Now, what about if we wanted to take the inner product in the other direction? Let's trace this through:
$$
\langle \Omega v_\psi, v_\varphi\rangle
=
\langle v_\psi, \Omega^{\dagger}v_\varphi\rangle
"="
\langle \psi \lvert \left(\hat{\Omega}^{\dagger} \rvert \varphi \rangle\right)
=
\langle \psi \lvert \hat{\Omega}^{\dagger} \rvert \varphi \rangle\,.
$$
This here is what is really meant by the notation $\langle \Omega \psi \lvert = \langle \psi \lvert\hat{\Omega}^{\dagger}$: it's that the $\hat{\Omega}^{\dagger}$ actually acts to the right on whatever kets are going to be put in that spot. You do not think of this as $\Omega^{\dagger}$ acting to the left. This is the answer to your following complaint:
In math text books, the action of $\Omega^\dagger$ on $\langle V|$ is, $\langle V|$ compose with $\Omega$, i.e. $\langle V|\Omega^\dagger$ should be the linear map which maps $|W\rangle $ to $\langle V|\Omega W\rangle$.
Well, $\langle V \lvert\hat{\Omega}^{\dagger}$ does not represent the action of $\hat{\Omega}^{\dagger}$ on $\langle V\rvert$. It represents the action of ${\hat{\Omega}}$ on $\lvert V\rangle$, than conjugated. That is, again,
$$
\langle V \lvert\hat{\Omega}^{\dagger} \rvert U \rangle
"="
\langle V, \Omega^{\dagger} U\rangle
=
\langle \Omega V, U\rangle\,,
$$
with a slight abuse of (my) notation where I am identifying $V$ (math notation) with $\lvert V \rangle$ (Dirac notation).
We could take this one step further and try to represent this more explicitly in the Dirac notation as
$$
\left(\hat{\Omega}\lvert V \rangle\right)^{\dagger}\lvert U \rangle
=
\langle V \lvert \hat{\Omega}^{\dagger} \rvert U \rangle\,,
$$
where the second expression is understood to be equivalent to $\langle V \lvert \left(\hat{\Omega}^{\dagger} \rvert U \rangle\right)$.
Note that this is consistent with the mathematical notation as well:
\begin{align}
\langle \psi \lvert \hat{\Omega} \rvert \varphi \rangle
"="
\langle v_\psi,\Omega v_\varphi\rangle
=
\langle \Omega^{\dagger}v_\psi, v_\varphi\rangle
=
\langle v_\varphi, \Omega^{\dagger} v_\psi\rangle^*
"="
\left(\langle \varphi \lvert \left(\hat{\Omega}^{\dagger} \rvert \psi \rangle\right)\right)^*
=
\langle \varphi \lvert \hat{\Omega}^{\dagger} \rvert \psi \rangle^*\,.
\end{align}
$^1$ Up to the order of the vectors. I've often seen this where the inner product is linear in the first argument rather than the second and anti-linear in the second rather than the frst, but I'll do it the other way that matches the Dirac notation better.
