IMHO it is better to avoid the use Dirac notation when first encountering the notion of adjoint operators. In the following, we'll assume that we're working with a finite-dimensional complex Hilbert space $H$. This will save us from discussing many technical details in infinite dimensions. To start, let $(\cdot,\cdot):H\times H \longrightarrow \mathbb C$ denote an inner product on $H$ which is anti-linear in the first and linear in the second argument. For $u,v \in H$ it is part of the definition of the inner product that
$$(u,v) = (v,u)^* \quad .\tag{1} $$
Now let $A: H\longrightarrow H$ denote an (everywhere defined) operator on $H$. For any $v\in H$, we have that $Av \in H$ is simply a vector again. Using equation $(1)$ we find
$$ (u,Av) = (Av,u)^* \quad .\tag{2}$$
The adjoint of $A$ is an operator denoted by $A^\dagger: H\longrightarrow H$ which obeys the following equation:
$$ (u,Av) = (A^\dagger u,v) \quad .\tag{3}$$
Note that we've not complex conjugated or transposed anything in the definition$^1$ of the adjoint. In particular, the complex conjugation used in $(1)$ and $(2)$ is only applied to $(u,v) \in \mathbb C$ - nothing is 'absorbed' into an operator. Finally, your last equation can be derived as follows:
$$(Av,u)^* \overset{(1)}{=} (u,Av) \overset{(3)}{=}(A^\dagger u,v) \tag{4} $$
and thus coincides with $(3)$.
Again: The complex conjugation was applied on $\mathbb C$-numbers only and we nowhere transposed anything.
The upshot of this is that it is important to distinguish between an operator and an associated matrix. However, if you have chosen an ordered orthonormal basis $\{e_j\}_{j=1,\ldots,\dim H}$ and define the matrix elements of $A$ and $A^\dagger$ in this basis as $(A)_{ij}:= (e_i,Ae_j) $ and $(A^\dagger)_{ij}:=(e_i,A^\dagger e_j) $, respectively, then it is easy to show that $(A)_{ji}^* = (A^\dagger)_{ij}$ by just using the above properties $(1)$ - $(3)$. Then you can say that the matrix representing $A^\dagger$ in the said basis is the complex conjugated and transposed matrix of $A$.
$^1$ The definition is as follows: For $u,v\in H$ and an operator $A$, consider the linear functional $(u,A\cdot) :H\longrightarrow \mathbb C$ defined by
$$ (u,A \cdot) (v) :=(u,Av) \tag{5} \quad .$$
We can now apply the Riesz representation theorem which guarantees the existence of a $z_u \in H$ such that for all $v \in H$ it holds that
$$(u,A \cdot) (v) = (z_u,v) \tag{6} \quad.$$
This in turn allows us to define an operator, denoted by $A^\dagger$, as follows:
$$ A^\dagger u:=z_u\tag{7} \quad ,$$
for all $ u \in H$. By definition, we then arrive at equation $(3)$. It is a standard exercise to verify that $A^\dagger$ defined through $(7)$ is indeed a well-defined linear map on $H$ with $\left(A^\dagger \right)^\dagger = A$ by using the properties of the inner product.
As mentioned in the beginning, things get much more complicated in infinite dimensions.