In the comments Alfred raises the notion of the dual space. In fact, if you try to read Dirac's principles of QM, you will find that he starts with dual space.
In Dirac notation $|z\rangle$ is an element of an abstract vector space $\mathcal{H}$. Then, there is a notion of dual space: the dual space $\mathcal{H}^*$ is the space of all (continious) linear functionals on $\mathcal{H}$. Here the continuity(as well as the topology) is required only in infinite-dimensional space, in finite-dimensional case with reasonable topology the continuity is granted. Now, linear functional is just a linear function $v:\mathcal{H}\to\mathbb{C}$. It takes a $|z\rangle\in\mathcal{H}$ to the number $v(|z\rangle)\in\mathbb{C}$ and you have $$v(\alpha|z\rangle+\beta|x\rangle)=\alpha v(|z\rangle)+\beta v(|x\rangle).$$
Now, the Dirac notation is to write $\langle v|z \rangle$ instead of $v(|z\rangle)$. That is, $|z\rangle\in\mathcal{H}$ while $\langle v|\in\mathcal{H}^*$.
Then an assumption is made. There is a hermitian inner product on $\mathcal{H}$. That is, for any pair of vetors $|x\rangle,|y\rangle\in\mathcal{H}$ we have a number $\left(\alpha|x\rangle,\beta|y\rangle\right)=\bar{\alpha}\beta\left(|x\rangle,|y\rangle\right)$. (Caution: mathematicians usulally put the bar above $\beta$). This inner product creates an isomorphism between $\mathcal{H}$ and $\mathcal{H}^*$. That is, for any vector $|x\rangle\in\mathcal{H}$ define the functional $\langle x|\in\mathcal{H}^*$ by its action on vectors:
$$
\langle x|z\rangle:=\left(|x\rangle,|z\rangle\right).
$$
In this formulation $\dagger$, hermitian conjugate, is defined for operators:
$$
\left(|x\rangle,A|y\rangle\right)=\left(A^\dagger|x\rangle,|y\rangle\right).
$$
For vectors it is defined usually in the matrix notation as the complex conjugate of the transpose. From the written below it is clear that it is natural to extend $\dagger$ to this formalism as $\dagger:\mathcal{H}\to\mathcal{H}^*$, $|z\rangle^\dagger=\langle z|$.
In finite-dimensional space you can pick a basis $|b_i\rangle$ and identify a vector with its coordinates: $|z\rangle=z^i|b_i\rangle$. No reason not to arrange them in a column $Z$. Then you can define a dual basis $\langle \beta^j|$ in $\mathcal{H}^*$ by
$$
\langle \beta^j | b_i\rangle=\delta^j_i\\
\beta^j(|b_i\rangle)=\delta^j_i
$$
(the last line is in 'standard' notation, to remind you that here is no scalar product involved). Then a functional can be identified with its coordinates $\langle a|=a_j\langle \beta^j|$. If we arrange them into a row $A$, then it can be checked that the number $a(|z\rangle)=\langle a|z \rangle$ is given by $AZ$.
So, you can think about rows as of the elements of the dual space $\mathcal{H}^*$ (and identify them with bras), and columns as of the elements of the space $\mathcal{H}$ (and identify them with kets). What about conjugate transpose? If you now say that there is a hermitian inner product on your space, and $|b_i\rangle$ is an orthonormal basis, then this product for two vectors represented by columns $X$ and $Y$ is given by $X^\dagger Y$, where $\dagger$ is the usual conjugate transpose. Then it is easy to see that the mentioned isomorphism $\mathcal{H}\to\mathcal{H}^*$ is provided by $\dagger$ taking columns to rows.
(This is not mathematically rigorous, start from the fact that actually in the hermitian case it is called anti-isomorphism, etc..)
