Let $H(n,\mathbb C) \subset M(n,\mathbb C)$ denote the vector space over $\mathbb R$ of Hermitian traceless $n \times n$ matrices. It is well-defined: As the trace is linear, a linear combination of traceless matrices is traceless too, and a real combination of Hermitian matrices is Hermitian as well.
Define
$$\langle A| B \rangle := Re\: tr(A^\dagger B) \:, \qquad A,B \in H(n,\mathbb C) \:.$$
It is easy to prove that $\langle \cdot |\cdot \rangle : H(n,\mathbb C) \times H(n,\mathbb C) \to \mathbb R$ is $\mathbb R$-bilinear and symmetric from the elementary properties of the trace. It is also strictly positive, indeed
$$\langle A|A \rangle = Re \: tr(A^\dagger A) = Re \left(\sum_{a,b=1}^n \overline{A_{ab}}A_{ab} \right) = \sum_{a,b=1}^n|A_{ab}|^2 \geq 0$$
where, evidently $\langle A|A \rangle=0$ implies $A_{ab}=0$ for all $a,b$.
All that proves that $\langle \cdot |\cdot \rangle$ is a well defined real scalar product over the real vector space $H(n,\mathbb C)$.
Proposition. $H(n,\mathbb C)$ equipped with the real scalar product $\langle \cdot |\cdot \rangle$ is a real Hilbert space.
Proof. $$||A||:= \sqrt{\langle A|A \rangle} = \sqrt{\sum_{a,b=1}^n|A_{ab}|^2}$$
is nothing but the standard Euclidean norm of $\mathbb C^{k}$ with $k=n^2$ which, viewed as a metric space with distance function $d(A,B) := ||A-B||$, is complete as is well known. In other words $M(n, \mathbb C) \equiv C^{n^2}$ is a complete metric space. To conclude it is enough to establish that $H(n,\mathbb C)$ is a closed subset of $M(n, \mathbb C)$ with respect to the topology induced by that distance.
In fact, a Cauchy sequence in $H(n, \mathbb C)$ is also Cauchy in $M(n, \mathbb C)$ which is complete and thus admits a limit therein. Assuming that $H(n, \mathbb C)$ is closed, it includes that limit, too. $H(n, \mathbb C)$ is therefore complete with respect to the norm induced from $\langle \cdot|\cdot \rangle$ and thus is a real Hilbert space as wanted.
Let us prove that $H(n, \mathbb C)$ is closed. The conditions $$\mbox{$tr A=0$ and $||A-A^\dagger|| =0$ for $A\in M(n, \mathbb C)$}$$ which define $H(n, \mathbb C)$, can be seen as defining the pre-images $f^{-1}(\{0\})$ and $g^{-1}(\{0\})$ of a pair of continuous maps $$f : M(n, \mathbb C) \ni A \mapsto tr A \in \mathbb C$$ and $$g: M(n, \mathbb C) \ni A \mapsto ||A-A^\dagger|| \in \mathbb R\:.$$ Continuity can be proved by direct inspection decomposing these functions into elementary continuous functions over $\mathbb R^p$. The two pre-images
$f^{-1}(\{0\})$ and $g^{-1}(\{0\})$
are closed sets because they are pre-images of the closed set $\{0\}$ and the functions are continuous. The intersection
$$H(n, \mathbb C) = f^{-1}(\{0\})\cap g^{-1}(\{0\}) $$
is closed because the intersection of closed sets is closed too. QED