Scalar Product and Adjoint Operator in CFT $\newcommand{ip}[2]{\left< #1,  #2\right>} \newcommand{\d}{\, \mathrm d \lambda^n}$For a Hilbert space $(H,\ip \cdot \cdot)$ and an operator$^1$ $A$ the adjoint of $A$ is defined via the relation:
$$ \ip{x}{ A y} = \ip{A^\dagger x} y  \qquad \forall x,y, \in H$$ 
Supposedly the conformal algebra has a representation on $L^2$ functions for example the dilatation take the form 
$$ D = -i x^\mu \partial_\mu $$
However, naively this operator is clearly not Hermitian with the usual scalar product of $L^2$ with Lebesgue measure $\d$
\begin{align}\ip f{Dg} &= \int \bar f (-i x^\mu \partial_\mu g) \d = \int \overline{(-i x^\mu \partial_\mu f)} g \d+ i \int (\bar f g)\, (\partial_\mu x^\mu) \d \\
&= \ip{D f} g + i n \ip fg \quad ,
\end{align}
where $n$ is the dimension of spacetime and I integrated by parts to get the second equality, lastly I used $\partial_\mu x^\mu = n$.
A second hint that there is a different choice of scalar product comes from the fact that for 2d CFT's the adjoint of an operator is defined (cf. [1] p. 32) via 
$$ A(z, \bar z) ^\dagger = A\left( \frac 1 {\bar z} , \frac 1z \right) \frac 1{\bar z^{2h}} \frac1 {z^{2\bar h}} $$
for $(h,\bar h)$ primary fields.
So my question is, what the scalar product one chooses for CFT's $^2$ and how one can derive the "adjoint" rule postulated above by this choice of scalar product.

$^1$ Bounded operator
$^2$ An educated guess would be the Haar measure for the conformal group but I am not sure.
[1] https://arxiv.org/abs/hep-th/9108028v1
 A: The adjoint formula for operators on the radially quantized plane (the one you have written in your question) is derived from a Wick rotation of the Lorentzian definition of adjoint on the cylinder (which is the standard definition).
Let $O(t,x)$ be a Hermitian conformal primary operator with dimension $(h,{\bar h})$ ($t$ is Lorentzian time). It satisfies
$$
O^\dagger(t,x) = O(t,x) , \qquad O(t,x) = e^{ i H t} O(0,x) e^{- i H t} 
$$
In the Wick rotated theory, we define the Euclidean operator $O_E(\tau,x)$ by
$$
O_E(\tau,x) = O(-i \tau,x) = e^{H\tau} O(0,x) e^{-H\tau} . 
$$
The Euclidean opertor has the following adjoint property
$$
O_E^\dagger(\tau,x) = O_E(-\tau,x). 
$$
Now, let us introduce complex coordinates on the plane $w = x - i \tau$ and ${\bar w} = x + i \tau$. The adjoint property then reads
$$
O_E^\dagger(w,{\bar w}) = O_E({\bar w},w). 
$$
Now, consider the theory on a circle with $x \sim x + 2\pi$. We use a conformal transformations to map this to a radially quantized plane. Define
$$
z = e^{iw} = e^{\tau} e^{ix}  , \qquad {\bar z} = e^{- i {\bar w}} = e^\tau e^{- i x } \quad \implies \quad w = - i \log z , \qquad {\bar w} = i \log {\bar z} 
$$ 
Under conformal transformations, primary operators transform as
\begin{align}
O'_E(z,{\bar z}) &= (\partial_z w)^{h} (\partial_{\bar z} {\bar w})^{{\bar h}} O_E(w,{\bar w}) \\
&= (iz)^{-h} (-i{\bar z})^{-{\bar h}} O_E(- i \log z, i \log {\bar z}) ,
\end{align}
or inversely,
$$
O_E(w,{\bar w}) = (ie^{i w})^{h} ( -i e^{-i {\bar w} } )^{{\bar h}} O'_E(e^{iw},e^{-i {\bar w} } )  . 
$$
Note that $O'_E$ is defined on the radial plane whereas $O_E$ is defined on the cylinder. 
We have everything we need! The adjoint of an primary operator on the radially quantized plane is now 
\begin{align}
O'^\dagger_E(z,{\bar z}) &= (-i {\bar z})^{-h} ( i z)^{-{\bar h}}  O_E^\dagger (- i \log z, i \log {\bar z}) \\
&= (-i {\bar z})^{-h} ( i z)^{-{\bar h}}  O_E(i \log {\bar z},- i \log z) \\
&= \frac{1}{ {\bar z}^{2h} } \frac{1}{ z^{2 \bar h} } O'_E \left(  \frac{1}{ {\bar z} }  ,\frac{1}{z} \right)  . 
\end{align}
A: I do not know much on CFT, but I suspect that, dealing with $L^2(\mathbb R_+^n, d^nx)$, there is a trivial adjustment.
The map $\psi(x) \mapsto \psi(\lambda x)$ is not unitary on $L^2(\mathbb R_+^n, d^nx)$:
$$\int \overline{\phi(\lambda x)}  \psi(\lambda x) d^nx = \lambda^{-n}\int \overline{\phi(\lambda x)}  \psi(\lambda x) d^n\lambda x = \lambda^{-n}\int \overline{\phi(x)}  \psi(x) d^nx\:.$$
So it cannot define a symmetry in the sense of Wigner. However
a unitary action of dilatations is the following
$$(U_\lambda \psi)(x):= \lambda^{n/2}\psi(\lambda x)\:.$$
The properly selfadjoint generator of this unitary one-parameter group is not $-ix^\mu\partial_\mu$ but is 
$$D = -i\left(\frac{n}{2} +x^\mu \partial_\mu\right)\:,$$
and $U_\lambda = e^{i\lambda D}$.
A: Your assumption that the representation you are talking about in the sentence

Supposedly the conformal algebra has a representation on $L^2$ functions 

should be Hermitian is incorrect. The space of states of a 2d conformal field theory is not functions on the plane - the functions on the plane are the fields/operators of the theory, not the states. There is no reason to expect the action of the algebra on the operators to be Hermitian w.r.t. to some inner product on the operators.
CFTs are field theories and so their spaces of states are more complicated than the simple $L^2(\mathbb{R}^n)$ spaces of ordinary quantum mechanics. E.g. the free 2d scalar field on a cylinder is a conformal field theory and simply has the Fock space of a free scalar field as its space of states.
