I am puzzled by the concept of scalar fields that arise in conformal field theory in curved backgrounds. In general relativity, so far as I understand it, a scalar field is basically a function defined on the spacetime manifold, meaning that it is invariant under any smooth coordinate transformation $ f(x)=x' $. In special relativity, these objects are defined as things that remain invariant under the Lorentz transformations. So far, everything seems to be ok.
Now imagine a lorentz scalar under the scale transformations: $x'=\lambda x$. We know that a Lorentz scalar field transforms like $\phi'(x')=\lambda^{-\Delta}\phi(x)$ as a scale transformation is not a Lorentz transformation, so the field $\phi(x)$ does not remain invariant under it.
The problem for me arises when people try to write conformal field theories in curved manifolds in $d$ dimensions. For example, they write the following action for a conformal field theory with scalars:
$$ S=\frac{1}{2}\int d^dx\sqrt{g}(g^{\mu\nu}\partial_\mu\phi\partial_\nu\phi+\xi R \phi^2).$$
Then they say that because the field $\phi(x)$ transforms like $\phi'(x')=\lambda^{-\Delta}\phi(x)$ by setting $\xi=(d-2)/4(d-1)$ the action would be invariant under conformal transformations.
However, I do not understand this setup. If $\phi(x)$ is a scalar on the manifold, then it should be invariant under any smooth transformation of coordinates, such as scale transformations. Thus there should not be a $\phi'(x')=\lambda^{-\Delta}\phi(x)$ behavior to begin with. If $\phi(x)$ is not a scalar on the manifold, what mathematical object allows for this behavior?
