Well, of course you have to pick the quantities in your dimensional analysis right.
Example: Use dimensional analysis to estimate the potential energy of a star, hold together only by gravitation.
Solution: Newtons gravitational constant $G$ better show up somewhere. This requires us to include something with units $kg^2 / m$. We can get this by inserting $M^2$, with $M$ the mass of the star, and by inserting $1/R$, with $R$ the radius of the star. Thus, the potential energy is estimated to be
$$E_\text{pot} \approx -G \frac{M^2}{R}$$
which is off by a factor $3/5$.
I could, of course, have inserted the mass of a hydrogen atom and then everything would be off by many orders of magnitude...
There is no general guarantee that the constant is of order one. But:
Why it often comes out as "order of 1" is, I believe, the following:
In many cases, the true solution involves an integral over a variable $x$ where the function to be integrated is of the form $f(x) \sim x^n$. In physics, $n$ is small in most cases, so the integration gives a factor $\frac{1}{n}$. Other corrective factors that get ignored in dimensional analysis are $\pi$, $2\pi$ or $4\pi$, i.e., some small integer multiples of $\pi$.
A counterexample... the fine-structure constant $\alpha = \frac{1}{137}$, maybe? But this constant itself can be obtained through dimensional analysis, so I am not sure.