It's a side effect of the unreasonable effectiveness of mathematics. You are in good company thinking it is a little strange.
Many quantities in physics can be related to each other by a few lines of algebra. These tend to be the models that we think of as "pretty."
Terms manipulated by pure algebra tend to pick up integer factors, or factors that are integers raised to integer powers; if only a few algebraic manipulations are involved, the integers and their powers tend to be small ones.
Other quantities may be related by a few lines of calculus. From calculus you get the transcendental numbers, which can't be related to the integers by solving an algebraic equation. But there are lots of algebraic transformations you can do to relate one integral to another, and so many of these transcendental numbers can be related to each other by factors of small integers raised to small integer powers. This is why we spend a lot of time talking about $\pi$, $e$, and sometimes Bernoulli's $\gamma$, but don't really have a whole library of irrational constants for people to memorize.
Most of constants with many significant digits come from unit conversions, and are essentially historical accidents. Carl Witthoft gives the example of $E=mc^2$ having a numerical factor if you want the energy in BTUs. The BTU is the heat that's needed to raise the temperature of a pound of water by one degree Fahrenheit, so in addition to the entirely historical difference between kilograms and pounds and Rankine and Kelvin it's tied up with the heat capacity of water. It's a great unit if you're designing a boiler! But it doesn't have any place in the Einstein equation, because $E\propto mc^2$ is a fact of nature that is much simpler and more fundamental than the rotational and vibrational spectrum of the water molecule.
There are several places where there are real, dimensionless constants of nature that, so far as anyone knows, are not small integers and familiar transcendental numbers raised to small integer powers. The most famous is probably the electromagnetic fine structure constant $\alpha \approx 1/137.06$, defined by the relationship $\alpha \hbar c = e^2/4\pi\epsilon_0$, where this $e$ is the electric charge on a proton. The fine structure constant is the "strength" of electromagnetism, and the fact that $\alpha\ll1$ is a big part of why we can claim to "understand" quantum electrodynamics. "Simple" interactions between two charges, like exchanging one photon, contribute to the energy with a factor of $\alpha$ out front, perhaps multiplied by some ratio of small integers raised to small powers. The interaction of exchanging two photons "at once," which makes a "loop" in the Feynman diagram, contributes to the energy with a factor of $\alpha^2$, as do all the other "one-loop" interactions. Interactions with two "loops" (three photons at once, two photons and a particle-antiparticle fluctuation, etc.) contribute at the scale of $\alpha^3$. Since $\alpha\approx0.01$, each "order" of interactions contributes roughly two more significant digits to whatever quantity you're calculating. It's not until sixth- or seventh-order that there begin to be thousands of topologically-allowed Feynman diagrams, contributing so many hundreds of contributions at level of $\alpha^{n}$ that it starts to clobber the calculation at $\alpha^{n-1}$. An entry point to the literature.
The microscopic theory of the strong force, quantum chromodynamics, is essentially identical to the microscopic theory of electromagnetism, except with eight charged gluons instead of one neutral photon and a different coupling constant $\alpha_s$. Unfortunately for us, $\alpha_s \approx 1$, so for systems with only light quarks, computing a few "simple" quark-gluon interactions and stopping gives results that are completely unrelated to the strong force that we see. If there is a heavy quark involved, QCD is again perturbative, but not nearly so successfully as electromagnetism.
There is no theory which explains why $\alpha$ is small (though there have been efforts), and no theory that explains why $\alpha_s$ is large. It is a mystery. And it will continue to feel like a mystery until some model is developed where $\alpha$ or $\alpha_s$ can be computed in terms of other constants multiplied by transcendental numbers and small integers raised to small powers, at which point it will again be a mystery why mathematics is so effective.
A commenter asks
Isn't α already expressible in terms of physical constants or did you mean to say mathematical constants like π or e?
It's certainly true that
$$ \alpha \equiv \frac{e^2}{4\pi\epsilon_0} \frac1{\hbar c} $$
defines $\alpha$ in terms of other experimentally measured quantities. However, one of those quantities is not like the other. To my mind, the dimensionless $\alpha$ is the fundamental constant of electromagnetism; the size of the unit of charge and the polarization of the vacuum are related derived quantities. Consider the Coulomb force between two unit charges:
$$
F = \frac{e^2}{4\pi\epsilon_0}\frac1{r^2} = \alpha\frac{\hbar c}{r^2}
$$
This is exactly the sort of formulation that badroit was asking about: the force depends on the minimum lump of angular momentum $\hbar$, the characteristic constant of relativity $c$, the distance $r$, and a dimensionless constant for which we have no good explanation.
