You are absolutely right that to have a consistent String Theory we need a CFT with the correct conformal anomaly (or conformal charge or central charge, these terms all mean the same thing). However, this does not always mean extra dimensions. For example the free fermionic models [1],[2] are constructed directly in 4d and the extra fields required for the conformal anomaly are implemented as worldsheet fermions.
It is whenever the extra degrees of freedom required to cancel the conformal anomaly are implemented as bosons that we can interpret them as coordinates or extra-dimensions of spacetime. This happens because both bosons and coordinates carry a spacetime index $\mu$ (eg $A^\mu$ and $X^\mu$). Such theories are for obvious reasons called geometric while the rest are called non-geometric.
All such theories are perfectly consistent mathematically, but here is the tradeoff:
i) You can have really simple CFTs (like the CFT of bosons compactified on a 6d torus) that we understand very well, but they give non-realistic models describing the real world. The previous one is an example of what you call a smooth manifold and it gives $N=4$ spacetime SUSY which we know is not a symmetry of nature in low energies.
ii) You can have really complicated CFTs that give realistic 4d physics, (realistic means the SM or the MSSM particles, no other exotic particles, 3 generations, $SU(3)\times SU(2)\times U(1)$ gauge group, etc...). Unfortunately, these CFTs are very difficult to study.
iii) You can have something in-between like orbifolds or some semi-realistic fermionic models that are have some nice features, yet are not too difficult to study.
So, to sum up, you do not have to have an orbifold. You don't even need to have an interpretation of the internal degrees of freedom as a compactified space! The goal is to find a theory that we can study that is as close to the real world as possible.