Expanding on Willie Wong's comment: to have a geometrical compactification, you must assume that geometry is still approximately valid at the string scale, so that you can use supergravity. The arguments for Calabi-Yau compactification were given by Candelas Horowitz Strominger and Witten in 1985. They worked in the supergravity approximation, but the argument was only based on the supersymmetry of the low energy approximation, so the Calabi-Yaus are expected to lift to exact solutions of string theory.
The stringent condition is that there is a low energy supersymmetry that survives compactification. This means that there is a spinor which is covariantly constant, that is, which is unchanged under parallel transport in the compactification manifold. Parallel transport on a 6 dimensional manifold gives SO(6) rotation for each loop going from point x back to x, and SO(6) is SU(4) (p to a double cover, it's just like SU(2) and SO(3)) and if there is a spinor at x which is constant under these rotations, you can make it (1,0,0,0) by doing an SU(4) rotation, and then the only SU(4) rotations which leave it constant are the SU(3) acting on the last three components (this argument looks like it is a miracle of Lie-group algebra, and restricted to six dimensions, but in any even dimension 2n there is an embedding of SU(n) into SO(2n) which just pairs up the real coordinates into complex coordinates--- this is familiar from the reduction of the SO(10) GUT to the SU(5) GUT).
The definition of Calabi Yaus is that their parallel transport on loops is restricted to an SU(3). So the compactification manifolds which preserve exactly one supersymmetry are classified by Calabi Yaus.