This is entirely based on the way light interacts with the materials. Mirrors are made of materials particularly chosen because they reflect light without distorting it.
Some materials might reflect a lot of light, but the light might (essentially) "bounce" around inside the material in such a way that it comes out at odd angles or with odd colorings in a way that distorts the image. I use quotes because light in the way it propagates through mediums is best described as a wave, and you have to be careful about how exactly you think of a wave "bouncing off" atoms and their electric fields and such. But anyways this distortion is similar to the way frosted glass distorts light passing through it in bathrooms, except that this would be a case of transmission of light, not reflection of light as with a mirror.
Some materials absorb the light, only to re-emit it later. These re-emission usually bear little resemblance to the light waves that originally hit such a material. Similarly, some materials are very selective about what colors and wavelengths of light they reflect and absorb.
All of these latter considerations provide a large class of materials which end up not making good mirrors.