From the first man to the present day men, all of them have made some sound. Sound is an energy, it can neither be created nor can it be destroyed. Therefore, every word spoken by each human that came into this world made some sound. Their voices might have converted into another form of energy. But, we have not lost that sound energy. Am I right?
(In fact, Michael has got most of the points. Er...)
First of all, Sound is a longitudinal wave which means it moves via compression / rarefaction. Whatever objects it interact (comparatively massive ones like a cloth, paper, stone, atoms), it affects them. Well, it can be easily noticed in a sub-woofer. Being a mechanical wave, it just tries to push, thereby disturbing objects. As Michael said, the sound energy is converted to heat energy and is lost as it propagates through the medium. The reason it can't be easily observed because, it is so negligible (similar to an elastic band or spring, after it is released from tension when elastic energy is converted to heat energy)
But, this can be observed in wood or plastic-like objects which are probably used for echo-prevention. For example, If you pass sound in a room completely covered with wood, no waves get reflected back. All are lost as heat-energy within wood itself. A great practical application would be Ultrasonic welding where high-frequency sound waves can be used to generate enough heat to weld plastics, etc.
Suppose you stand in the middle of a room and make a sound. When you make a sound you are causing air molecules to vibrate. The sound wave travels away from you and hits the walls, and it makes the molecules in the wall vibrate. These vibrations spread through the wall, then into the ground and in principle make the entire Earth vibrate, though of course the energy in the original sound gets spread more and more thinly and by the time the vibration has spread to the Earth it's energy is swamped by thermal noise.
So in principle the energy is still there, but in practice it's so widely spread that it would be impossible to reconstruct it. This is one of the many consequences of the second law of thermodynamics.
Actually the situation is worse than I described above because liquids don't have the vibrational modes of solids and if the sound hits a liquid the energy is rapidly dispersed as kinetic energy of the molecules in the liquid. This would be even harder to track backwards than the vibrations in the wall.