The reason you can hear sound around the door is only in part due to diffraction. As you said, the walls are not completely rigid. In fact, the sound is passing through the door as well. Consider a speaker on the other side of a wooden door that is producing a sound. The door is completely shut. The reason you can hear it is because the sound is passing through the door, as well as "reflecting" off of it. It is analogous to a light wave interacting with water. The glare you see off the water is light reflecting off the surface, but the water is translucent, which means light is also entering the water. In the same sense, the sound will bounce off the door (this can result in an echo) as well as pass through the door, which is why you can hear it on the other side.
Which is the most important when hearing indirectly? Well it depends on the situation. If the door is open, then diffraction will be the significant contributor. However, if the door is shut, then assuming the door is sealed well, the wave passing through the door will obviously be the biggest contributor. The material of the door is important since sound waves pass through some materials better than others.
Yes, the contributions can be calculated, however, it would be very difficult to calculate in some situations.