In a propagation of waves, we can see the interplay of two components: one represents force (pressure, voltage, electric field), the other represents motion (flow, current, magnetic field).
These two components, the yin and yang of waves, move a wave forward by continuously changing each other.
For example, when air pressure increases, it causes air molecules to move, which increases pressure downstream and causes other molecules to move, etc.
For a given type of a wave, in a given type of a medium, the ratio of these two components is determined by the medium and is called impedance, e.g., p/Q for acoustic waves, V/I for transmission line waves, E/H for electromagnetic waves).
Obviously, in a low impedance medium the same force will cause greater motion and vise versa.
As long as the impedance of a medium stays the same, a wave propagates at the same speed and in the same direction. However, when the impedance of the medium abruptly changes, say, due to a transition to a different medium, the same force won't be able to cause the same motion.
For instance, if the second medium has a higher impedance, only part of the wave would be able to proceed forward, forcing the remaining part to go back or reflect.
So, waves are reflected when they encounter discontinuity in the media or discontinuity of impedance, i.e., due to impedance mismatch.
The reflection can also be explained from the energy prospective. As we know, the energy (or power) transfer is at its maximum when the impedance of the load matches the impedance of the source. This means that, whenever the impedance of the medium changes, increases or decreases, not all energy of the wave will be able to go through, i.e., some of the energy will have to be reflected.
More precisely, the ratio of the force and motion components at the boundary will have to satisfy boundary conditions for both media.
Following this logic, we could derive a coefficient of reflection at the boundary, which is the ratio between the force representing the reflected wave and the force representing the incident wave:
Γ=(Z2-Z1)/(Z2+Z1), where Z1 and Z2 are impedances of the first medium and second medium, respectively.
Looking at this formula, we can see what happens with a wave at a boundary under different scenarios.
For instance, if Z1=Z2, Γ=0, so there is no reflection and the wave continues in its merry way.
If Z2 is very high, Γ=~1, i.e., the wave hits a wall or an open circuit and is completely reflected. The boundary condition here demands that nothing flows beyond the wall, and, therefore, the reflected wave must cancel the flow associated with the incident wave. To achieve that, the force at the wall has to be twice as large as the force associated with the incident wave.
If Z2 is very low, Γ=~-1, i.e., the wave encounters no resistance or a short, the force must drop to zero. This means that the force associated with the reflected wave has to cancel the force associated with the incident wave or, we can say, that it will be inverted or out of phase with that force.
This last case could explain why electromagnetic waves reflect from metals with the inversion of the electric field.
The EM wave impedance in free space (and in air) is about 377ohm. The wave impedance of a metal could be close to zero. This forces the electric field on the surface of the metal to be close to zero, which means the electric field associated with the reflected wave has to be out phase with the electric field of the incident wave, in order to cancel it.
The EM wave reflected by a metal is produced by an electric current flowing (mostly) on the surface of the metal and trying to kill the electric field associated with the incident wave in order to satisfy the boundary condition, E=0.
