We certainly can understand sound as a displacement wave, and sometimes it is convenient to. In solids, this is actually essential, because the displacement determines the strain of the material which is critical for describing sound in the solid.
What determines the phase shift is the impedance of the two materials on either side of the boundary, as said here: http://hyperphysics.phy-astr.gsu.edu/hbase/Sound/reflec.html "That is, reflections off a lower impedance medium will be reversed in phase." When a sound wave inside a solid reflects off the air solid interface, it DOES experience a phase shift. This page: https://www.acs.psu.edu/drussell/demos/reflect/reflect.html also has a wonderful description of what is going on here. Try to think about the questions posited with the animations for varying impedance.
Beyond this, really all I can tell you to understand this better is to get dirty and solve the wave equation $\frac{d^2f}{dt^2}=c^2\frac{d^2f}{dx^2}$ yourself. You can say that for $x \leq 0$, $c=1$ and for $x > 0$, $c=2$, and require continuity of $f$ and $\frac{df}{dx}$ as your boundary conditions.
Edit: I couldn't help myself, and you may not know how to do this (it is a common exercise in introductory quantum mechanics, Griffiths' QM book does it) so I'm going to.
Plane waves $e^{\pm i(kx-\omega t)}$ are solutions to this equation, with $\omega^2 = k^2c^2$. I'll drop the time dependence because it only obfuscates and isn't relevant: $e^{\pm ikx}$
Let's send in a plane wave from negative infinity with amplitude one and wavenumber $k$, and let the right side have wavenumber $q$: $ f(x) =
\begin{cases}
e^{ikx} + Be^{-ikx}, & x\leq0 \\
Ce^{iqx}, & x>0
\end{cases}$
At the boundary, $1+B=C$ is required by continuity of $f$ and $k(1-B)=qC$ is required by continuity of $\frac{df}{dx}$. So $B=\frac{k-q}{k+q}$, and $C=\frac{2k}{k+q}$. You can get the reflection and transmission coefficients by $R=|B|^2$, $T=1-R$, but your question is about the phase shift. You get a phase shift in the reflected wave if $B<0$, which happens if $k<q$, which happens if the speed of sound is lower (i.e. the impedance is higher) on the right side. You don't get a phase shift otherwise. There is in neither case a phase shift in the transmitted wave. You can see that all this came from requiring continuity and smoothness while obeying the dispersion relations of the materials.
I hope this helps, cheers!