What is weak coupling of photon polarization to a pointer?

This question is refered to those who are familiar with the concept of weak measurement.

In short:

How can the polarization of a photon be coupled to the position of a pointer state? What is the pointer state? The position of some particle? How to realise that in a lab?

In detail:

Let's say we want to weakly measure the polarization of a photon. We write the observable $A=|H\rangle\langle H| - |V\rangle\langle V|$ and the initial state shall be $|\psi\rangle = \alpha |H\rangle + \beta |V\rangle$. Now we weakly couple it to a pointer by sending it through a birefringent element, i.e. depending on the polarization the photon needs more time to pass the birefringent element.

Formally:

The Hamiltonian whichs represents this coupling is given by $H=\lambda A \otimes P$ where $\lambda$ is the coupling strenght and $P$ the momentum operator of the pointer. Now $\exp{(-\text i H t)}$ will act as follows:

$|\psi\rangle \otimes |g\rangle = (\alpha |H\rangle + \beta |V\rangle) \otimes |g\rangle \;\longrightarrow\; ( \alpha |H\rangle \otimes|g_+\rangle + \beta |V\rangle\otimes|g_-\rangle)$

where the first degree of freedome is clearly the polarization of the photon and the second is the pointer state (a gaussian say). Both $g_+$ and $g_-$ refere to small shifts due to the birefringent element.

Then there is this well known fact that a post-selection whichs leads to an imaginary weak value $A_w$ causes a shift in the momentum-space of the pointer beeing proportional to $\text{Im} A_w$.

The weak value is defined as $A_w = \frac{\langle \phi|A|\psi\rangle }{\langle \phi|\psi\rangle}$ where $|\phi\rangle$ is the post-selected polarization state of the photon.

Mathematically this is easy to understand, but I hace a very conceptual problem with all that: How do I have to understand the pointer state? Is it a position of a particle? If yes, how can its position be coupled (in a physically understandable way) to a photon? And how can a post-selection affect its momentum?