For polarization and angular momentum, rotating the basis corresponds to a very straightforward physical transformation, namely, the physical rotation of an experimental apparatus about an axis in space. But what is the corresponding physical transformation corresponding to a rotation of basis for linear position-momentum? For example, consider a two-slit apparatus. Rotating the basis cannot correspond to rotating the slits. That just transforms the measurement of position along one axis to position along a different axis. It is not a transformation of measuring position to measuring momentum. To measure momentum you have to measure wavelength, which means transforming the slits into a diffraction grating or something like that, but that's as far as I've gotten in figuring this out myself.