How does smartphone VR really work? I know it doesn't sound like a physics question. But I couldn't find any satisfying answer on tech related forums, so I think that a pinch of scientific rigor is needed.
I've recently tried Google's cartboard VR headset, which allows you to use your smartphone as a virtual reality device. 
The headset basically splits the screen in your left and right eyes, which makes 3D vision possible.
This is a well known cognitive mechanism that I won't discuss here. 
Now, what bothers me is actually the sensor side of the question.
Let us use the following frame of reference for the rest of this discussion:

I've been astonished by the precision with which your head's movement is detected.
Sure, the integrated accelerometer (which is sensitive to gravity), can account for rotations around x and y (roll and pitch) since it knows the direction of the ground. 
But how do you deal with rotations around z (yaw), which is obviously collinear with the direction of gravity?
My guess is that the system uses the smartphone's magnetometer as a compass (your orientation is then computed relatively to the Earth's magnetic north)
However, many people argue that yaw is detected via the gyrometer. Which doesn't seem right to me, for two reasons:


*

*First, even though it's made to detect a rotation, it will easily get lost in some cases. Indeed, if you rotate you head around z at a constant rate (i.e. without any angular acceleration), the device can not tell whether you're rotating clockwise or anti-clockwise (the tangential acceleration is non-existent). In practice, I agree that this problem could perhaps be fixed by looking at the previous time-steps (if an acceleration has stopped but no subsequent deceleration has been recorded, it means the user is still rotating at constant speed). But would it be precise? Since there is no absolute frame of reference, wouldn't the results diverge quickly? 

*Second, the very fine precision of the measurement. I've tried to rotate the phone around its vertical axis as slow as I could, and the program was still sensing my moves. Can a smartphone's gyrometer measure such tiny accelerations?


For these reasons, I believe that the magnetometer (i.e. the compass) is the way to go. Even if you turn and shake you phone very aggressively, the virtual reality reference frame doesn't seem to be disturbed. Which lets me think that the device is taking the external world as a reference, rather than relying on it's internal sense of position.
Now, can a magnetometer achieve such a precision of measurement as well? How is the detection of the Earth's magnetic field not disturbed by surrounding sources (I've put a magnet next to my phone, but never managed to mislead the VR program).
Or could the solution lie in a combination of different sensors? (comparing the measurements of the accelerometer, the gyroscope and the magnetometer to cancel most sources of error)
 A: Phones and other VR systems typically have a 6 axis Inertial Measurement Unit (IMU).  These devices can sense accelerations in all 3 translational axes and all 3 rotational axes.  Software (or sometimes hardware) then integrates these accelerations to give you positions and velocities.  So, to one of your points about how can the smartphone detect small movements about the z-axis, the answer is that those accelerometers are good at their job and can detect very faint accelerations.
The issue IMUs have to deal with is drift, and that's where many of your other questions come in.  If you rotate your phone about the z-axis slowly, then quickly snap it back to where you started, you'll often find that the direction you are virtually facing in is not the same as you started.  This is because the IMU acquired errors during its integration over that long period of time and lost track of where you are.
The solution to this is sensor fusion.  Instead of relying on one sensor, you rely on multiple and attempt to ratify all of them against each other.  The magnetometers are an excellent example.  The magnetic fields around you change much slower than the accelerometers, so they can give more reliable long term pointing information.  You mix the results with complex algorithms.  See Kalman filters for the simple case, though I use the word "simple" very loosely there.
To your question about putting a magnet near the phone, this does not confuse it because it doesn't need to know "north," it just needs a reliable magnetic field that it can fuse with the accelerometer data.  This is where the software gets smart.  If it notices that the magnetic field is substantially increasing, it may assume that the change is not due to movement, but due to something in the environment moving a magnet around.  As a result, it may temporarily ignore the magnetic information and rely almost entirely on integrating the IMU data until the magnetic field stabilizes enough to be useful.
Another fun example of fusion is the Wiimote, before the Wiimote-plus came out.  The original Wii controller had only a 3 axis accelerometer.  It didn't have gyros.  Accordingly, it could easily lose track of where it's being pointed because it's missing key information.  However, most games which used the Wiimote had code to detect a steady 1g acceleration in one direction.  If it detected that, it could assume that that was down because its really hard for a human hand to maintain a steady 1g acceleration.  It then reset the software and maintained its accuracy.
Augmented reality goes one step further.  It adds a camera into the mix.  Now you can not only fuse an IMU and a magnetometer, but you can also try to capture features from the video taken by the camera.  This is very effective because the video sensors don't have drift like an IMU does.
So really, the answer is less "amazing sensors" and more "awesome signal processing."  We can do some incredible things with fused sensor data these days!
A: Looks like magnetometers are indeed used at least in some smartphone VR systems (https://cmoar.com/specifications/)
