What is the physical significance of the off-diagonal moment of inertia matrix elements? The tensor of moment of inertia contains six off-diagonal matrix elements, which vanish if we choose a reference frame aligned with the principal axes of the rotating rigid body; the angular momentum vector is then parallel to the angular velocity. But while considering the general case, what are the off-diagonal moment of inertia matrix elements? That is, do they have any physical significance as [say] the components of a vector? Or is it merely a mathematical construction with no definite physical meaning (which seems rather wrong to me)?
A similar thread exists here but they are more interested in the principal axes of the body. It also says:

The physical significance of the off-diagonal components is that
  you're using a coordinate system not aligned with the principal
  directions of the object. They tell us nothing interesting about the
  object itself.

Is that all or is there more to it, perhaps related to properties of tensors in general?
 A: Product moment of Inertia (PMI, off-diagonal elements like $I_{xy}$) is a measure of "symmetry about a plane".
If an object is symmetrical about a plane, then the PMI about the plane is zero.
Unlike the MI, PMI is NOT about an axis, but about a plane. 
A: Suppose the body is rotating around some axis. Them, each element of mass off the axis of rotation is rotating instantly around the axis of rotation in a circular motion. Because of inertia there will be an inertial force directed outside applied on that element and on the body since the body is rigid. All the mass elements will contribute to the total inertial force. Given some reference frame, usually one with origin at the center of mass, if the moment of the total inertial force is different from zero, the body will tend to change the direction of the axis of rotation. That's the case when the initial axis of rotation is not principal axis of rotation and the products of inertia are different from zero (they depend on the reference frame selected). The total inertial force is not balanced when the product of inertia of the axis of rotation is different from zero, changing the axis of rotation.
It is not necessary for the total inertia force to be zero - the body can move "to the side" without changing the axis of rotation. That's the case when the total inertia (centrifugal) force is different from zero but its moment relative to the origin is null. 
A: The moment-of-inertia (MOI) tensor is real (no imaginary terms), symmetric, and positive-definite.  Linear algebra tells us that for any (3x3) matrix that has those three properties, there's always a set of three perpendicular axes such that the MOI tensor can be expressed as a diagonal tensor in the basis of those axes.  These are called the principal axes (or eigenvectors) of rotation, and the physical meaning behind them is that if you rotate the object around one of those axes, the angular momentum will lie along the axis.  So one important thing to realize is that there is nothing fundamentally meaningful about off-diagonal elements; you can always rotate your coordinates to get rid of them.  If the object has a symmetry axis, then that will be a principal axis.  (Though, having a principal axis does not imply any symmetry of the object.)
On the other hand, what if the body is rotating about an axis that isn't one of the principal axes?  This is equivalent to writing your MOI in a basis where the rotation axis is one of the basis vectors, in which case there are off-diagonal elements, which is what your question is asking about.  So, off-diagonal elements in your MOI are equivalent to having a rotation axis that is not aligned with any of the principal axes.  Again, this only happens when your body is not symmetric about the rotation axis.
And what does this mean physically?  For one thing, the angular momentum is not aligned with the angular velocity.  For example, imagine your object spinning inside a nicely symmetric little satellite in space.  You can see its rotation axis, but if the satellite grabs onto the object, it will absorb the angular momentum, and you'll find the satellite spinning on a different axis.  
Alternatively, you can think of the expression relating torque and angular acceleration $\vec{\tau} = I \cdot \vec{\alpha}$.  An off-diagonal element in the MOI means that if I apply a torque about a certain axis, the object will accelerate its rotation about a different axis.
