The angular momentum of a rigid body respect to a pole $O$ located on its axis of rotation $z$ is uniquely determined if we know its angular velocity:
$\vec{L}_O = I_z\vec{\omega} - \omega\iiint_V r_zr_y \hat{u}_y \,dm - \omega\iiint_V r_z r_x \hat{u}_x \,dm$
where the first term is the component of the angular momentum parallel to the axis of rotation, that is indipendent from the pole that we chose as long as it lays on the axis of rotation, and the other two terms are the components perpendicular to the axis of rotation, and dipendent from the pole that we choose. Furthermore, we know that if a rigid body is symmetric to its axis of rotation the latter integrals result to zero, so that whichever pole we choose on the rotation axis the angular momentum will be uniquely determined and will be parallel to angular velocity.
So now, let's imagine a physical pendulum , where its center of mass doesn't lay on the rotation axis and that is asymmetric with respect to the axis of rotation (the axis of rotation in this case is perpendicular to the plane of the drawing). We will use the intersection between the rotation axis and the pendulum as our pole $O$. The force of gravity applied to the center of mass generates a torque that is parallel to the axis of rotation. Recalling the second cardinal law of dynamics:
$\vec{{\tau_o}^{(ext)}} = \frac{\partial L_O}{\partial t}$
At moment 0 the angular momentum of the physical pendulum is 0, as a torque of different module but constantly parallel to rotation axis is applied throughout time, an angular momentum parallel to the rotation axis is generated. So in a physical pendulum we get a parallel angular momentum to the rotation axis even if the rigid body is asymmetric to the rotation axis. My physics professor said that it was because the constraint of the fixed rotational axis (he cited also normal force) avoided that an angular momentum perpendicular to the rotational axis could be developed.
How is that?
In the following lines I will try to give my answer.
If we take a look back at the equation of the angular momentum and we take a look at the two integrals we can notice that if our pendulum is flat with respect to the rotational axis the component of the position vector with respect to the rotational axis $r_z$ is 0 for each particle and, therefore, the two latter integrals are zero. Notice that if we change pole along the rotational axis the latter integrals will not be zero, but at the same time the torque calculated with respect to the new pole will also be not parallel to the rotational axis. It all adds up.
But let's imagine that our body is elongated along the rotational axis and take the pole $O$ as the intersection between the rotational axis and the rigid body. For simplicity imagine an irregular 3 dimensional rock of homogeneous density. Now, I'll try to make some heavy claims and I'll hope someone could prove them analitically. If the center of mass lies along the xy plane the latter integrals are 0 and the situation is the same as a flat physical pendulum. My intuition for this claim is that in order for the center of mass to lay on the xy plane, the z component above and under the xy plane should balance and so cancel out. If, instead, the center of mass is not located on the xy plane, than the resultant torque is not parallel to the rotational axis and the latter integrals are not zero, giving an angular momentum that is not parallel to the rotational axis. But let's notice, what does it mean for a rigid body to have its angular momentum not parallel to the angular velocity.
If the body is rotating and the angular momentum is not parallel to the angular velocity, than the angular momentum is changing direction in space. There should be a torque countinously perpendicular to the angular momentum and laying on the xy plane. But in this case it doesn't exist. The unique possible force is the normal force, but being centered in the pole cannot contribute with any torque.
What happens then???