Does angular acceleration depend on the reference point? Assume that we have a free rigid body, its own weight being the only force acting on it. If I want to know its angular acceleration, then I calculate the total moment acting on the body. Now, if I choose to compute the moment wrt the center of mass, then I will get zero (because that's where the weight is being applied) yielding zero angular acceleration, but there are other points for which the moment of the weight is not zero, yielding a nonzero angular acceleration.
Where is this inconsistency coming from?
 A: The expression τ = Iα = dL/dt derives from Newton's second and assumes that the chosen axis of rotation is in an inertial frame. It cannot be within a falling object.  You can choose an arbitrary horizontal line in the fixed frame as an instantaneous axis. You can then assume that the torque about that axis results from the total weight acting along a vertical line which passes through the the center of gravity (instead of the actual distributed weight).  That torque can be shown to be equal to the rate of change of the angular momentum of the falling object about the chosen axis.
A: Angular momentum is not just "rotation".  A body moving in a straight line has a non-zero angular momentum about a point not along the line of travel.  $L = mvr$, where $r$ is the perpendicular distance between the line of travel and the axis of consideration.
If the velocity changes while the others are constant, this angular momentum will change as well.  In your scenario, the body is accelerating downward.  If your point of reference is not vertically aligned with the center of mass, then the angular momentum will be changing during the fall.

how do you know if the eq $\tau = I \alpha$ holds for that point?

It holds if the angular momentum change can only go into rotation.  If the center of mass can accelerate in a direction that is not in line with your point of consideration, then it does not hold.
For a body falling, then any point in a vertical line from the center of mass is safe.
