Not understanding the minus sign in the torque equation for a simple pendulum 
The pic above is from Introduction to mechanics by Kleppner. In the torque equation they justified the minus sign because the torque has a clockwise sense. This makes sense to me if I pick the y axis to be upward and the x axis to point to the right in the sketch given in the book. Then, the torque would point in the negative z direction. But what if I picked my y axis to point downwards? Then the torque would point in the positive z direction and so my torque would be positive, which would give me a completely different differential equation for the same system. And what about the case when the pendulum is coming down from the other side? Then it has an 'anticlockwise sense' and the torque should be positive. This is all very confusing to me.
 A: 
This makes sense to me if I pick the y axis to be upward and the x
  axis to point to the right in the sketch given in the book. Then, the
  torque would point in the negative z direction.

In the diagram $x$ points down, $y$ points right (conventionally). (See: polar coordinates)

But what if I picked my y axis to point downwards? Then the torque
  would point in the positive z direction and so my torque would be
  positive, which would give me a completely different differential
  equation for the same system.

Flipping the direction of $y$ while keeping the direction of $x$ is equivalent to redefining the positive direction of the $\phi$ coordinate. So rather than the negative sign dropping out from the cross product, it arises from the angle being negative: $\tau_a=Wr_\perp=Wl\sin(-\phi)=-Wl\sin\phi$ .

And what about the case when the pendulum is coming down from the
  other side? Then it has an 'anticlockwise sense' and the torque should
  be positive.

This scenario is the same as the above.
A: How to get correct the torque equation:

I) you choose arbitrary pendulum rotation , my choice is clockwise positive  (blue arrow).
II) the inertia  torque is allays opposite direction to yours  pendulum rotation choice
III) the    torque of the mass is clockwise
IV) take the sum of the torque  equal zero.
$$\sum\tau_+=-\tau_I+\tau_{mg}=-I_a\ddot{\phi}-m\,g\,l\sin(\phi)=0$$
Edit:
the "mass torque"  is negative because the y coordinate is upwards ($-m\,g$)
A: It comes down to choosing whether counterclockwise or clockwise will be positive. That's the easiest way for me to think about it. If the pendulum starts on the right hand side and the angle is positive, that states that CCW is positive since you had to move CCW to get to that position. Therefore, since the pendulum will rotate CW the torque should be negative. If the pendulum starts on the left hand side and the angle is positive, that inherently states that CW is positive and the pendulum would move CCW so the torque would also be negative. It also holds when the angles are negative using the same logic.
