Imagine the coil is totally free to move on a flat surface the plane of which is perpendicular to the external magnetic field.
The torques and forces acting on the coil will make the coil orientate itself so that the magnetic field that is produced inside the coil is in the same direction as the external magnetic field as this is the lowest potential energy state.
The "plane" of the coil is now at right angles to the external magnetic field and so the forces on the coil will be outwards.
These forces will thus make the coil form a circle to reach a stable equilibrium state with the net force and the net torque on the coil both zero.
If the coil found itself with its magnetic field in the opposite direction to the external magnetic field and was unable to flip over then the forces would be inwards and the coil would collapse in on itself as this would be a minimum potential energy state for this configuration.