That's a pretty good way to look at it.  To be more mathematically explicit, notice that the energy of an electric dipole ([see here][1]) with dipole moment $\mathbf p$ in an electric field $\mathbf E$ is
$$
  U = -\mathbf p\cdot\mathbf E = -|\mathbf p||\mathbf E|\cos\theta
$$
This expression is minimized when $\cos\theta = 1$, which is when the angle between the dipole moment vector (which for a physical dipole points from the negative to the positive charge) and the field is $0$.  This is precisely the condition for the dipole to be aligned with the electric field as you described.

If the dipole is not aligned with the field, then it will experience a torque that tends to align it with the field.  You can see why this happens in the physical dipole; the positive charge feels a force in the direction of the field, while the negative charge feels a force in the direction opposite the field, and these both tend to rotate the dipole to align in with the field.  


  [1]: http://en.wikipedia.org/wiki/Dipole