Do neutron star binary systems come from previously active-star binaries, where where both stars have gone supernova and left behind neutron stars that are still in orbit? Or do they form when two previously unbound neutron stars approach each other and fall into orbit?
Neutron star binaries are thought to mostly form from binary systems containing two massive stars, both of which must go through a supernova stage.
Details can be surmised from this talk by Podsialowski (a noted authority on the topic). In order to get neutron star binaries that are close enough to merge (via gravitational wave emission) in the time available since the start of the universe, it is necessary for both mass transfer and common envelope evolution to occur, although another possibility could be 3-body interactions between the binary and other members of a dense cluster.
EDIT: I'm revising my answer slightly in the light of stumbling across a recent paper by Belczynski et al. (2018). In this paper they discuss THREE channels by with close neutron star binaries can occur. (1) The evolution of an isolated binary ststem, as I described above. (2) Complicated interactions between binary systems within a dense stellar aggregate (i.e. a globular cluster). (3) Multi-body interactions and dynamical friction within dense nuclear clusters at the centres of galaxies. The simulations in this paper result in the concluson that although there are routes to form the progenitors of merging neutron star binaries by channels (2) and (3), that the rate of neutron star mergers produced by channel (1) is 100 times higher.