In particle physics, what is non-resonant decay? How does it manifest itself in invariant mass frame? Does it not peak? It has to peak, right, otherwise, how would anyone know, that there is a decay and not only random noise?
In >2-body decays, non-resonant decays are those which aren't associated with an intermediate resonance. They don't form peak-like structures (at least not narrow ones) in the phase space of the decay, but it is not a problem to distinguish them from background. Of course they still appear in the usual peak of the initial-state particle when reconstructing the invariant mass of the final-state particles. We are talking about the behaviour of intermediate states.
In amplitude analyses using the isobar formalism, it's quite common to include, in addition to the coherent sum of resonances, one (or more) non-resonant terms. The number and shape of the non-resonant terms depend on the spins of the initial and final state particles as well as the amount of available phase space.
In a classic $D \to abc$ Dalitz analysis, where $D$, $a$, $b$ and $c$ are all (pseudo)scalars, the non-resonant term is typically a complex constant that is uniform in phase space, whereas resonances are descrbed by Breit-Wigners or similar functions. In this case you may think of non-resonant decays as "true 3-body" decays instead of sequential $D \to R(\to ab) c$ decays.
Here are the resonances in electron positron annihilation as recorded in the particle data. The energies are positive: the sum of the four masses of the resonant decay products is smaller than the energy scanned for the resonances.At rest these decay products cannot form the "particle" which appears in the invariant mass, and is called a resonance.
The simplest bound state, the hydrogen atom,
If the electron is at an n=2 state, it will decay to n=1 with the width of the bound state, emitting a photon . All bound states are at negative energies. In four vectors the sum of the proton and electron masses is larger than the hydrogen atom mass.
Look at the free neutron decay,
The sum of the outgoing masses is less than the neutron mass, the neutron is a bound state of the strong interaction, and decays due to the existence of the weak interaction.