At this stage I feel like I've heard two things: that energy goes into holding the nucleus together OR that energy is released (in the fusion process... I guess?).
It's both, if you admit a sloppy and misleading interpretation of "energy ... holding the nucleus together".
The fusion process releases energy, which winds up as kinetic energy of the reaction products. Therefore, the rest mass of the products is less than that of the reactants, since rest mass is simply total energy measured from the at-rest-relative-to-the-mass frame. To harvest fusion energy, one has to engineer a way to convert the reaction products' kinetic energy to whatever form you want it in.
Conversely, to break the helium nucleus apart by the reverse reaction, we have to put energy back into the system to make this happen. As nucleusses form from smaller parts in exothermic nuclear reactions, the strong nuclear force eventually overwhelms the initially repulsive electromagnetic force such that there is a nett release of energy, rather like a stretched spring's relaxing and thereby converting its potential to kinetic energy as it does so. To reverse the process, i.e. split the nucleus apart again or stretch the spring, energy has to be put back into the system and the reversal can't happen till this happens. It is in this sense that the "energy holds the nucleus together": an energy "debt" must be "repaid" before the inverse fission / string stretching process happens. The notion of a nucleus continuously doing work to hold itself together is wrong and unfortunately tends to be the way people read the statement "energy goes into holding the nucleus together". This was a common layperson's explanation in popular / childrens science books in the 1970s and 1980s that unfortunately would still seem to have life.
As in David Elm's answer, only relatively light nucleusses release energy through fusion. The most stable nucleusses of all, i.e. the ones with the lowest rest mass per nucleon and thus the most tightly bound nucleusses, are iron 56 and and nickel 62. Exothermic reactions for nucleusses heavier or lighter than these tend to be fission and fusion reactions, respectively.