Certain particles come together to form a nucleus, this causes a certain amount of energy to be released in the form of photons, the mass of the nucleus is less than the sum of the masses of its components, it is said that a certain mass has been converted into energy. If this itself is considered as a closed system, the energy of the resulting photons is considered as part of the mass of the system, which means that the mass of the system is conserved and is equal to the sum of the initial components. In the same phenomenon, it has been described in the first place that there is a loss of mass and in the second that the mass is conserved. It doesn't seem consistent. Is there something wrong with this reasoning?
Indeed, the whole idea of “converting” mass into energy by $E=mc^2$ is a popular misconception. (Note that $E=mc^2$ applies in the center of momentum frame, so I will assume that from here on). Since energy of an isolated system is conserved $E$ must be constant, so there is no loss of mass and no gain of energy. The mass and the energy are both constant and they are related by $E=mc^2$ at all times.
What is more correct is that a system of particles with some given energy can be converted in to other particles with the same total energy, and therefore the same system mass. So, a slow moving electron and positron can be converted into two photons. The photons have an energy of 511 keV each, so that implies that the electron and positron each had 511 keV of rest energy.
Similarly, since the electron and positron together had 1 MeV/c^2 mass so also the system of the two photons have 1 MeV/c^2 mass. This means that the mass of a system is more than the sum of the masses of the pieces of the system. Mass of an isolated system is conserved, not the sum of the masses of the parts.
I've always found "binding energy" to be an unhelpful term. It just exists for chemists and nuclear scientists to do their book-keeping.
Energy is possessed by particles. It does not exist on its own. And mass is simply a property that energy has when it is confined.
Look up "photon in a box" and you can read about how if you were to confine massless photons in a massless box, that box would still start exhibiting properties like inertia, i.e. it would have mass. Atoms are essentially nothing but this.
So it's not a question of "mass being converted to energy," although it's said this way as a shorthand. It's really "confined energy being released as a particle's kinetic energy." To your question, mass is not conserved except approximately in macroscopic chemical reactions and physical processes. Energy is.
What happens in nuclear fusion reactions for example is:
(Deuteron) + (Deuteron) = (Helium) + (some photons, electrons, and neutrinos)
The particles of light and matter in that last term carry away energy X. Helium has lower energy than the two Deuterons combined did (by an amount X), which is why Helium doesn't spontaneously decay back into two Deuterons. It doesn't have enough energy to do so, unless something else (like for instance some other photon or matter particle with at least kinetic energy X) comes and crashes into it. That's why in this latter case they call X the Binding Energy of Helium, because it's the energy you have to add to get it to break into something else. And that also means you can't talk about binding energy without specifying the reaction you are talking about. The energy to break He into 2 Deuterons is much less than the energy to break it into 2 protons and 2 neutrons, which is itself MUCH less than the energy to break the quarks apart into pairs. So it's binding energy "with respect to" a certain process the atom could undergo, not an inherent thing every Helium atom has.
Hope that is helpful.