Ok, let me try an answer that I will learn from, if and when it's shot down (but not by pedantry please :).
I want to stay pragmatic and stick what physics is about, measurements and predictions based on those measurements. @JEB gives a nice summary, but I am not sure it's what the OP is after, which is why I write this.
Take a rock and call it matter and say matter is exactly the same thing as mass, as most non physicists would view it. A rock is stuff, and stuff is mass.
Ignore the problems regarding what actually "is" an electron, and go along with the fact that we can assign a charge (through measurement) to it, in this case a charge called mass.
The same idea with the other "permanent" fermions in the rock, quarks in particular, and say that they have a charge as well, again this charge is mass.
So that leaves us with bosons.
Electromagnetic force carriers can travel forever, as far as we know, so we can say they have no mass. As mentioned in the answer above, we can strongly relate photons to the z boson of the weak force, but as for the electroweak force, for this answer I am going to pretend that it has not been discovered.
So the electromagnetic forces that hold the atoms together do not contribute any mass.
The weak force.
This force is weak, because the range of interaction is very small, in turn precisely because the bosons involved are so massive. More on this at The Weak Force : Wikikpedia. So in the spirit of pragmatism, at any particular instant, when you check the mass of the rock, these bosons are making a contribution to the mass.
Gluons and strong nuclear force.
No mass here (to the limits of our ability to measure any), but this is better explained by Do Gluons have mass?
The nuclear force, since it is a residual effect of the strong nuclear force, does not contribute mass either. Nuclear Force: Wikipedia
In total, you have two main contributions to the matter/mass of your rock, measured at any given time, the fermions and the weak massive bosons.