How do gauge boson interact with elementary particles? We know that gauge bosons are the force carriers of fundamental interactions, but how do the gauge bosons themselves interact with particles? 
 A: I think that the only honest answer to this "how" question is (for the electromagnetic interaction -- the strong and weak interactions are analogous) with a mathematical formula: $$\mathcal{L}_\text{int} = e\overline{\psi} \gamma^\mu A_\mu \psi.$$
This is the term that we add to our model to describe the interaction of electrons ($\overline\psi,\psi$) with the electromagnetic field ($A_\mu$). ($e$ is the electron charge and the $\gamma^\mu$ stands for matrices that essentially account for electron spin.) There really is no honest way other than the mathematical to answer the question how, because our fundamental theories are mathematical models. 
Since the fundamental interactions are, well, fundamental, it would be dishonest to say that "it's like a spring or a ball bearing" or some other explanation in terms of familiar objects. Then you will just ask how a spring works. Well, it results from the electromagnetic interaction between the atoms in the spring, and the atoms are held together by the strong interaction, so my explanation is circular. Watch Feynman explain it.
But there is one way that is based on the mathematical model but a little easier to reason with: Feynman diagrams. Some reasoning (and this will be in every quantum field theory textbook, e.g., Srednicki; Peskin & Schroeder; or Weinberg, vol 1; if you want to look up the details) tells you that if you want to calculate the probability of some process in particle physics -- like the probability of producing a Higgs boson in your experiment -- you can do it by summing up progressively more complicated integrals of matrix products, usually very nasty stuff. Feynman diagrams provide a graphical way to quickly and systematically find out which ones you need for the problem at hand. 
It turns out that the electromagnetic interaction1 is captured (here I'm lying a little to you, see the footnote2) by this diagram

where the wavy line represents an electromagnetic gauge boson - a photon - and the straight lines electrically charged particles, for example electrons. You can take it to represent one electron absorbing or emitting a photon, whereby its momentum changes since momentum is conserved. This description of the diagram is honest because it really represents a term in the calculations composed of three objects that each do the following to the state of the system 


*

*produce a state with one more (fewer) photon with a certain momentum and polarization

*produce a state with one fewer electron with a certain momentum and spin

*produce a state with one more electron with a certain momentum and spin


Then to calculate things like how electrons scatter, or how electrons-positron pairs annihilate, we put copies of this diagram together to form more complicated diagrams. The electromagnetic interaction is nice in the sense that the bigger the diagram the less important it will be, so this works well. (This is one of the reasons the strong interaction is harder to understand!) For example these diagrams 
describe the most important contribution to electron-positron scattering.
You can read the left diagram as: an electron emits a photon which is then absorbed by a positron (or vice versa), momentum being conserved in each step; and the right diagram as: an electron and a positron annihilate, then the photon produces an electron-positron pair. In both cases this is what the factors involved in the mathematical expression the diagram represents do to the physical state.
Sometimes the particles represented by internal lines are called virtual particles and people talk about particles "popping in and out of existence". It is important to not reify these too much. First of all,  diagrams also only relate what is called in-states and out-states. Roughly, if the collisions in your experiment are far from the beam sources and detectors, you can use Feynman diagrams to calculate what your detectors will show given what you send from the beam sources. Feynman diagrams cannot tell you what is happening during the collision. They are also inappropriate for bound states like atoms.
In a sense the diagrams are a natural companion to Feynman's other great idea, the path integral or "sum over histories". Essentially the path integral formalism says that to calculate the probability amplitude to go from the initial state to the final you should add up (integrate) certain phase factors for each "path" between them. This translates to: draw all diagrams composed of copies of the fundamental interaction diagrams, such that initial and final particles are as you want them, then calculate and sum them.
It is problematic to elevate infer something about "particles popping in and out of existence" or virtual particles existing for times "too short to be measured" or other quantum mysticism from this description of the path integral, however. Virtual particles are calculation tools. Quantum states only have real particles.
1 Electromagnetic interaction because the strong and the weak interactions have more particles, so they have more building block diagrams. 

2 The lie here is that Feynman diagrams describe the interactions completely. This is not true. Feynman diagrams are what is called a perturbative technique that do not work for very strong fields. There are also some very interesting phenomena in the strong interaction that cannot be discovered by perturbative methods yet are relevant for our universe.
A: Gauge bosons, such as the photon, are indeed the force carriers of fundamental interactions. The interactions are built upon local gauge symmetries of a Lagrangian. A Lagrangian is a list of interactions in our theory. A symmetry is an operation (such as a rotation) that leaves the Lagrangian invariant.
Local gauge symmetries are very restrictive. If we wrote down any old random interactions, they would violate a symmetry. In fact, local gauge symmetries exactly specify the allowed interactions between gauge bosons and fermions and the allowed self-interactions between gauge bosons. That's how we know how the gauge bosons interact.
We cannot deduce everything from gauge symmetry though: we still have to measure/postulate the strength of the interactions and the charges of the fermion fields. Once we've done that, the gauge symmetry tells us the interactions. For fermions in the Standard Model, like electrons, muons and quarks, the interaction is indeed in the form
$$
\text{fermion}\quad\leftrightarrow\quad\text{fermion and photon}
$$
or simiarly,
$$
\text{fermion and anti-fermion}\leftrightarrow\quad\text{photon}.
$$
However, these interactions only occur as part of a bigger series of interactions, otherwise they would violate energy-momentum conservation.
