There is a one to one correspondence between symmetry and group theory for the simple reason that if A is a symmetry and B is a symmetry, then so is B followed by A. This implies that symmetries form a group, where the group law is composition of maps (a symmetry is a map).
Translation along the real axis is the physical symmetry of time translation. As pointed out by Wiener, it is an important fact that if we begin an experiment at time $ t =0$ and measured the results at time $t = 4$, the answers will be, physically, the same as if we had started the experiment at time $t=21$ and measured the results at time $t=25$.
This is the reason why Fourier analysis is useful... whenever you have a symmetry group, the representations of that group are useful. The representations of the translation group are the exponential functions so Fourier analysis, the decomposition of an arbitrary function into combinations of different exponential functions, is useful.