What is different in representation? I'm sorry if this is somewhat a dumb question.
First: "Representation theory is a branch of mathematics that studies abstract algebraic structures by representing their elements as linear transformations of vector spaces"
I know little about particle physics, but to what I know, physicists only deal with the groups of (linear) symmetric operators acting on vector space of states.
So in fact physicists are dealing with the italic part of the representation theory. Why would them bring it in? What is a significance of the action "representing an element of a group as linear transformation" in the work of physicists, who are already dealing with groups of linear transformations?
 A: A group $G$ by itself is not a group of linear transformations, it is an abstract algebraic object. Only its representations map its elements (injectively if the representation is faithful) to elements $\mathrm{Aut}(V)$ of some vector space $V$.
Now, physics seems to have no need of such abstract language at first. Our "vector space" is pretty much our spacetime, and its pretty much $\mathbb{R}^4$, so your symmetries are really just matrices on that spacetime. The Lorentz symmetry is just $\mathrm{SO}(1,3)$ in its fundamental representation on Minkowski space $\mathbb{R}^{1,3}$, right? Or non-relativistic, rotational symmetry is just $\mathrm{SO}(3)$ on $\mathbb{R}^3$, right?
...and then there is angular momentum and spin. If you solve the Schrödinger equation for the energy levels of a Hydrogen atom, you find that the energy levels are characterized by "quantum numbers" $(n,l,m,s)$. Now $n$ is boring. But $l$ and $m$ are eigenvalues of the spherical laplacian, and lead to the beloved spherical harmonics $Y^l_m$ as independent solutions. Turns out, if you rotate the system in space, these harmonics behave differently depending on their $l$! Formally, the space 
$$H_l := \{\sum_m c_m Y^l_m | m \in \{-l,-l+1,\dots,l\}\wedge c_m \in \mathbb{R}\}$$
is a vector space, and it carries a representation of the rotation group $\mathrm{SO}(3)$! But not the fundamental one, if $l > 1$. So there's your non-fundamental representation arising solely by solving the equations describing a physical system.
It gets even weirder for these rotation groups, since it also turns out that there are objects, the fermions, which do not transform in a representation of $\mathrm{SO}(1,3)$ or $\mathrm{SO}(3)$, but in a representation of their universal covers, $\mathrm{Spin}(1,3)$ or $\mathrm{SU}(2)$, respectively. You have no chance to describe the kinds of phenomena you observe for fermions without accepting that they transform that way.
And that's not the end of the story. If you build a gauge theory with gauge group $G$, you will find that the associated field strength of the gauge field must transform as an element of the adjoint representation of $G$. Non-fundmental representations pervade many aspects of (quantum) field theory in that way.
A: Here is what I thought:
In particular physics system a group of symmetric operators (let's say acting on the Hilbert space V) is a subgroup of the group L(V).
Therefore with representation theory instead of dealing with L(V) we can deal with an irreducible representation L(A) which is substantially simpler than L(V).
It's quite f* awesome to figure out something make sense on your own. (Even if it may not be true)
