In particle physics the groups $SU(n)$ and its representations are more important than the groups $U(n)$. Is it only because these groups happen to give a theory that agrees with experiments, or is there some fundamental reason for the unit determinant condition to be important? I know $U(1)$ appears in electrodynamics, so if there is such a fundamental reason then QED must somehow be exceptional.
As an experimental physicist my vote goes to :"because these groups happen to give a theory that agrees with experiments"
The SU(2) represented spin, and then isotopic spin. Then the plethora of new resonances was organised in the eightfold away in SU(3) representations.
It is the flavor SU(3) representation.
And then the strong force came in in order to keep the quarks bound in the resonances of the eightfold way :
Greenberg and Han–Nambu independently resolved the problem by proposing that quarks possess an additional SU(3) gauge degree of freedom, later called color charge. Han and Nambu noted that quarks might interact via an octet of vector gauge bosons: the gluons.
This is the SU(3) color interaction of the quanrks in the standard model, and SU(2) is for the weak interactions of the quarks.
Definitely experimental results led to these groups for representing the elementary particle interactions.