How do people historically have come to use the Yang-Mills theory in physics? There are many books, in which Yang-Mills theory is introduced "just like that". But I didn't find some book with set of historical arguments, which had led people to using it in quantum field theory. Can you tell me about this?
Maybe, my question leads to the next question: how did people guess that they need to expand the group of local gauge invariance for describing, for example, quarks?
 A: I believe the milestone about the introduction of Yang-Mills theory from gauge invariance is the 1973 article by Ernest Abers and Benjamin Lee. You can easily find the original article on the web through a simple google search. This is a fundamental article I would recommend to everybody interested in Quantum Field Theory.
Also, I remember I found good historical, as well as logical, introductions to Y-M theory in the book by Aitchinson and Hey, and in the older one by Cheng and Li of the 1983.
These are certainly good references for the history of the development of gauge theories.
Another milestone is the 1980 article by Gerardus 't Hooft, which I think could provide a great answer to your second question. 
A: Yang-Mills theory is used in two parts of the standard model, The electroweak interactions and strong interactions (QCD).
For the history of Yang-Mills itself see  https://en.wikipedia.org/wiki/Yang%E2%80%93Mills_theory#History_and_theoretical_description
For the Electroweak theory see the histroy of the Higgs Boson https://en.wikipedia.org/wiki/Higgs_boson#History
For the history of how QCD became the theory of the strong force and quarks see https://en.wikipedia.org/wiki/Quantum_chromodynamics#History
If you want still more details see Wikipedia pages of the individuals involved or find one of the many popular books written on the history of the standard model of particle physics. 
