This question cannot be aswered as it is asked. There is no general mathematical rigorous definition of QFT in general, but rather different approaches with different goals and applications.
First of all, there is what is called Axiomatic Quantum Field Theory, which are attempts to formulate quantum field theories in an mathematical axiomatic way. Note that physical speaking, such approaches start already at the quantum level and do not discuss the process of quantization. Examples of books discussing the famous "Gårding–Wightman axioms" are for example:
- R. F. Streater and A. S. Wightman: PCT, Spin and Statistics and all that. volume of Princeton Landmarks in Physics. Princeton University Press, Princeton, New Jersey, 1964.
- E. de Faria, W. de Melo: Mathematical Aspects of Quantum Field Theory. volume 128 of Cambridge studies in advanced mathematics. Cambridge University Press, Cambridge 2010.
- M. Reed, B. Simon: Methods of Modern Mathematical Physics. II: Fourier Analysis, Self-Adjointness. Academic Press, 1975.
- M. Schottenloher: Chapter 8: Axioms of Relativistic Quantum Field Theory in A Mathematical Introduction to Conformal Field Theory. volume 759 in Lecture Notes in Physics. Springer, Berlin, Heidelberg, 2008.
There are also some good lecture notes, which you can find online (just google them):
- W. Dybalski: Lectures on mathematical foundations of QFT.
- M. Keyl: Mathematical Aspects of Quantum Field Theory.
A great success of these axioms is for example the "Haag-Ruelle scattering theory". Parts of this is discussed in the references cited above.
There are also axioms for Euclidean QFT (called "Osterwalder-Schrader axioms"). These axioms can either be formulated in terms of Schwinger functions or in terms of path integrals (using measures on the space of tempered distributions; using the "Bochner-Minlos Theorem"). A standard reference for this is
- A. Jaffe, J. Glimm: Quantum Physics: A Functional Integral Point of View. Springer, New York, 1987.
Related to axiomatic QFT is "Constructive Quantum Field Theory", which is the area of mathematics trying to find examples of non-perturbative and interactive QFTs satisfying these axioms. The book by A. Jaffe and J. Glimm is also a nice starting point for this point of view.
As pointed out in the comments, up to now, one was only able to construct such theories in low dimensions (see for example this physics SE post). Furthermore, note that finding similar axiomatic approaches for quantum gauge theories is still an open question. In the end, this is one of the "Millennium Prize Problems".
Another nice book discussing both the Wightman axioms and Osterwalder Schrader axioms and some general aspects of non-perturbative QFT is
- F. Strocchi: An Introduction to Non-Perturbative Foundations of Quantum Field Theory. Oxford Science Publications, 2013.
Another mathematical topic in QFT is "Causal Perturbation Theory" (sometimes also called "Finite Quantum Field Theory"), which is a mathematically rigorous construction of perturbative quantum field theory, based on the "Epstein-Glaser approach" of renormalization. Standard references include
- G. Scharf: Finite Quantum Electrodynamics. The Causal Approach (3. edition). Dover, Mineola, New York, 2014.
- G. Scharf: Gauge Field Theories: Spin One and Spin Two: 100 Years After General Relativity. Dover, Mineola, New York, 2016.
(Note that the name of the second volume was changed in later editions. The first edition was called Quantum Gauge Theories – A True Ghost Story. As the name suggest, in the later edition, the author added some discussion of spin 2 particles and gravity as an effective quantum field theory ("perturbative quantum gravity").)
A modern approach to quantum field theory, which axiomatizes the assignment of algebras of observables, is "Algebraic Quantum Field Theory", based on the "Haag-Kastler-Axioms". There are also many good books about that approach. Examples are
- H. Araki: Mathematical Theory of Quantum Fields, Oxford Science Publications, 1999.
- R. Haag: Local Quantum Physics. Fields, Particles, Algebras. Springer, Berlin, Heidelberg, 1996.
- R. Brunetti, C. Dappiaggi, K. Fredenhagen and J. Yngvasson (editors): Advances in Algebraic Quantum Field Theory. volume of Mathematica Physics Studies. Springer International Publishing, 2015.
A book about perturbative aspects of algebraic quantum field theory and locally covariant quantum field theory (including curved spacetime) is
- K. Rejzner: Perturbative Algebraic Quantum Field Theory. An
introduction for Mathematicians. volume of Mathematical Physics
Studies. Springer International Publishing,
However, this is a very active research area and there are much more books out there. Check for example this nlab page or this website.
Another modern approach to QFT is so-called "Functorial Quantum Field Theory", which is based on the discussion of topological QFT in terms of the "Atiyah-Segal Axioms", which in turn are based on previous axiomatic formulations of conformal field theory by G. Segal. I am not an expert on this. Check for example this nlab page.
If you are interested in the process of Quantization, there are also some mathematical approaches like "Geometric Quantization" and "Deformation Quantization". You can find many books about this topic.
Some books covering various different mathematical aspects and tools of QFT are the books by E. Zeidler:
- E. Zeidler: Quantum Field Theory I: Basics in Mathematics and
Physics. A Bridge between Mathematicians and Physicists. Springer,
Berlin, Heidelberg, 2006.
- E. Zeidler: Quantum Field Theory II: Quantum Electrodynamics. A
Bridge between Mathematicians and Physicists. Springer, Berlin,
- E. Zeidler: Quantum Field Theory III: Gauge Theory. A Bridge between
Mathematicians and Physicists. Springer, Berlin, Heidelberg, 2011.
These books contain many different topics, but are more on mathematical tools than on QFT itself. In my personal opinion, they cover some really interesting stuff, however, they are written in a rather chaotic style (some things are discussed several times, etc.).
Quite famous are also the two books
- P. Deligne, P. Etingof, D. Freed, L. Jeffrey, D. Kazhdan, J. Morgan, D. Morrison and E. Witten, (editors): Quantum Fields and Strings, A course for mathematicians. Volume 1. American Mathematical Society, Providence, Rhode Island 1999.
- P. Deligne, P. Etingof, D. Freed, L. Jeffrey, D. Kazhdan, J. Morgan, D. Morrison and E. Witten, (editors): Quantum Fields and Strings, A course for mathematicians. Volume 2. American Mathematical Society, Providence, Rhode Island 1999.
These books also cover many interesting aspects of QFT (the first volume also includes a discussion of the Wightman axioms). However, note that thes books are not really textbooks, but rather collections of various lecture notes. Furthermore, they do not only cover QFT, but also other topics like string theory (especially the second volume).
At this point, I should probably also add the book
- G. B. Folland: Quantum Field Theory: A Tourist Guide for Mathematicians. volume 149 of Mathematical Surveys and Monographs. American Mathematical Society, Providence, Rhode Island, 2008.
This books is basically a "bottom-up" approach, where the author translates QFT from physics to a mathematical language.
Not directly related to QFT, but if you are interested in Mathematical Gauge Theory, there are also some nice mathematical books, like:
- M. J. D. Hamilton: Mathematical Gauge Theory. volume of Universitext.
Springer International Publishing, 2017.
- G. L. Naber: Topology, Geometry and Gauge fields. volume 25 of Texts
in Applied Mathematics (2. edition). Springer, New York, 2011.
- G. Rudolph and M. Schmidt: Differential Geometry and Mathematical
Physics. Part II. Fibre Bundles, Topology and Gauge Fields. volume of
Theoretical and Mathematical Physics. Springer Netherlands, 2017.
This is of course for classical gauge theory, but is for example of interest when you would like to understand the Lagrangian of the standard model in mathematical terms.
Last but not least, let me mention that my list is of course by no means complete. There are also many other topics in mathematical QFT. For example, there is also some literature for mathematical QFT in condensed matter physics, or literature on some more specialized stuff, like on CFTs or supersymmetry. Furthermore, there is also what is called "non-commutative QFT", which is a approach of QFT based on non-commutative geometry (e.g. "non-commutative standard model"). I do not add more references on this more advanced and/or specialized things.