Let me add just a couple of things to what was already mentioned. I do think that the best source for QFT for mathematicians is the the two IAS volumes. But since those are fairly long
and some parts are not easy for mathematicians (I participated a little in writing those down, and I know that largely it was written by people who at the time didn't understand well what
they were writing about), so if you really want to understand the subject in the mathematical way, I would suggest the following order:
1) Make sure you understand quantum mechanics well (there are many mathematical introductions to quantum mechanics; the one I particularly like is the book by Faddeev and Yakubovsky
http://www.amazon.com/Lectures-Mechanics-Mathematics-Students-Mathematical/dp/082184699X)
2) Get some understanding what quantum field theory (mathematically) is about. The source which I like here is the Wightman axioms (as something you might wish for in QFT, but which almost never holds) as presented in the 2nd volume of the book by Reed and Simon on functional analysis; for a little bit more thorough discussion look at Kazhdan's lectures in the IAS volumes.
3) Understand how 2-dimensional conformal field theory works.
If you want a more elementary and more analytic (and more "physical") introduction - look at Gawedzki's lectures in the IAS volumes. If you want something more algebraic, look at Gaitsgory's notes in the same place.
4) Study perturbative QFT (Feynmann diagrams): this is well-covered in IAS volumes
(for a mathematician; a physicist would need a lot more practice than what is done there), but on the spot I don't remember exactly where (but should be easy to find).
5) Try to understand how super-symmetric quantum field theories work. This subject is the hardest for mathematicians but it is also the source of most applications to mathematics.
This is discussed in Witten's lectures in the 2nd IAS volume (there are about 20 of those, I think) and this is really not easy - for example it requires good working knowledge of some aspects of super-differential geometry (also disccussed there), which is a purely mathematical subject but there are very few mathematicians who know it.
There are not many mathematicians who went through all of this, but if you really want
to be able to talk to physicists, I think something like the above scheme is necessary
(by the way: I didn't include string theory in my list - this is an extra subject; there is a good introduction to it in D'Hoker's lectures in the IAS volumes).
Edit: In addition, if you want a purely mathematical introduction to Topological Field Theory, then you can read Segal's notes http://web.archive.org/web/20000901075112/http://www.cgtp.duke.edu/ITP99/segal/;
this is a very accessible (and pleasant) reading! A modern (and technically much harder) mathematical approach to the same subject is developed by Jacob Lurie http://www.math.harvard.edu/~lurie/papers/cobordism.pdf (there is no physical motivation in that paper, but mathematically this is probably the right way to think about topological field theories).