The best math book I ever read with respect to being useful for physics is

• Vector Calculus, Linear Algebra, and Differential Forms: A Unified Approach (2nd Edition), by Hubbard and Hubbard.

It is an absolute gem. It gets you through linear algebra and differential forms starting from square one, assuming you only know algebra and calculus. The proofs are legitimate and in some cases really creative. The best part is that it's aimed at people who want to use math for applications. Extremization of functions on manifolds is developed really well and the authors give insightful information on how to approach the analytical topics presented in the book numerically. Really useful things like finding Taylor series for implicit functions is done well. I really can't give this book enough endorsement.

After I read that I read

• Analysis On Manifolds by Munkres

This book does integration of differential forms formally. Still, it's amazingly readable, and I never found one single mistake in the entire book. This was a great read and reinforced my understanding, but was not directly relevant to physics.

Then later I read

• Spacetime and Geometry: An Introduction to General Relativity, by Sean Carroll

which is an excellent introduction to curved manifolds. It's nice because he clearly explains the difference between vectors and co-vectors ("up" and "down" indices) and relates it all to real life (ie. physics).