I have a master's degree in operations research and took several PhD level math courses. In particular I'm most familiar with analysis/measure theory, probability theory, and stochastic processes, and have more limited knowledge of algebra, topology, combinatorics, differential equations. I took a course in operator theory and it was very interesting, although the examples were mostly from physics, which were completely over my head. I took intro physics I/II several years ago, and thought electromagnetism was really cool.
I'd like to branch out a bit and study a bit of physics instead of just pure math (in my spare time, not in school anymore), but don't really know where to get started. Is the MIT OCW course decent? What are some texts that would be appropriate given my background? Specifically, I am looking for resources that are mathematically rigorous, but don't require too much prior knowledge of physics itself.