Nice question. I don't know much about either differential geometry or algebraic topology, but having studied groups a little, I think I can provide some references for Lie groups. So here are the books I found useful

- [Samelson, Notes on Lie Algebras](http://math.stanford.edu/~ykatznel/samelson/liealgeb.pdf) written in a *Definition, Theorem, Proof* style, so it's little hard to grasp (I recommend mutliple rereading) but gives a good overview of the structure, classification (root systems and Dynkin diagrams) and representations (highest weight theory) of Lie algebras.

- [Humphreys, Introduction to Lie Algebras and Representation Theory](http://www.amazon.com/Introduction-Algebras-Representation-Graduate-Mathematics/dp/0387900535) less theorem-heavy and more talkative than *Samelson* and contains huge number of great exercises.

- [Fulton, Harris, Representation Theory A First Course](http://www.amazon.com/Representation-Theory-Graduate-Mathematics-Readings/dp/0387974954) discusses more or less everything a physicist needs to know about groups (also mentions some finite groups). Lacks the systematic theorem based approach of the two books above, but boasts great explanations and nice pictures. I'd suggest it as a nice first reading about groups it if weren't for its length.

- [Goodman, Wallach, Representations and Invariants of the Classical Groups](http://www.amazon.com/Representations-Invariants-Encyclopedia-Mathematics-Applications/dp/0521663482) this is an ultimate bible on groups. Authors take an algebraic geometrical approach to the Lie groups (instead of the usual differential geometrical) which makes the book somewhat hard to read for a regular physicist. But besides this the book provides an in-depth look at lots of concrete representations (e.g. tensor representations and connection with symmetric group; this is often omitted elsewhere), discusses highest weight theory at great length, provides a nice introduction to spinors and also mentions branching rules. And lots of other stuff. Definitely recommended.