I have studied Chern-Simons (CS) theory somewhat and I am puzzled by the question of how diff. and gauge invariance in CS theory are related, e.g. in $SU(2)$ CS theory. In particular, I would like to know about the relation between large gauge transformations and large diffeos. If you know any good sources, I would be really grateful. Thank you!
A useful textbook for your purposes is "Gravitation and gauge symmetries" by M. Blagojevic (IOP, 2002). It has a chapter on Chern-Simons theory and its relation to 3-dimensional gravity.
The first two section of Ed Witten's paper http://arXiv.org/abs/arXiv:0706.3359 and references therein should also be useful.
BTW, the SL(2)xSL(2) Chern-Simons theory is basically the "Palatini" formulation of 3D gravity in terms of Cartan variables (dreibein and dualized spin connection). It is a unique feature of 3 dimensions that you can linearly combine the Vielbein with the dualized connection, since only in 3 dimensions the dual of an antisymmetric tensor is a vector. The gauge symmetries of this Chern-Simons theory correspond to diffeos and local Lorentz trafos (at least on-shell).
Take a look at the book "Lecture notes on Chern-Simons-Witten theory" by Sen Hu, it consists of notes of Witten's lectures on the subject. You can easily download it with google search "lecture notes on chern-simons-witten theory djvu". I am not a specialist, but it seems that in classical theory the action is invariant with respect to diffeomorphisms and not gauge invariant. Though the theory is well-defined if the action takes integer values. So the constant k is quantized. There are some subtleties with boundary data, I hope the book can clarify these.