No, that's not correct (except as a far-field approximation - see the note at the bottom). You're barking down the wrong tree. Your second equation $$E_{big} = \frac{kQ}{r^2}$$
is valid only for a point charge! You're approximating your cylinders as though they were single point charges at the same point in space, which is of course going to lose most of the complexity of your situation.
There are two ways to go - you can either integrate the electric field contribution from every differential element of charge on both cylinders. That's gonna get ugly.
The best way to go is to use Gauss's law with a cylindrical gaussian surface. This relates the flux through the surface to the charge contained inside the surface. If you pick a convenient symmetrical surface, you can deduce the electric field.
Also, note that you dropped the $k$ accidentally - your last equation should read
$$E_r = \frac{k 2\pi l\sigma}{r^2}(b-a)$$
Interesting side note - your final equation will be valid for $r >> l$, since when you are very far from the cylinders, they are indistinguishable from two superimposed point charges. So you have in fact found what the electric field approaches as you get very far away from your two cylinders.