Why shadow of light that goes through colored glass becomes colored? The way I understand us seeing colors: for non transparent objects, light of specific wavelength (that matches the color) bounces off object (the rest is absorbed by object) and hits cones in our eye, then based on combination of cones activated (RGB) we perceive color. 
For transparent things light does not get absorbed but goes through the object (nothing gets reflected, nothing hits retina) and we see through it.
For colored transparent objects all but that specific wavelength goes through, specific color gets reflected, all others go through. 
Now obviously I must be making a mistake somewhere in the above...

Why when light goes through colored glass shadow becomes colored to the color of the glass? (Shouldn't it be 'White Light' minus color of the glass? e.g anything but that color)
 A: Let's consider that transparent colored glass, which color wavelength is $\lambda$, It is colored because all the other wavelengths are absorbed and that specific wavelength $\lambda$ gets reflected, In the case when light goes through it, same thing can be observed, all the other wavelengths are absorbed and only that specific $\lambda$  gets through.
For illustration consider the image bellow:


The first filter looks red($\lambda = 700nm$) because all other wavelengths are absorbed and only red is able to go through it or get reflected, Same can be observed with the rest of the filters.
A: 
For colored transparent objects all but that specific wavelength goes through, specific color gets reflected, all others go through.

If that were true, then when you were in between the light source and the colored transparent object (let's say it's red), then it would appear red, but if the colored object were between you and the light source, then it would appear cyan (=anti-red) because all light except red would pass through it.  But this is not how translucent objects work in the real world.
Instead, it must be the case that they absorb all colors but one (just like a regular colored object), and that one color they partially transmit and partially reflect.  (Of course they might absorb some of that color light too.)  This explanation explains why you can't see perfectly through a colored translucent object (because some of the light is being reflected rather than transmitted), and also why the shadow is the same color (because the same color is the one being transmitted.)
Kosta Butbaia's answer is on the right track, but he missed the fact that quite of bit of the one color of light gets reflected as well as transmitted.  His diagram suggests that if you and the light source were on the same side of the translucent object, it would appear completely black, which isn't the case for most colored transparent objects - they usually look more or less the same color on both sides, no matter where the light source is.
