If such a material exists and it absorbs no light at any frequency, then it must have absolutely no optical activity. This is a consequence of the Kramers-Kronig relations, which are very, very basic constraints on how absorption and dispersion in a material can be related to each other, and represent mathematically the physical principle of causality. (That is: you just can't do away with them.)
If $\chi(\omega)=\chi_1(\omega)+i\chi_2(\omega)$ is the material's electric susceptibility at angular frequency $\omega$, then $\chi_1(\omega)$ regulates dispersion and $\chi_1(\omega)$ is proportional to the absorption coefficient. These two functions must obey the relation
$$
\chi_1(\omega)=\frac{1}{\pi}\mathcal{P}\int_{-\infty}^\infty \frac{\chi_2(\omega')}{\omega'-\omega}\mathrm{d}\omega'
$$
and an analogous one giving $\chi_2(\omega)$ in terms of $\chi_1(\omega)$. This means that if $\chi_2(\omega)=0$ for all $\omega$ - if the material absorbs no light, no matter the frequency - then $\chi_1(\omega)$ is also zero and the material has absolutely no dispersion. This is unlikely: all matter is made of charged constituents and they will react to EM radiation to some (nonzero) extent.
For some very nice insights into why dispersion and absorption are so intimately tied up, see this answer,
Causality and linear response in classical electrodynamics. Alex J Yuffa and John A Scales. Eur. J. Phys. 33 no. 6, 1635 (2012),
and
Causality and the Dispersion Relation: Logical Foundations. John S. Toll. Phys. Rev. 104 no. 6, pp. 1760-1770 (1956).
That said, you do stand a chance of having a non-absorptive material at a given, fixed frequency, of course!