In modern theoretical physics you can actually look back to a point in which they call "inflation" which is just 6.36 x 10^-34 of a second, i.e. the Planck Time, after the bang.
The moment just when "our time" starts up, or comes into being if that makes sense. We are pretty sure about 300,000 after the big bang the universe was so hot the leftover heat is all we can see present day and for ever because matter did not exist yet, due in part because there was no gravity yet. The E.M. force just broke away so that electrons could actually do some kind of motion which would be somewhat like a orbit around the atom.
Remember electromagnetism which is light, which is the cosmic background radiation, which is now measured as heat, which we see and measure today.
We had to wait until it was cool enough for electromagnetic force to even have an effect. So you would or could look back to see the leftover heat from the big bang.
Please Note: Currently in mainstream modern physics there our two theories.
Big Bang Model and the "Cyclic Theory"
(Cyclic theory came from = M-Theory = from a collective String theory = introduced by Edward Witten in 1995 during a summer conference.
The "Cyclic Theory claims that two Giant Mem-branes or P-Branes" banged into each other and have been banging into each other this introduces the exact same Cosmic Microwave Background Radiation of 2.65 degrees Kelvin which is actually observed and also predicted by the Big Bang Theory.)
Each model still needs the inflationary theory.