From my understanding objects moving close to the speed of light experience a time distortion in that time moves slower and slower as they get closer to the speed of light. If that is the case shouldn't photons be affected by this as well and cause us to wrongly calculate the age of the universe and distance between stars as we don't take into consideration time distortion?
The time dilation (to use the proper term - not distortion) occurs in the frame of reference of the moving object. So only the photon "feels" that time has slowed down. In our reference frame, we simply observe a photon whizzing alone at precisely c. So all our observations that depend on knowing the speed of light remain accurate.
Actually, if you consider the magnitude of the time dilation in the case of a photon:
$t'$ = $t \over \sqrt(1-v^2/c^2)$
as $v \rightarrow c$, the denominator $\rightarrow 0$ so the time dilation is undefined. The photon can then traverse the entire universe in zero time in its reference frame.