Can we define an average time for the entire universe relative to the Big Bang and call this the universal time since the beginning of the universe? (time, averaged relative to all possible reference frames)
I'm guessing that the OP is uncomfortable with the idea of a universal time existing in cosmology interfering with the spirit of the principle of relativity.
The missing piece in seeing why there isn't a problem here is the central assumption one makes in building the matter fields for a cosmological spacetime -- that the universe is homogenous and isotropic and has nonzero mass density.
This then means that there is a special reference frame in the universe -- namely the reference frame that is stationary relative to the mass density of the universe, or more concretely, the stationary frame relative to the cosmic background radiation, which you can detect because if you are moving relative to the CMB, it will be blueshifted in front of you and redshifted behind you, so the special frame is the only reference frame in which the CMB is isotropic in the approximation that you can ignore all of the large scale structure of the CMB.