By Stone's theorem on one-parameter unitary groups we know that there is a one-to-one correspondence between strongly continuous one-parameter unitary groups and self-adjoint operators. Hence, if $H$ (here the Hamiltonian) is a self-adjoint operator, there is a corresponding one-parameter unitary group of the form $U(t) = e^{-itH/\hbar}$. The mathematics is clear. What I am confused about is the physics. How do we know that the parameter $t \in \mathbb{R}$ corresponds to time and not some other parameter? Is this simply taken as a postulate?
Note I have already seen the answers on this post Why does time evolution operator have the form $U(t) = e^{-itH}$? but the questions seem to address more of the mathematics and not the physics.