The notation
$$ \tag 1 | \psi(t) \rangle$$
denotes the state of the system at the time $t$,
and  $|\psi(0) \rangle$ is nothing but the state of the system at the time $t=0$.

The probability amplitude of the state $|\psi(t)\rangle$ being in some state $| x \rangle$ is $\langle x | \psi(t) \rangle$, which when $x$ represents the spatial position of a particle is nothing but the wave function $\psi(x,t)$.