The [ket] $ | \psi(t) \rangle$
denotes the state of the system at the time $t$,
and  $|\psi(0) \rangle$ is nothing but the state of the system at the time $t=0$.

The [probability amplitude][prob amp] of the state $|\psi(t)\rangle$ being in some state $| x \rangle$ is $\langle x | \psi(t) \rangle$, which when $x$ represents the spatial position of a particle is nothing but the wave function $\psi(x,t)$ (see e.g. [this Phys.SE question][1]).

Regarding the relation between probability and probability amplitude, see for example [this][2] and [this][3] Phys.SE questions.

[ket]:https://en.wikipedia.org/wiki/Bra%E2%80%93ket_notation
[prob amp]:https://en.wikipedia.org/wiki/Probability_amplitude
[1]:https://physics.stackexchange.com/q/65794/58382
[2]:https://physics.stackexchange.com/q/51962/58382
[3]:https://physics.stackexchange.com/q/57595/58382