What information do $|\psi(0)\rangle$ and $|\psi(t)\rangle$ represent? I am starting to feel comfortable with the role of the unitary operator in quantum mechanics. For instance, one of the equations I have seen is
\begin{equation} 
|\psi(t)\rangle = U(t) |\psi(0)\rangle
\end{equation} 
I understand what a unitary operator is in that 
\begin{equation}
U^*U=UU^*=I
\end{equation} 
I also understand that if we have a vector space containing our wave function $|\psi(0)\rangle$, then the operator maps to $|\psi(t)\rangle$.
My question:
What information do $|\psi(0)\rangle$ and $|\psi(t)\rangle$ represent? 
If the answer is probability amplitude, then perhaps someone can clarify what exactly that is. As far as I understand, the $\|\psi\|^2$ represents the probability density. But that the amplitude is more fundamental for some reason. 
I ultimately hope to understand why we are using unitary operators in the first equation, but I think I need to figure out this question first before tackling that.  
 A: The ket $ | \psi(t) \rangle$
denotes the state of the system at the time $t$,
and  $|\psi(0) \rangle$ is nothing but the state of the system at the time $t=0$.
The probability amplitude of the state $|\psi(t)\rangle$ being in some state $| x \rangle$ is $\langle x | \psi(t) \rangle$, which when $x$ represents the spatial position of a particle is nothing but the wave function $\psi(x,t)$ (see e.g. this Phys.SE question).
Regarding the relation between probability and probability amplitude, see for example this and this Phys.SE questions.
A: @glance has a good answer, but let me try to answer why amplitude is more fundamental than density.
Go back to the double-slit experiment.
Consider light as a wave, having a phase.
Like any wave, when it goes through the two slits, the separate waves coming out the other side will interfere.
In the places where the waves have the same phase, you get energy.
In the places where the waves have the opposite phase, they cancel out - no energy.
Now, oddly enough, photons work the same way, even though they are particles.
They have an interference pattern, as if they were riding on waves.
In fact, the probability of a photon landing at a particular spot is just the square of the sum of the two light waves at that spot.
So the probability amplitude, which is the height of the wave and its phase, is basically identical with the light wave.
Since the energy in a light wave is essentially the square of its height, the probability of a photon landing in a spot is just proportional to the energy (brightness) of the wave at that point.
ADDED: My understanding of why evolution operators are unitary, is that they are just pure rotations.
They do not change the size of the amplitude, so they don't affect the probability/energy.
In the case of a light wave, they only affect its phase.
If the state vector has N dimensions, what it means is the vector is being rotated in N-space, but its length is not affected.
An interesting consequence of this is that the rotation can always be reversed by applying the opposite rotation.
This means that a quantum system never loses or gains information, and that leads to all kinds of interesting properties, such as quantum cryptography.
A: The Schrödinger equation is an ordinary differential equation of first order and its solution requires one initial condition which is your $\psi(0)$. Imagine you would like to do a quantum experiment. the first step is to prepare your system in a known state ( eg $|\uparrow\rangle$ for a qubit). This state corresponds to your $\psi(0)$. Then you want to do your experiment which is described by the Hamiltonian and a corresponding unitary time evolution $U(t)$. Your system which was prepared initially in the state $\psi(0)$ is after a certain time t in the state $\psi(t)=U(t)\psi(0)$.
Basically you need the unitary operator to describe the time evolution of a system. 
A: The state vector ket describes all that can be known of the system at future time. Unlike classical mechanics where two quantities velocity and position are required to describe the future here a single quantity does it, done at the price of requiring the wave function to be complex. The representation in which you describe the state vector can range from many. If done using the x representation then you get the usual wave function but momentum representation can also be used. However wave function obtained from momentum representation can be used just like the position representation in obtaining the values of various dynamical variables.
A: In general, kets represent states of maximal information. Without more context, it's a completely abstract notion.
For example, suppose that you have a system with just three possible mutually exclusive measurement outcomes. Let $P_k$ be the proposition "the outcome will be $k$" ($k=1,2,3$), and consider building other propositions from them through the logical operators or and and. You will quickly see that this forms a Boolean algebra that's just like the algebra of projection operators to linear subspaces spanned by subsets of some set of mutually orthogonal vectors $\{\mathbf{e}_1,\mathbf{e}_2,\mathbf{e}_3\}$ in any inner product space, e.g.,
$$P_1\,\mathrm{or}\,P_2 \longleftrightarrow \Pi_{12} = \Pi_{\mathrm{span}\{\mathbf{e}_1,\mathbf{e}_2\}}\text{,}$$
with the logical and represented by straight multiplication.
Because each $\Pi_k$ is a projection operator a one-dimensional subspace, it can be written as an outer product of a vector with itself: $\Pi_k = |k\rangle\langle k|$. Given any discrete probability distribution over our three mutually exclusive example outcomes, with probability $p_k$ for outcome $k$, we can represent it with the operator
$$\hat{\rho} = \sum_k p_k|k\rangle\langle k|\text{,}$$
which we'll call a (mixed) state (also density matrix).
If a state is in the form $\hat\rho = |\psi\rangle\langle\psi|$, and therefore is itself a projection to a one-dimensional subspace, then we call it a pure state. In that case, we can just take $|\psi\rangle$ by itself without loss of generality.
Slightly less trivial example: suppose we have a particle in one dimension, and all we're interested in is its position. Then our mutually exclusive outcomes are $\Pi_x = |x\rangle\langle x|$, and in terms of kets, mutual orthogonality means $\langle x|x'\rangle = \delta(x-x')$.

But that the amplitude is more fundamental for some reason. 

Morally speaking, the reason for probability amplitudes is very simple: since probability distributions in general are represented by density matrices, and in a special case (pure state) they can be written as vector "squared" (outer product with itself), so of course when working in a basis on that vector, you'll have to square the components.
I wouldn't call it "fundamental". Probability amplitude is a component of a pure state written in some particular basis, e.g., $\psi(x) \equiv \langle x|\psi\rangle$ is the probability amplitude of outcome $x$. I'd rather consider the states as more fundamental than any particular basis representation. 
Probability amplitudes are general feature of representing outcomes by linear subspaces and then considering probability distributions over them. What's special about quantum probability specifically is that your observables can involve projection operators to any linear subspace you like, so that superposition of orthogonal states like ${\hat\rho}_1 = |\psi\rangle\langle\psi|$ with $|\psi\rangle = \alpha|1\rangle + \beta|2\rangle$ is measurably different from ${\hat\rho}_2 = |\alpha|^2|1\rangle\langle 1| + |\beta|^2|2\rangle\langle 2|$, whereas classically, they are indistinguishable.
