This is a supplement to freude's correct answer:
Hamiltonian is the infinitesimal generator of time translation defined as
$$\mathrm{\hat{U}}(\mathrm dt)= 1- \frac{i}{\hbar} \mathrm{\hat{H}}(t)~\mathrm dt\;.$$
Time-Evolution Operator:
Let the system be at $|\phi\rangle\;.$ Now, let's wait for some time.....
What is the probability amplitude of finding our system at $|\chi\rangle\;?$
It shouldn't be $\langle\chi|\phi\rangle$ as now we have waited for a certain time-interval; this delay must be taken into consideration.
Time-evolution operator $\mathrm{\hat{U}}$ then comes to rescue.
Suppose, the system is prepared at $|\phi\rangle$ at $t_1$. What is the probability amplitude of finding our system at state $|\chi\rangle$ at time $t_2\;?$
The required amplitude is written as
$$\left\langle\chi\left|\mathrm{\hat{U}}(t_2,t_1)\right|\phi\right\rangle$$
Or if expanded over base states, this can be written as
$$\sum_{jk}\langle\chi |j\rangle\left\langle j\left|\mathrm{\hat{U}}(t_2,t_1)\right|k\right\rangle\langle k|\phi\rangle\;.$$
The $\mathrm{\hat U}$ matrix:
The probability amplitude of finding our system at a different state at some later time after preparing it at another state can be written as
$$|\psi(t+\Delta t)\rangle = \mathrm{\hat{U}}(t+\Delta t, t)|\psi(t)\rangle$$
Multiplying both sides by $\langle j|$, the base state, we get
$$\langle j|\psi(t+\Delta t)\rangle = \left\langle j\left|\mathrm{\hat{U}}(t+\Delta t, t)\right|\psi(t)\right\rangle$$
Resolving our state vector $|\psi(t+\Delta t)$ to our concerned base states, we get
$$\langle j|\psi(t+\Delta t)\rangle =\sum_{k} \left\langle j\left|\mathrm{\hat{U}}(t+\Delta t, t)\right|k\right\rangle\langle k|\psi(t)\rangle\;.$$
$$\mathrm{U}_{jk}\equiv \left\langle j\left|\mathrm{\hat{U}}(t+\Delta t, t)\right|k\right\rangle$$ constitutes one of the elements of $\rm{\hat{U}}$ matrix.
Then we can reform the probability amplitude as:
$$\langle j|\psi(t+\Delta t)\rangle =\sum_{k} \mathrm{U}_{jk} C_k(t)$$
where $C_{k}(t)$ represents the probability amplitude of finding our system in the base state $|k\rangle$ at time $t\;.$
What does this imply?
This means that the amplitude of finding the system at a certain base state at $t+\Delta t$ is proportional to all the other amplitudes $C_k$ at time $t\;.$
The Hamiltonian:
We can write the probability amplitude as:
$$C_j(t+\Delta t) =\sum_{j} \mathrm{U}_{jk} C_k(t)\;.$$
As $\Delta t\to 0\,,$ $$\mathrm{ U}_{jk}\to \delta_{jk} \;.$$
So, we can write $$\mathrm{U}_{jk}(t+\Delta t,t)= \delta_{jk}+ \left(\frac{-i}{\hbar}\right)\mathrm{H}_{jk}\,\Delta t $$ where $\mathrm{H}_{jk}$ is defined as
$$\mathrm{ H}_{jk}= \lim_{\Delta t\to 0}\,\frac{\mathrm{U}(t+\Delta t)_{jk}- \mathrm{U}(t)_{jk}}{\Delta t}\;.$$
Using this, we re-write our amplitude as:
$$C_j(t+\Delta t)= \sum_{k}\left[\delta_{jk}- \left(\frac{i}{\hbar}\right)\,\mathrm{H}_{jk}(t)~\mathrm dt\right]\, C_k(t)$$
The elements $\mathrm{H}_{jk}$ constitute the Hamiltonian matrix.
$\mathrm H$s determine the time-variation of the state of the system; they include the "physics of situation" which cause the coefficients to change over time.
The physical situation can correspond to electric field, varying magnetic field- anything. $\rm H$s ascertain what will happen over time.
tl;dr:
To translate the state over a time-interval or to know about the time-development of a system, we use time-evolution operator as
$$\mathrm{\hat U}(t_2,t_1)|\psi(t_1)\rangle= \exp\left[\frac{-i}{\hbar}\int_{t_1}^{t_2}\,\mathrm{\hat H}(t')\,\mathrm dt'\right] |\psi(t_1)\rangle\;.$$
Here, $\rm{\hat H}$ generates infinitesimal time-translation.
But how should I interpret $\langle \phi | H | \psi \rangle$?
It represents the probability amplitude of transition per unit time of finding our system at $\phi$ provided that the system was prepared at $\psi\;.$
References:
$\bullet$ Lectures on Physics by Feynman, Leighton, Sands.
$\bullet$ A Modern Approach to Quantum Mechanics by John S. Townsend.