In most cases, the last term cannot be neglected, as it contains the important electron correlation effects. This term is usually taken into account after the mean-field procedure, via e.g. many-body perturbation theory (MBPT). To make the notations easier, let us write the Hamiltonian as
$$
H=E_\text{vac}+\sum_{p,q}f^q_p :\hspace{-1mm}a^p_q\hspace{-1mm}:+\frac{1}{4}\sum_{p,q,r,s}\tilde{v}^{qs}_{pr} :\hspace{-1mm}a^{pr}_{qs}\hspace{-1mm}: \ ,
$$
where
$$
:\hspace{-1mm}a^p_q\hspace{-1mm}:=:\hspace{-1mm}a^{{\color{black}\dagger}}_pa^{{\color{white}\dagger}}_q\hspace{-1mm}: \ \ , \ \
:\hspace{-1mm}a^{pr}_{qs}\hspace{-1mm}:=:\hspace{-1mm}a^{{\color{black}\dagger}}_pa^{{\color{black}\dagger}}_ra^{{\color{white}\dagger}}_sa^{{\color{white}\dagger}}_q\hspace{-1mm}: \ ,
$$
$$
E_\text{vac}=\sum_{i}h^i_i+\frac{1}{2}\sum_{i,j}\tilde{v}^{ij}_{ij} \ ,
$$
and the Fockian matrix elements are
$$
f^q_p=h^q_p+\sum_{i}\tilde{v}^{qi}_{pi} \ \ , \ \ \tilde{v}^{qs}_{pr}=v^{qs}_{pr}-v^{qs}_{rp} \ .
$$
Indices $p,q,r,s$ stand for general spin orbital indices, while indices $i,j,k,l$ and $a,b,c,d$ denote occupied and virtual orbitals, respectively (hole and particle states).
Note that you can choose in principle any kind of molecular orbitals in the above equations, as the second-quantized Hamiltonian does not care about the underlying one-particle basis (you are not tied to Hartree-Fock, you could choose Kohn-Sham orbitals, etc.). Of course, you have to build all integrals $h^q_p$, $v^{qs}_{pr}$, $f^q_p$ (and consequently the Fermi vacuum energy $E_\text{vac}$) with the chosen orbitals.
Let us introduce the MBPT partitioning $H=H^{(0)}+W$, where
$$
H^{(0)}=E_\text{vac}+\sum_{p}f^p_p :\hspace{-1mm}a^p_p\hspace{-1mm}:
\ \ , \ \ W=\sum_{p\neq q}f^q_p :\hspace{-1mm}a^p_q\hspace{-1mm}:+\frac{1}{4}\sum_{p,q,r,s}\tilde{v}^{qs}_{pr} :\hspace{-1mm}a^{pr}_{qs}\hspace{-1mm}: \ .
$$
The eigenvalue problem of the one-body operator $H^{(0)}$ can be solved using the Wick theorem and the definition of normal ordering:
$$
\begin{aligned}
H^{(0)}|\Phi\rangle&=E_\text{vac}|\Phi\rangle \ , \\
H^{(0)}:\hspace{-1mm}a^a_i\hspace{-1mm}:|\Phi\rangle&=\Big(E_\text{vac}+f^a_a-f^i_i\Big):\hspace{-1mm}a^a_i\hspace{-1mm}:|\Phi\rangle \ , \\
H^{(0)}:\hspace{-1mm}a^{ab}_{ij}\hspace{-1mm}:|\Phi\rangle&=\Big(E_\text{vac}+f^a_a+f^b_b-f^i_i-f^j_j\Big):\hspace{-1mm}a^{ab}_{ij}\hspace{-1mm}:|\Phi\rangle \ ,
\end{aligned}
$$
and so on, where $|\Phi\rangle$ is the Fermi vacuum that is used to define the normal ordering. If canonical Hartree-Fock orbitals are used, then $f^q_p=\delta^q_p\varepsilon_p$, and the zeroth-order excited state energies take the probably more familiar form expressed with HF orbital energies; also, in that case, the first term of the perturbation operator vanishes.
The zeroth-order energy in the Rayleigh-Schrödinger PT is just $E^{(0)}=\langle\Phi|H^{(0)}|\Phi\rangle=E_\text{vac}$, while the first-order energy correction vanishes because of the normal ordering: $E^{(1)}=\langle\Phi|W|\Phi\rangle=0$. The second-order correction can be found with the Wick theorem:
$$
\begin{aligned}
E^{(2)}&=-\sum_{i,a}\frac{\langle\Phi|W:\hspace{-1mm}a^a_i\hspace{-1mm}:|\Phi\rangle\langle\Phi|:\hspace{-1mm}a^i_a\hspace{-1mm}:W|\Phi\rangle}{E_\text{vac}+f^a_a-f^i_i-E_\text{vac}}
-\frac{1}{4}\sum_{i,j,a,b}\frac{\langle\Phi|W:\hspace{-1mm}a^{ab}_{ij}\hspace{-1mm}:|\Phi\rangle\langle\Phi|:\hspace{-1mm}a^{ij}_{ab}\hspace{-1mm}:W|\Phi\rangle}{E_\text{vac}+f^a_a+f^b_b-f^i_i-f^j_j-E_\text{vac}} \\
&=
-\sum_{i,a}\frac{f^a_if^i_a}{f^a_a-f^i_i}
-\frac{1}{4}\sum_{i,j,a,b}\frac{\tilde{v}^{ab}_{ij}\tilde{v}^{ij}_{ab}}{f^a_a+f^b_b-f^i_i-f^j_j} \ ,
\end{aligned}
$$
and so on for higher orders. For HF orbitals, $E_\text{vac}=E_\text{HF}$ and $f^i_a=0$, and we get the usual Moller-Plesset formula:
$$
E=E_\text{HF}-\frac{1}{4}\sum_{i,j,a,b}\frac{\tilde{v}^{ab}_{ij}\tilde{v}^{ij}_{ab}}{\varepsilon_a+\varepsilon_b-\varepsilon_i-\varepsilon_j}+ \, \text{higher orders} \ .
$$
For a more detailed discussion on diagrammatic MBPT and other many-body methods (e.g. MBPT resummation methods like Coupled Cluster), see Many-Body Methods in Chemistry and Physics by Shavitt & Bartlett.