Can we make sense of a Hamiltonian $a^\dagger a^\dagger + a a$? If I have a Hamiltonian given by $$ H = a^\dagger a^\dagger + a a$$ where, $[a,a^\dagger] = 1$,
Can I make sense of it, by generalizing the notion of vacuum? 
If not what sort of troubles I would run into? Has there been any instance such Hamiltonians have been considered? 
 A: This Hamiltonian is typical of parametric down-conversion processes, where a strong field (usually a coherent state parametrized by $\alpha$) interacts with some medium to produce two photons $a^\dagger a^\dagger$. 
The Hamiltonians have the slightly more general form 
$$
H\sim \alpha^* \hat a\hat a + \alpha \hat a^\dagger \hat a^\dagger\, .
$$
In your specific case the photons would be degenerate (identical frequencies).  See Eq.(23.12) of these lectures as an example.  This is covered in many quantum optics textbooks.
This Hamiltonian is used to generate squeezed states, as per Eq.(17) of this paper (one can adjust the phase between the terms to be $-1$ if need be).
Finally, note that the operators 
$$
K_+=\frac{1}{2}\hat a^\dagger a^\dagger\, ,\qquad 
K_-=\frac{1}{2} \hat a\hat a\, ,\qquad K_0=\frac{1}{2}\left(\hat a^\dagger \hat a + \hat a\hat a^\dagger\right)
$$
span the Lie algebra $su(1,1)$ so your Hamiltonian is basically the "boost" $K_x$.  I do not remember the eigenstates of $K_x$ as being discrete but Google was unable to find useful link and I could be wrong.   
The action of $H\sim K_x$  (or its exponential if you neeed a time evolution) is perfectly well defined on harmonic oscillator states and expressible in terms of $SU(1,1)$ group functions, given by Ui this paper
A: No, the ground state is not well-defined because the energy is unbounded below. To see this, switch back to the variables $x$ and $p$ using $a \sim x + ip$ to find
$$H \sim p^2 - x^2.$$
This is the Hamiltonian for a particle in a potential that just pushes it further away from the origin, so you can make the energy as negative as you want, and there's no ground state to expand about. Alternatively, if you flip the signs and get $H \sim x^2 - p^2$, you get a negative mass particle, and again you can get arbitrarily negative energy by making it faster and faster. 
This problem can't be fixed by applying a Bogoliubov transformation. These transformations diagonalize Hamiltonians of the form
$$H = a^\dagger a + \alpha (aa + a^\dagger a^\dagger).$$
However, for sufficiently large $|\alpha|$, the Bogoliubov transformation fails to exist, and it certainly fails to exist here, where $\alpha$ is infinite. This failure directly corresponds to the fact that a ground state does not exist, so you can't define a new ground state $|\Omega \rangle$ and excitations about it.
A: I don't believe you'd have a proper notion of a ground-state energy. Typically we prefer something like $a^\dagger a$ because we can have a formal notion of $\langle 0| a^\dagger a|0\rangle$.  So even if you change your definition of what the creation and annihilation operators do to $|0\rangle$, you shouldn't get any new physics out of it.  If this does come up in physics somewhere it would be interesting though.
