I am taking a quantum mechanics class and was assigned this problem:

Among other things, I am asked to find the eigen values of H in terms of a, b and sigma. I'm sort of lost of even how to approach this.

Since A_hat is hermitian, I am assuming that |a> and |b> are orthonormal and complete. After that I'm pretty lost about even how to start. I have roughly written down that

$ \hat H | \lambda_n> = \lambda_n|\lambda_n>$ where $\lambda$ is an eigen value and $|\lambda_n>$ is an eigenvector. What I don't grasp at all, is how to reconcile the outer product,

$|a><b| + |b><a| $ since that result is a matrix, right?

Please be gentle, I'm a beginner.

Thanks!