When your matrices are all finite, the eigenvectors of a self-adjoint matrix $\mathbf{A}$ form an orthogonal basis for some space. It's almost too trivial to mention that each basis vector is, indeed, inside of the space that it is part of a basis for.
The momentum operator $\mathbf{\hat p}$ is self-adjoint, and also has a spanning basis of eigenfunctions that span Hilbert space. However, none of the eigenfunctions of $\mathbf{\hat p}$ are square-integrable. What's going on?