If you only had a finite number of rational numbers, you could add them a finite number of times, and multiply them a finite number of times and never worry about whether $\pi$ was a rational number. And in particular you wouldn't have to worry about whether there is a number that 3, 3.1, 3.14, ... etcetera converges to.
You could do quantum mechanics the same way, if you only ever had a finite number of state vectors, and just took inner products of them and sums of them and scalar multiples of them, you'd never have to worry about completeness, because you wouldn't worry about whether $3|\Phi_0\rangle$, $3|\Phi_0\rangle+0.1|\Phi_1\rangle$, $3|\Phi_0\rangle+0.1|\Phi_1\rangle+0.04|\Phi_2\rangle$, ... converges (where all the $|\Phi_n\rangle$ are mutually orthogonal).
So if you know you are just going to work with some particular solutions to a particular equation, you absolutely don't need it. And that covers a lot. If every realistic experiment can only distinguish a finite number of states reliably, then for any particular experimental setup you can make a theory that correctly and adequately predicts the probability for every experimentally observable outcome. So that's fully how much it does not matter. It matters if you want to talk about a whole series of better and better experiments, if you have to make a new theoretical setup each time that could get annoying, so if you hear a mathematician tell you that you can setup one system and find all your answers there, the appeal is maybe obvious, assuming you trust the mathematician.
This is not how it happened historically. If you assume the canonical commutators $[x,p]=i\hbar$ and consider the original Heisenberg matrix mechanics, an infinite basis is required if you want to assume all the normal things. But it's not wise to confuse a hypothesis you use for generating experiemental predictions and the actual tools you use to make the actual predictions. In the early days you are trying to learn about a new way of doing things, that's different than doing the actual doing.
But we don't need some state that a whole bunch of different experimental setups converge to. If you can make a state in the lab or in nature, make it, and have it be one of the finitely many. If you can't make it, be honest and realize that it is only a calculation convenience at best: not an experimentally realized setup, outcome, or process.
Mathematicians like completeness, and it can be useful for talking about quantum mechanics, so you should probably learn it. Finite dimensional inner product spaces are already complete, so it's not like you are running around using non-complete spaces.
All inner product spaces have a metric determined by the inner product, it's only brought up since completeness is a property of the metric, i.e. is it complete with respect to that metric.