With the completion of finite theory of physics, even if the theoretical black hole singularity is completely eliminated, how can we be sure that it does not exist in nature? Physics theory is essentially verified by experiment. So, apart from the theoretical elimination, the actual singularity should also be refuted by the experiment, but is this possible?
The scientific method requires that measurements are made, shared, and replicated. It is true that in this sense all the data predicted by Einstein gravity beyond the black hole horizon is beyond the reach of the scientific method because no one will share their results with people outside the horizon. (There might be a society of people inside the black hole for whom the question is relevant, but let's omit that right now.)
To underline this argument, consider the following: the field equations of general relativity allow to be extended beyond null infinity. Null infinity is kind of like spatial infinity, but for good reasons, relativists make a distinction between the two. Null infinity is the point where light-rays go after an infinite amount of oscillations of their waves. Below you can see a Penrose diagram including the extensions of the Schwarzschild space-time beyond null infinity (in the corners). (Taken from Haláček & Ledvinka, 2014) In fact, it turns out that it is convenient to have these "ghost regions" in numerical simulations to compute radiation leaving the space-time! On the other hand, these are obviously beyond the scientific method as well, because to probe them, you have to send a massless object to infinity, and this massless object will never send a signal back.
Consider yet another argument from quantum mechanics. Bohmian quantum mechanics posit that behind the wave-function of the theory, there are real particles that are "piloted" by the wave field. However, the theory is built so that even though the particles have real trajectories and behaviors, their real states are unmeasurable without collapsing the wave-function. Additionally, the theory has a singularity in the moment of the wave-function collapse, and the predictions are identical to usual quantum mechanics. In other words, there is a fundamental experimental horizon beyond which we cannot see the behaviour of these particles. We instead choose to describe a "reduced" theory describing only the "enveloping" objects, normal quantum mechanics describing the wave-function, and we even understand it as more fundamental or "correct" than Bohmian mechanics.
In many approaches to curing the singularity of the black hole, a similar approach is taken as well. These approaches essentially tell you that the black hole interior is not physical at all. You should instead describe an "enveloping", "reduced" theory of black holes and gravity in general. In this theory, it is automatically obvious why a single region of space-time is "the" priviliged physical part, and this will lead you to a more fundamental insight (and true observable corrections outside of the horizon). It might be convenient to keep the interior of the black holes for numerical simulations in a classical, Einstein-gravity limit, but this is somehow only a crutch, the region beyond the horizon is a "ghost region" that tells you nothing of the real physics. A number of these ideas can be summarized as the Holographic principle and various incarnations thereof.
I dont think you can be really sure of anything in science.
By experiment, you can refute theories that lead to singularities and show, that the only (or simplest) theory we concieved of which fits the data is the one without singularities. But i dont think we can ever make such a bold claim as to say "we cannot in principle concieve of any theory with singularities that would fit the data". Data are, after all, always faulty and unprecise. You are always left with regimes you cannot probe. And who knows what awaits us in those regimes?
In science you say something is true if it works and fits together. You may put forward more theories with same predictions (quantum mechanics has several of them) and scientist will choose the true one to be the one with the easiest math and the rest will be left as curiosities. Even though philosophers (and especially general public) could argue science doesnt know what is actually true, for scientists, the truth has more pragmatical meaning. They dont seek or care about absolute truths. They care about truths, that works the best. And that is what is meant, when scientist presents some claims as true.