The current non-existence of a solution to the Yang-Mills millenium problem doesn't really have anything to do with perturbation theory. Instead it has to do with mathematical rigor.
At the physics level of rigor, we have both perturbative and non-perturbative formulations of Yang-Mills theories. For instance, lattice models are non-perturbative and are frequently used to investigate, e.g., QCD at strong coupling, and a crucial feature of instantons is also that they are invisible to pure perturbation theory (see e.g. this question and its answers).
At the mathematical level of rigor, both perturbative and non-perturbative approaches require establishing the formal existence of quantum fields as operator-valued distributions obeying the Wightman axioms or something mathematically equivalent to this. For instance, the usual rigorous formulation of perturbation theory is via Epstein-Glaser renormalization, which also requires establishing the Wightman axioms (in particular their causality condition) first.
Such a formal and rigorous construction is unknown in four dimensions for Yang-Mills theories like the Standard Model, and that is what the millennium problem is about, so both perturbative and non-perturbative QFT are similarly nonrigorous. The claim that the Yang-Mills millennium problem somehow means that perturbative QFT is more well-defined than non-perturbative QFT is just wrong. The claim that in many cases perturbative approaches are the only ones that are computationally tractable is correct, but this is not directly related to the problems with rigor.
"Perturbative" and "non-perturbative" are orthogonal issues to "rigorous" or "non-rigorous". We lack rigorous QFT, not non-perturbative QFT.
