In perturbation theory for non-relativistic quantum mechanics, you begin with a Hamiltonian of the form $$H=H_0+\lambda H'$$ and assume that the perturbed eigenstates and eigenvalues can be written as power series in $\lambda$:
$$\left|\psi_n\right>=\left|\psi_n^0\right>+\lambda\left|\psi_n^1\right>+\lambda^2\left|\psi_n^2\right>+\dots;$$
$$E_n=E_n^0+\lambda E_n^1+\lambda^2 E_n^2+\dots.$$
Then you plug these expansions into the eigenvalue equation, $H\left|\psi\right>=\lambda\left|\psi\right>$, and set the coefficients of like powers of $\lambda$ equal to each other.
At least three concerns arise:
Why are we justified in assuming these Taylor expansions exist, i.e., have nonzero radii of convergence?
Why are we justified in Taylor expanding $\left|\psi\right>$, as it is an abstract vector in Hilbert space and not a scalar-valued function?
Why are we justified in setting like coefficients of $\lambda$ equal to each other?
For #3, I have seen some mathematically precise arguments for why, if $P(x)=Q(x)$ for all $x$, where $P$ and $Q$ are finite polynomials, then their coefficients must be equal. But we now have infinite Taylor series, as well as coefficients of $\lambda$ that are not scalars but rather abstract vectors in infinite-dimensional Hilbert space—should it still be obvious that we can do this?
EDIT: To clarify question 1, by "convergence" it don't just mean convergence at all, but even stronger: convergence to the correct value. There are non-analytic functions, like $f(x) = \{0$ if $x=0, e^{-1/x^2}$ otherwise$\}$ which have Taylor expansions that converge (to some value) everywhere, but to the correct value only at the single point of expansion (at zero, for this example).