There is nothing mysterious about it.
Let's look at Green function
$$
\Gamma(x)
$$
which is dependent on some parameter $x$. The parameter $x$ could be anything, such as, external momentum $\vec{q}$ or the mass $m$ of a particle. Note that $x$ can not be the loop integral variable, i.e. the internal momentum $\vec{p}$. And we also assume that $x$ resides in the denominator of the integrand of $\Gamma(x)$, so that differentiating $\Gamma(x)$ with $x$ would make it more convergent.
Let's expand $\Gamma(x)$ in turn of $x$
$$
\Gamma(x) = c_0 + c_1 x + c_2x^2 + \dots
$$
Now comes the punchline: for a given divergent Green function $\Gamma(x)$, only the first few coefficients of $c_0, c_1, c_2, \dots$ are divergent, while the rest are convergent.
Let's take for example the typical logarithmically divergent $\Gamma(x)$, where only $c_0$ is divergent and $c_1, c_2, \dots$ are convergent. The usual text book way of renormalization is to introduce some regularization to make $c_0$ finite. But there is actually a slick way to avoid explicit regularization: just differentiate $\Gamma(x)$ with respect to $x$:
$$
\frac{d\Gamma(x)}{dx}= c_1 + 2c_2x + \dots
$$
Since $\frac{d\Gamma(x)}{dx}$ is finite ($c_0$ has been differentiated away!), we can proceed to calculate
$$g(x) = \frac{d\Gamma(x)}{dx}
$$
without explicit regularization. Once we have $g(x)$, we can obtain $\Gamma(x)$ via integration:
$$
\Gamma(x) = \int g(x)dx
$$
The divergence of $c_0$ is therefore bypassed: actually $c_0$ is replaced by an unknown finite integration constant, which is to be determined by experimental observation.
A variation of above approach is to calculate the finite difference (rather than differentiation $\frac{d\Gamma(x)}{dx}$):
$$
\Delta\Gamma(x) = \Gamma(x) - \Gamma(x_0)
$$
For more details on the finite difference approach, please see this PSE thread and references therein.
The renormalization group could be also formulated this way via differentiation against a parameter such as mass. See, for example, here.
