I don't know much about QFT at all, but I have read some discussions of Casimir's effect in Matthew Schwartz's QFT textbook and I hope I can offer some valuable ideas about why the different regulators agree in some cases but not others.

Essentially, in Schwartz's treatment, there is a class of regulators that yield the same physical result, namely the $-\frac{1}{12r}$ term in the energy. More precisely, he derived the necessary constraints on the regulators.

Here are the details. Suppose the cutoff angular frequency in the hard-cutoff regulator is given by $\omega_{cutoff} = \pi \Lambda$ (the energy will be shown to be independent of the cutoff!). Then a general regulator $f$ turns the original sum:$$ E(r) = \sum_n \frac{\omega_n}{2} \quad \omega_n = \frac{n \pi}{r}$$

Into the sum: $$E(r) = \sum_n \frac{\omega_n}{2} f(\frac{\omega_n}{\omega_{cutoff}})$$

Using the general expression to calculate the total energy, we find two constraints that are necessary. First of all, if $$ \lim_{x ->\infty} xf(x) = 0$$ Then we end up with a nice and clean expression for the total energy: $$E_{total} = E(r) + E(L-r) = \frac{\pi}{2} \Lambda^2 L - \frac{\pi f(0)}{24r} + ...$$ If we then add the constraint that $f(0) = 1$, then the $r$ dependent energy term matches the experimental results. And the two constraints are not very strong at all. A wide variety of regulators satisfy these constraints. For example:

1. The heat kernel regulator: $f(x) = e^{-x}$

2. The Gaussian regulator: $f(x) = e^{-x^2}$

3. The Hard cut-off: $f(x) = \theta(1-x)$

For your case of $S$, you used the regulator $(1-r)^n$. Maybe you can write down more general regulators and derive some constraints to ensure consistency? That's a just a thought.

P.S. In the Casimir case, if you just started with analytic continuation on the series: $$ E(r) = \sum_n \frac{\omega_n}{2}$$ The regulator is simply $f(x) = 1$. That does not satisfy the vanishing constraint mentioned above. But a direct application of analytic continuation still gives you the correct result... this is confusing for me too. Hopefully, people who actually know QFT can add much more insightful answers.

I don't know much about QFT at all, but I have read some discussions of Casimir's effect in Matthew Schwartz's QFT textbook and I hope I can offer some valuable ideas about why the different regulators agree in some cases but not others.

Essentially, in Schwartz's treatment, there is a class of regulators that yield the same physical result, namely the $-\frac{1}{24r}$ term in the energy. More precisely, he derived the necessary constraints on the regulators.

Here are the details. Suppose the cutoff angular frequency in the hard-cutoff regulator is given by $\omega_{cutoff} = \pi \Lambda$ (the energy will be shown to be independent of the cutoff!). Then a general regulator $f$ turns the original sum:$$ E(r) = \sum_n \frac{\omega_n}{2} \quad \omega_n = \frac{n \pi}{r}$$

Into the sum: $$E(r) = \sum_n \frac{\omega_n}{2} f(\frac{\omega_n}{\omega_{cutoff}})$$

Using the general expression to calculate the total energy, we find two constraints that are necessary. First of all, if $$ \lim_{x ->\infty} xf(x) = 0$$ Then we end up with a nice and clean expression for the total energy (the detailed calculation is a straightforward application of Euler-MacLaurin Series): $$E_{total} = E(r) + E(L-r) = \frac{\pi}{2} \Lambda^2 L - \frac{\pi f(0)}{24r} + ...$$ If we then add the constraint that $f(0) = 1$, then the $r$ dependent energy term matches the experimental results. And the two constraints are not very strong at all. A wide variety of regulators satisfy these constraints. For example:

1. The heat kernel regulator: $f(x) = e^{-x}$

2. The Gaussian regulator: $f(x) = e^{-x^2}$

3. The Hard cut-off: $f(x) = \theta(1-x)$

For your case of $S$, you used the regulator $(1-r)^n$. Maybe you can write down more general regulators and derive some constraints to ensure consistency? That's a just a thought.

P.S. In the Casimir case, if you just started with analytic continuation on the series: $$ E(r) = \sum_n \frac{\omega_n}{2}$$ The regulator is simply $f(x) = 1$. That does not satisfy the vanishing constraint mentioned above. But a direct application of analytic continuation still gives you the correct result... this is confusing for me too. Hopefully, people who actually know QFT can add much more insightful answers.