Your question seems to come in two parts. First, it seems the use of functional determinants to represent (formally) the result of taking a path integral may be new to you. Is that so? If it is, then I would suggest reading about this idea first. It's broader than zeta regularization. Once you're comfortable with that, then I would suggest thinking about the second part of your question, which is how zeta regularization is used. It falls under the general umbrella term of mathematical regularization of formal, divergent quantities in QFT (unlike, say, Pauli-Villars, dimensional regularization, hard momentum cut-offs, and other physically motivated regularization schemes). It is closely related to the use of heat kernel techniques in QFT. I know of an entire book on the subject on Amazon, and there are many papers about it on the arXiv. The fact that it doesn't involve making any changes to the spacetime makes it very useful for applications involving spacetime with boundaries or finite extent, e.g. the Casimir effect or strings.
Edit: I have a bit of time now to expand on this result.
You know that
$$
\int_{R} dx\,\exp(-\tfrac{1}{2}a x^2) = \sqrt{2\pi/a}\,.
$$
It's easy to extend this result to
$$
\int_{R^n} d^nv\,\exp(-\tfrac{1}{2}v^i M_{ij} v^j) = (2\pi)^{n/2}/\sqrt{\det M}\,,
$$
where $M$ is an invertible $n\times n$ matrix. We can formally extend this result to the infinite dimensional case. Consider the action
$$
S[\varphi] = \tfrac{1}{2}\int dx\,\partial\varphi(x) \cdot \partial\varphi(x) = -\tfrac{1}{2} \int dx\,dy\,\varphi(y) \delta(x-y)\partial_x^2 \varphi(x)
$$
in the path integral
$$
\int [d\varphi] \exp(-S[\varphi] )\,.
$$
Since formally there is an analogy between the vectors $v^i$ and $\phi(x)$ and between the operators $M_{ij}$ and $-\delta(x-y)\partial_x^2$, we can surmise that
$$
\int [d\varphi] \exp(-S[\varphi]) = \frac{A}{\det[-\delta(x-y)\partial_x^2]^{1/2}}\,,
$$
where we now have a functional determinant. The numerator $A$ represents a divergent quantity and the functional determinant is also divergent without regularization. We can think of it as an infinite product of eigenvalues.
$$
\det [-\partial^2] = \prod_{i=1}^\infty \lambda_i\,,
$$
where we will drop the $\delta(x-y)$ because we know this operator is ``diagonal'' (i.e., local). Then, construct the $\zeta$ function associated to the operator $\mathcal{O}$ whose determinant we're computing.
$$
\zeta_{\,\mathcal{O}}(s) = \sum_{n=1}^\infty \frac{1}{\lambda_i^s}\,,
$$
where we need $\lambda_i > 0$ for all $i$ and ${\rm Re}\,s > 0$. The latter restriction can be relaxed through analytic continuation. But then
$$
\zeta_{\,\mathcal{O}}^\prime(s) = -\sum_{i=1}^\infty \frac{\ln \lambda_i}{\lambda_i^s}
$$
so that through analytic continuation we can compute $\zeta_{\,\mathcal{O}}^\prime(0) = - {\rm tr}\, \ln \mathcal{O} = -\ln \det \mathcal{O}$. For the case of two infinite parallel plates in 3-dim. space, we see that the wavenumber is quantized in the direction between the plates, but continuous in the other two directions. This leads to an interesting spectrum for the Laplacian operator. To see zeta regularization used in the computation of its functional determinant, look here.
