The term you are asking about does come from the $\frac{1}{4}\dot{x}^\mu\dot{x}_\mu$ term of the Lagrangian.
Start by inserting the first of Eqs.(2.23) back into Eq.(2.22) to obtain the expression for $\dot{x}(\lambda)$:
$$
\dot{x}(\lambda) = e^{2e{\bf F}\lambda}\dot{x}(0) = e^{2e{\bf F}\lambda}\frac{1}{e^{2e{\bf F}s} - 1} 2e{\bf F}\; (x' - x") = \frac{2e{\bf F}\; e^{2e{\bf F \lambda}} }{e^{2e{\bf F}s} - 1} \; (x' - x")
$$
Now notice that the equation of motion
$$
\ddot{x}^\mu(\lambda) = 2e {\bf F}^{\mu\nu} \dot{x}_\nu(\lambda)
$$
implies
$$
\ddot{x}^\mu(\lambda)\dot{x}_\mu(\lambda) = 2e\; \dot{x}_\mu(\lambda) {\bf F}^{\mu\nu} \dot{x}_\nu(\lambda) = 0
$$
and
$$
\frac{d}{d\lambda}\left[ \dot{x}_\mu(\lambda)\dot{x}^\mu(\lambda) \right] = 2\; \ddot{x}^\mu(\lambda)\dot{x}_\mu(\lambda) = 0
$$
Hence we have
$$
\int_0^s{d\lambda \;\dot{x}_\mu(\lambda)\dot{x}^\mu(\lambda) } = s\; \dot{x}_\mu(0)\dot{x}^\mu(0)
$$
But from the expression for $\dot{x}(\lambda)$ above we have
$$
\dot{x}(0) = \frac{2e{\bf F}}{e^{2e{\bf F}s} - 1} \; (x' - x") = (x' - x") \frac{2e{\bf F}e^{2e{\bf F}s}}{e^{2e{\bf F}s} - 1}
$$
where the last form on the right hand side follows from $F^{\mu\nu} (x'-x")_\nu = - (x'-x")_\nu F^{\nu\mu}$. With this the action integral term becomes
$$
\int_0^s{d\lambda \;\dot{x}_\mu(\lambda)\dot{x}^\mu(\lambda) } = s\;(x' - x") \frac{2e{\bf F}e^{2e{\bf F}s}}{e^{2e{\bf F}s} - 1}\frac{2e{\bf F}}{e^{2e{\bf F}s} - 1} \; (x' - x") =\\
= (x' - x") e{\bf F}s \frac{4e{\bf F}e^{2e{\bf F}s}}{\left(e^{2e{\bf F}s} - 1\right)^2}\; (x' - x") = (x' - x") e{\bf F}s \frac{e{\bf F}}{\sinh^2(e{\bf F}s)}\; (x' - x") = \\
= - (x' - x")\; (e{\bf F}s) \frac{d}{d s}\coth(e{\bf F}s)\; (x' - x")
$$
Assuming $x'$ and $x"$ are fixed, the last expression can be rearranged to obtain
$$
\int_0^s{d\lambda \;\dot{x}_\mu(\lambda)\dot{x}^\mu(\lambda) } = (x' - x")\; e{\bf F} \coth(e{\bf F}s)\; (x' - x") - \frac{d}{d s} \left[(x' - x")\; (e{\bf F}s) \coth(e{\bf F}s)\; (x' - x") \right]
$$
The first term is the one we are looking for. The second one is not only a total time derivative, but can also be rewritten as the integral of a total time derivative, since
$$
\int_0^s{d\lambda \frac{d^2}{d \lambda^2} \left[(x' - x")\; (e{\bf F}\lambda) \coth(e{\bf F}\lambda)\; (x' - x") \right] } = \\
= \frac{d}{d s} \left[(x' - x")\; (e{\bf F}s) \coth(e{\bf F}s)\; (x' - x") \right] - \lim_{\lambda \rightarrow 0} \frac{d}{d \lambda} \left[(x' - x")\; (e{\bf F}\lambda) \coth(e{\bf F}\lambda)\; (x' - x") \right]
$$
and
$$
\lim_{\lambda \rightarrow 0} \frac{d}{d \lambda} \left[(x' - x")\; (e{\bf F}\lambda) \coth(e{\bf F}\lambda)\; (x' - x") \right] = 0
$$
But a term of the form $\int_0^s{d\lambda \frac{d^2}{d \lambda^2} \left[(x' - x")\; (e{\bf F}\lambda) \coth(e{\bf F}\lambda)\; (x' - x") \right] }$ would only add a total time derivative to the Lagrangian, so it can be safely discarded and the final result is
$$
\int_0^s{d\lambda \;\dot{x}_\mu(\lambda)\dot{x}^\mu(\lambda) } = (x' - x")\; e{\bf F} \coth(e{\bf F}s)\; (x' - x")
$$
