Let $J$ be a certain Noether current $$ J=J[\phi] $$ where $\phi$ is a field. This object is classically conserved, although in the quantum-mechanical case it may be anomalous.
In the functional integral formalism, the failure of $J$ to be conserved is associated to a non-trivial Jacobian1. One typically finds $$ \langle \partial\cdot J\rangle=\langle A[\phi]\rangle $$ where $\langle\,\cdot\,\rangle$ denotes an expectation value, and $A[\phi]$ is the anomaly function (the trace of the logarithm of the Jacobian of the transformation).
In the operator formalism, the anomaly holds as an operator equation, $$ \partial\cdot \hat J=A[\hat \phi] $$ where $\hat J=J[\hat\phi]$.
The non-conservation of $\hat J$ is usually attributed to its singular nature2. Indeed, being a non-linear operator, the coinciding space-time points in its definition lead to an $0\times\infty$ indeterminate, and one must introduce a regulator. When the regulator is removed, one usually finds a non-zero finite contribution, which we identify with $A[\hat\phi]$.
There is a paradox here, because $A[\hat\phi]$ is usually non-linear in the fields too, and therefore it includes coinciding space-time points too. In other words, even if $\partial\cdot \hat J$ has been constructed using an explicit regulator and taking the limit carefully, a divergence remains. Take as an example the chiral anomaly, where $A\propto \hat F\wedge \hat F\cdots \wedge \hat F$. Here, $\hat F$ is a singular operator, so the anomaly $A$ is ill-defined. Still, $\partial\cdot\hat J$ was supposedly finite.
What's going on here? How is this paradox resolved? How can we make sense of the singularities in the anomaly function, when this object was precisely constructed by collecting the singularities in $\hat J$ and identifying the finite contribution as the regulator is removed?
1: See for example Weinberg's QFT, Vol.II, §22.2. See also Peskin & Shroeder, §19.1 (in particular, the discussion around equation 19.61 on page 664).
2: See for example Peskin & Shroeder, §19.1 (in particular, the discussion around equation 19.22 on page 655). See also Itzykson & Zuber, §11-5-3 (in particular, the discussion around equation 11-229 on page 559).