It all boils down to formally manipulating matrices with "continuous indices". The price to pay for these formal manipulations will be the need for regularization in the end.
If you have a diagonal matrix $M = \operatorname{diag}(\lambda_1,\dots,\lambda_n)$ and you have a function $f(x)$ you can define $f(M)$ to be the diagonal matrix $f(M)=\operatorname{diag}(f(\lambda_1),\dots, f(\lambda_n))$. This straightforwardly generalizes to operators and should be familar from Quantum Mechanics. Indeed, if ${\cal O}$ is an operator with basis of eigenvectors $|\lambda\rangle$ with $${\cal O}|\lambda\rangle=\lambda |\lambda\rangle,\tag{1}$$
we define $f({\cal O})$ by its action on this basis $$f({\cal O})|\lambda\rangle = f(\lambda)|\lambda\rangle,\tag{2}$$
and extension by linearity. All that said, observe that $J(x,y)$ given in your equation (77.17) can be thought of as a matrix whose entries are indexed by two continuous labels instead of the familiar discrete indices labeling rows and columns. In particular, if you see it like that, you will see that $\delta(x-y)$ is the continuous version of the identity matrix and therefore $J(x,y)$ is a diagonal matrix.
This means that $\log J$ will be the continuous matrix with entries $[\log J](x,y)$ given by $$[\log J](x,y) = \delta^{(4)}(x-y)\log \exp[-i\alpha(x)\gamma_5]=-\delta^{(4)}(x-y) i\alpha(x)\gamma_5\tag{3}.$$
With this result known use your un-numbered equation to write $$\det J = \exp[\operatorname{Tr}\log J]\Longrightarrow (\det J)^{-2} = \exp[-2 \operatorname{Tr}\log J]\tag{4}.$$
To finish, how do you take the trace of a matrix $M_{ij}$? Well, you basically sum the diagonal entries, i.e., we have $\operatorname{Tr} M = \sum_{i=1}^n M_{ii}$. Since you know the diagonal matrix $\log J$ from (3) you can construct the continuous version of the trace by integrating over $x$.
In other words $$\operatorname{Tr}\log J = \int d^4x\ \operatorname{Tr}[\log J](x,x) = -i\int d^4x\ \alpha(x) \operatorname{Tr}\delta^{(4)}(x-x)\gamma_5\tag{5}.$$
Observe that a trace remains on the integral because of the (finite) matrix that is there. Plugging this into (4) you find (77.19)
$$(\det J)^{-2} = \exp\left[2i\int d^4x\ \alpha(x)\operatorname{Tr}\delta^{(4)}(x-x) \gamma_5\right]\tag{7}$$