We agree with OP that Ref. 1 does not explain the averaging procedure $\langle \cdots \rangle$ adequately. For reference, the relevant section of Ref. 1 reads:
One way of circumventing this problem is to take seriously the fact that the energy–momentum of a gravitational field at a point in spacetime has no real meaning in general relativity, since at any particular event one can always transform to a free-falling frame in which gravitational effects disappear. This suggests that, at each point in spacetime, one should average $G^{(2)}_{\mu\nu}$ over a small region in order to probe the physical curvature
of the spacetime, which gives a gauge-invariant measure of the gravitational
field strength. Denoting this averaging process by $\langle \cdots \rangle$, one should thus replace (17.55) by
$$\tag{17.57} t_{\mu\nu}~\equiv~\frac{c^4}{8\pi G} \langle G^{(2)}_{\mu\nu}\rangle.$$
Having made this identification, our task is now an algebraic one of determining the form of $\langle G^{(2)}_{\mu\nu}\rangle$ as a function of $h_{\mu\nu}$. This is rather a cumbersome calculation, but the job is made somewhat easier by averaging over small spacetime regions. Since we are averaging over all directions at each point, first derivatives average to zero. Thus, for any function of position $a(x)$, we have $\langle \partial_{\mu}a(x)\rangle$.
Here the superscript $(2)$ refers to terms that are second-order in $h_{\mu\nu}$.
The averaging procedure is explained in greater detail in Ref. 2 as part of the shortwave approximation/formalism in the limit of small typical wave amplitude $A\ll 1$ and typical wavelength $\lambda \ll R$ much smaller than the typical radius $R$ of curvature.
This is essentially a Wilsonian effective theory, where one integrates out UV modes to be left with IR modes. However, one should keep in mind that Ref. 2 are considering the UV modes as classical configurations (as opposed to quantum fluctuations), so rather than integrating out in path integral sense, Ref. 2 is averaging out.
Technically, the averaging procedure $\langle \ldots \rangle$ in Ref. 2. is called the Brill-Hartle average.
To convey the main idea in an oversimplified manner, the average procedure $\langle \ldots \rangle$ is taken over several wavelengths of a plane gravitational wave of the form
$$\tag{1} a(x)~=~f\left(c(x)+A\sin(k\cdot x)\right),$$
where the IR part $c(x)$ depends so slowly on position $x$, that it can be treated as a constant in the average procedure, and the derivative $\partial_{\mu}c(x)$ is negligible. So we are essentially just averaging the first derivative
$$\tag{2} \partial_{\mu}a(x)~=~f^{\prime}\left(c(x)+A\sin(k\cdot x)\right)~\partial_{\mu}\left(c(x)+A\sin(k\cdot x)\right)~\approx~f^{\prime}\left(c(x)+A\sin(k\cdot x)\right)~k_{\mu}A\cos(k\cdot x)$$
over several wavelengths and getting zero. The detailed form $(1)$ of $a(x)$ does not matter as long as it is approximately periodic.
References:
M.P. Hobson, G.P. Efstathiou, and A.N. Lasenby, General Relativity: An Introduction for Physicists, 2005, p. 488.
C.W. Misner, K.S. Thorne, and J.A. Wheeler, Gravitation, 1973, Section 35.13-35.15, p. 964-973.