Lambertian phase function is,
$$ \Phi(\theta) = \cos(\theta) $$
This means at normal $\theta=0$ maximum intensity is reflected form the cloth surface, which falls of to zero when $\theta=\pi/2$.
Reflection from the surface is diffuse, meaning that a beam of light entering at a constant angle, will be reflected into all angles and will be distributed according to the phase function.
This requires integrating over all angles to calculate to total intensity reflected.
The integral in the numerator looks like this,
The integrals adds up all possible directions of reflected light outside the cone.
$$ \int \Phi(\theta) d\Omega $$
Element of solid angle is defined as
$$ d\Omega = \sin(\theta)d\theta d\phi $$
Substituting in and including the correct limits,
$$ \int_0^{2\pi} \int_{\theta_c}^{\pi/2} \cos(\theta)\sin(\theta) d\theta d\phi $$
We can integrate directly in $\phi$,
$$ 2\pi \int_{\theta_c}^{\pi/2} \cos(\theta)\sin(\theta) d\theta $$
This is what is written above.
The second integrals is just adding up all possible reflected directions in the hemisphere; it’s used to normalise the the reflected value to give a probability.