Sound Pressure Level
I think what you're looking for isn't loudness in the technical sense of the word. Loudness is the subjective perception of sound pressure.
What you may want instead is the sound pressure level (SPL):
$$
\text{SPL}=20\log_{10}\left(\frac{p_\text{rms}}{p_0}\right) \ \ \ \ \text{or just}\ \ \ \ 10\log_{10}\left(\dfrac{\text{avg}(p^2)}{p_0}\right)
$$
Where $p_0$ is your reference, since the decibel $dB$ is a unitless measurement expressing the logarithmic representation of a ratio.
The lower limit of human hearing is typically taken to be $20μPa$ at $1\text{KHz}$.
Actual Loudness
In audio mixing the LUFS unit is typically used. See EBU R128 and it's referenced ITU-R BS.1770 which actually defines the loudness unit.
It first runs the audio through a couple filters to match the human perception of loudness across frequencies, and then it essentially applies the $dB_{SPL}$ formula to it, but instead of being relative to $20μPa$ it's relative to the loudest sound that can be digitally stored in the format you're using. My question about that very unit has more information (I also wrote an answer in which I attempt to document it with more detail).
For a continuous audio file, loudness is measured by keeping a rolling windowed average, gated to exclude sounds either below an absolute $-70$ LUFS or below $-10$ LUFS relative to the current average value (as you're running through the file).
You can use ffmpeg's loudnorm filter to determine this loudness parameter, as well as a few statistically relevant others for any given file. It will output as json.
But as I said, this unit is relative to the loudest sound that can be stored digitally on file, not any real-world reference which I assume is what you're looking for. For that you'd have to record a reference of your own and then use that as an offset to further LUFS measurements.
This is pointless if your recording setup automatically amplifies/attenuates the input signal (like phones trying to keep a constant volume when you're on a call in loudspeaker). Though this is true for the nature of your task so I imagine you've already taken measures to avoid it.
One last note is how the filters used by this LUFS unit are computationally efficient approximations that do not attempt to faithfully reproduce the Equal-Loudness contour to a precise degree. If you need a very accurate solution, you will have to find some other solution with either higher-order filters or an FFT, as well as probably keep a more faithful distribution of the calculated loudness throughout the file (the rolling windowed average is really good but scientific/research work might require more granular information).