I know Navier-Stokes equations rely on the continuum assumption. In this context, how is the flow velocity mathematically defined? Is it merely a spatial average of the micrscopic particles velicities inside the Representative Volume Element? Or is it a mass-weighted average, so that it results in the centre-of-mass velocity of the RVE? Or is it a even different kind of average?
Any link to a detailed description of this topic would also be appreciated
First of all, I would not necessarily call it the continuum approximation. I think that physically, "coarse grained" is a better word. Obviously, there is no system for which $\vec{v}(\vec{x})$ has meaning at arbitrarily short distances.
Also note that fluid dynamics is about the motion of conserved charges in a system close to thermal equilibrium. Quantities like $T(\vec{x}),\mu(\vec{x})$ and $\vec{v}(x)$ refer to suitable definitions of thermodynamic variables that can be used to express the conserved currents in such a system using "constitutive relations". Since the system is not in perfect equilibrium, there is some ambiguity in defining what we mean by these quantities. This ambiguity is unavoidable -- all we can ask is that the predictions of fluid dynamics do not depend on these ambiguities order by order in an expansion in gradients of $T,\mu,\vec{v}$.
Having said this, there is an essentially universally agreed upon definition of the velocity of a non-relativistic fluid (the relativistic case is more tricky). Take the total momentum in a volume element (a well defined object in any microscopic theory), and divide by the mass $$ \vec{v}_{cell}=\vec{P}_{cell}/M_{cell} $$ This leads to the constitutive relation $$ \vec{\pi} = \rho\vec{v} $$ for the momentum density of the fluid. In Navier-Stokes theory, this is the definition of $\vec{v}$, so it receives no correction in $\nabla_i v_j$. Contrast with the energy current, which receives corrections involving $\nabla_i T$.
The connection between molecular motions and Navier-Stokes takes a derivation pathway through Boltzmann kinetics to the Chapman-Enskog equation and a quick internet search turns up a paper that has the relationship you're looking for. That reference is one of many and I didn't do a rigorous search or reading of it, but it should have all the terminology and some references to get you going.
To answer your specific question, given a mass probability distribution $f(t,x,u)$ such that $f(t,x,u)dxdu$ is the amount of mass at position $x$ with velocity $u$, then flow velocity is defined as:
$$ \bar{u}(t,\vec{x}) = \frac{1}{\bar{\rho}} \iiint u_i f(t,\vec{x},\vec{u}) d\vec{u} $$
where:
$$ \bar{\rho}(t,\vec{x}) = \iiint f(t,\vec{x},\vec{u}) d\vec{u} $$
is the density of the fluid at the position in space. The integrals are over the possible velocities of the distribution, which generally are assumed to be from negative to positive infinity (and in classical derivations, there is no Lorentz effects or speed-of-light speed limit because the possibility of velocities coming close to that are vanishingly small -- that would have to change if the molecular motion could be relativistic).
You can regard it as a mass weighted average, so that it results in the centre-of-mass velocity of of a volume element. I have not seen the term Representative elementary volume used in this context however, as that appears to have a strict empirical meaning, and in the case of Navier-Stokes we are thinking in terms of much smaller volume elements than can be measured, and also we do not suppose that it is meaningful to actually take an average of the velocities of individual particles which are not individually observed.
So we don't take an average as such. We merely take an intuitive idea that there exist volume elements small enough to be treated as infinitesimal (i.e. smaller than measurable, and so smaller than the RVE), and yet large enough that random thermal motions can be neglected (as well as the individual particles entering and leaving the volume) and that it is thus meaningful to talk of the centre of mass velocity of the volume element. Then we apply Newton's laws and derive the Navier-Stokes equations.
Let's take few examples to break this down. In Kinetic gas theory the Average velocity is pretty straight forward function of the Temperature;
$v_{rms}=\sqrt{3{k_BT}/{m}}$
But as the Navier-Stokes Equations doesn't even recognize the Temperature as an Variable but still is considered to be valid in compressible fluids, it's actually quite obvious that these equations are incomplete simplification of the reality; Pressure $P,p$ alone can't describe the particle, as it's bound to some certain volume $V$ through ie. Ideal gas Law $PV=Nk_BT$
Now, we can easily define the flow though some chosen volume units flowing in their average velocity $v$ as described by the Navier-Stokes equations, and think that we could just neglect temperature related velocities inside this volume and solve the equations with this average velocity.
But neglecting this Temperature leads to a situation where locally the two particles have such a velocities that they go through phase transition (liguid->gas or Gas-plasma) causing huge trouble with the continuum assumption.
The present Navier Stokes Equations cease to work in that case, and can never produce a smooth continuous solution, as it ceases to exist. The fluid is broken.
This kinetic-gas alike aspect must be added to the equations in some statistical method, and this method can indeed be derived pretty straight forward from the bernoulli equations, if simultaniously is understood that the $>1$ values are not describing energy (impossible) but rather order/chaos in the sense that Temperature velocities are transferred to rectified fluid velocities.
The full solution for this problem is described here.
