It's probably reasonable to neglect height differences, but it's funny that it's not really necessary.
Indeed, if we note $P^*= P + \rho g z$ Bernoulli's theorem is written $$P^*(M)+\frac{1}{2} \rho {\vec{v} (M) }^2 = cst$$ and the law of statics is written $$P^*= cst$$
Note $A$ the lateral pressure tap and $B$ the dynamic pressure tap (where the speed is zero). In general, we consider two points $A'$ and $B'$ at the same height as $A$ and $B$ and very upstream in the flow.
We will have $ \vec{v_{A'}}= \vec{v_{B'}} = \vec{v_{A}} = \vec{v}$ and also $\vec{v_{B}}=0$
If we write twice Bernoulli's theorem: $P^*(A')+\frac{1}{2} \rho {\vec{v_{A'}}}^2 = P^*(A)+\frac{1}{2} \rho {\vec{v_{A}}}^2 $
from where $$P^*(A')=P^*(A)$$
and $P^*(B')+\frac{1}{2} \rho {\vec{v_{B'}}}^2 = P^*(B)+\frac{1}{2} \rho {\vec{v_{B}}}^2 $
from where $$P^*(B')=P^*(B)+ \frac{1}{2} \rho {\vec{v}}^2$$
But we also know that, in a direction perpendicular to a unidirectional flow, the pressure varies as in static : $$P^*(A')=P^*(B')$$
Finally we have :
$$P^*(A)-P^*(B)= \frac{1}{2} \rho {\vec{v}}^2$$
We must not forget that the manometer is connected to points $A$ and $B$ by two tubes within which the fluid is at rest. So if we note $A''$ and $B''$ the two connections on the manometer, we will have the law of statics: $P^*(A'')=P^*(A)$ and $P^*(B'')=P^*(B)$.
We will therefore have :
$$P^*(A'')-P^*(B'')= \frac{1}{2} \rho {\vec{v}}^2$$ with this time $A''$ and $B''$ the two connections of the manometer. If these two points are at the same height : $$P^*(A'')-P^*(B'')=P(A'')-P(B'')$$ and so (finally !!!)
$$P(A'')-P(B'')= \frac{1}{2} \rho {\vec{v}}^2$$
We see that the differences in height compensate each other and that what counts is the difference in height between the two entry points of the manometer.
Hope it can help and sorry for my poor english.
