which will go negative if $h>10m$. How is it possible?
It is not possible that $P_{up}$ will go negative. It will experience cavitation. This is one of those cases where you have to realize that the equation you're using is a special case of a larger theory, step back, and employ the larger theory.
If you step back from the concept of water as an absolute liquid, then it's clear that it has a state diagram. Going from some liquid place on the state diagram to $P=0$ will cross a phase transition line. Probably just once, and probably the liquid to vapor line. That means the water boils.
As you hit this threshold of $10 m$ between atmosphere and the lowest pressure point in the system, that point will turn to gas. There will possibly be some significant change in temperature, but we can imagine it happens slowly so everything is held at room temperature. So draw a line on the PT diagram of water of constant T, decreasing pressure.
I have to make some adjustments to your diagram now. Firstly, you imagined an open tube pointing down. I'll have to revise this to a tube that snakes around so that it has a defined level. Take a pipe filled with water and turn the open end down - the water is likely to fall out. We can often ignore this in small (common aquarium) pipes because of surface tension effects.
In addition to that, let's imagine the cavitated area that is now filled with water vapor. I'll, of course, assume your pipe is strong enough. Not to worry, plenty of pipes are.
There's some detail that I didn't include. But let's start making observations:
- The pump's exit is greater than atmosphere pressure, because the head above it is $>10m$
- The white areas in the pipe are filled with water vapor, in fact, saturated water vapor at room temperature
- The pressure of the vapor region will be the saturation pressure at room temperature. This is not zero, but it is very low.
As the pump pushes more liquid through the system, that liquid will overflow through the vapor region and fall down, joining the water in the outlet pipe.