Why doesn't an initial velocity affect the final velocity? I've been stuck on a basic kinematics problem for some time, which usually goes something like this.

Three balls are dropped from some height, $h$, with initial velocity $|v|$. Ball #$1$ is given $v_i$ upwards, ball #$2$ is given $v_i$ horizontally, and ball #3 is given $v_i$ downwards. Rank the balls according to the magnitude of their final velocity.

The answer, of course, is that each ball has the same final velocity. I understand why this is, and how to obtain the result (we have conservation of mechanical energy... since each ball has the same $KE_i$, each ball must have the same $KE_f$, which implies each ball has the same $|v_f|$), however I find it hard to wrap my head around why, intuitively, balls #$1$ & #$3$ don't have larger final velocities.
I know this is wrong, however my intuition tells me:
For ball #$1$, I can imagine the ball with some initial upward velocity, eventually reaching some height where $v=0$, then falling from a larger height than ball #$2$, implying more time accelerating and a larger velocity on impact.
For ball #$3$, it seems incredibly odd that an initial downward velocity doesn't lead to a larger final velocity. Is it just that the decrease in time spent falling (due to $v_i$, i.e. less time accelerating) exactly compensates for $v_i$?
I think it's fascinating that this sort of thing goes completely against our intuition, however I've struggled for some time trying to visualize this simple result without resorting to the black box of equations. Is there another way to look at this?
 A: I think you are simply forgetting the sideways speed of #2. Case #2 will indeed have a smaller vertical final velocity component of the reasons you describe. Your logic is fine. Now add it to the sideways component and the magnitude will turn out the same as the others.
A: I think the confusion here is by not focusing on the scalar nature of Energy.  You have everything right for the vertical component, but Kinetic Energy doesn't care about the direction of the velocity, just its magnitude, which is the same in all cases.
As a side note, you logic is right for ball #3 vs ball #1.  By the time ball #1 reaches $h$ again, it will have the same downward energy as ball #1.
A: On its way down, Ball #1 will visit the point with height $h$ again with same absolute velocity $|v_i|$, which form then one is clearly equivalent to Ball #3. Ball #1 just took extra time for the initial detour.
Ball #2 case is different. If you focus on the vertical component of the velocity only, then it indeed would be less than $v_f$. You need to consider both the horizontal and the vertical components.
