Without wind resistance, the speed you each at a given moment is a function of the potential energy you lost (which is equal to the kinetic energy you gained).
When you add wind friction, then the terminal velocity will be reached when the force pushing you forward is equal to the wind resistance pushing back. Now wind resistance goes as the square of velocity (approximately), and the force of gravity along the slope (the direction of motion) is proportional to $\sin\theta$ where $\theta$ is the angle of the slope ($\theta=0$ is horizontal).
It follows that the terminal velocity will be greatest in free fall ($\sin\theta=1$).
The point you made "horizontal velocity adds a component" is a red herring as the total kinetic energy can be no greater than the potential energy lost (and the work done against drag, for a given drop in height, will be greater if you have covered a greater distance to get there).
I could add more detailed equations but I don't think that's needed for the argument to hold.