During a lightning strike, the flashes appear as cloud to ground or cloud to cloud. Why is this the only manner of propagation? Why do the flashes not go upwards from the clouds into the sky?
They exist, they're just rarer, which is why you don't often see them. They're called gigantic jets, and they connect storm clouds to the reservoir of charge in the ionosphere. Here's one that was photographed during Tropical Storm Cristobal in 2008 (source: https://www.livescience.com/10572-gigantic-lightning-jets-shoot-clouds-space.html):
Unfortunately, not very much is known about the conditions under which they form, so there's not really a good explanation why they're so rare at the moment.
If you look at slow-motion videos of lightning strikes you'll notice that a "pilot lightning", called a step leader, originates indeed in the cloud and propagates, branching out roughly radially from its point of origin.
The general direction is certainly dictated by the gradient of the electric field while the chaotic element is introduced by the explosive expansion of the air turning into plasma.
Why does it start in the cloud and not on the ground? The general answer is that the lightning will start where the field strength is sufficient to cause an arc; presumably the concentrated charges in the cloud create an inhomogeneous electric field with a strong gradient close to the charge in the cloud, where the field strength then should be higher than in the surroundings.
This hypothesis is consistent with the observation that the step leaders actually grow in many directions, but generally radially away from the point of origin, minus some chaotic element. This direction of growth indicates the direction of the field driving it: Primarily radially away from the charge in the cloud, and only secondarily towards the ground.
Close to flat ground the electric field will be near-homogeneous because the ground is a relatively good conductor: Charges dissipate until balanced, equaling out field inhomogeneities. That's why a lightning bolt will emerge from the clouds before the field strength for an arc will be reached on the ground.
Pointy spires etc. — as opposed to flat ground — do have inhomogeneous fields at the tip so that some "positive arcing" starts there, known as St. Elmo's Fire.
I'm not sure whether a true lightning bolt could emerge from such a positive tip. Coincidentally I stumbled over a recent video which seems to show an incident where a lightning boolt indeed emerges from the tip of Toronto's CN Tower, in this Newsflare article.. There are multiple bolts in the video; the one at 10s clearly appears to originate at the top of the tower and works its way up into the sky. That's also supported by the fact that it branches out away from the tower, much like "normal" lightning branches out from a point in the cloud. This indicates the radial orientation of the electric field around the tip of the tower.
The eventual main lightning bolt which emerges once a conductive path between the separated charges has been established is not "directed"; it simply is a runaway discharge through the plasma arc created by the step leader.
According to Wikipedia, charge separation in a thunderstorm occurs as follows:
The differences in the movement of the precipitation cause collisions to occur. When the rising ice crystals collide with graupel, the ice crystals become positively charged and the graupel becomes negatively charged. ... The updraft carries the positively charged ice crystals upward toward the top of the storm cloud. The larger and denser graupel is either suspended in the middle of the thunderstorm cloud or falls toward the lower part of the storm
A lightning flash consists of multiple rapid discharge events in which current flows both from cloud to ground and vice versa.
The lightning has a visible direction of propagation, from top to down, that is normally the same as the electrons it is carrying: free electrons get accelerated (downward) by the electric field and ionize other molecules, thus progressively creating the lightning plasma pathway.
Another reason why lightnings start from top, is that air at higher altitudes is less dense, and need less voltage to initiate a spark. With a uniform electric field, a spark will always start from high altitude, and propagate downward by the accelerated electrons. Paschen Law