Batteries use a type of reaction called a redox reaction that involves the transport of electrons. Rather then the carbon zinc battery, which is a bit complicated consider the simpler example of a zinc copper battery as taught in school science lessons across the world. The reaction is:
$$ Zn + Cu^{2+} \rightarrow Zn^{2+} + Cu $$
So the reaction dissolves the zinc electrode and produces copper metal at the copper electrode. The reaction goes this way because the overall free energy of the Zn/Cu system is reduced in doing so.
If we look more closely the reaction involves three steps:
$Zn \rightarrow Zn^{2+} + 2e$
transport of the electrons to the copper
$Cu^{2+} + 2e \rightarrow Cu$
So as the reaction goes electrons flow from the zinc through the battery to the copper. In effect the reaction acts as an electron pump that pumps electrons from the zinc end to the copper end. So if you connect an external wire from the copper to the zinc the electrons flow out of the copper, through the wire and back to the zinc, then complete the loop by flowing from the zinc to the copper inside the battery. Electrons flow in that direction because the chemical reaction forces them to.
The traditional zinc-carbon battery uses a reaction between zinc and manganese - the carbon is actually just an electrode and doesn't take part in the reaction. While the reaction is more complicated the basic principle is the same. The zinc reacts to form $Zn^{2+}$ and electrons and the manganese absorbs the electrons. So just like the zinc-copper battery the electron flow is driven by the chemical reaction.