The reason is because liquids don't have preferred directions.
It's true that if you squeeze a regular solid by pushing down on it, it'll push back up on your hand but it won't exert any force to the sides (though it might bulge out a bit). If you model a solid as a cubic lattice of masses connected by springs, this makes sense, because only the vertical springs get compressed. A solid has enough order to 'remember' which way you pushed on it.
A liquid doesn't have this long-range order: pushing down on a block of water just makes the water shear to the side. On a microscopic level, you can't push the atoms together only in one direction because directional correlation decays fast; all you can do is push them together in general. Then the liquid responds by pushing out in all directions too.
Your example with a bin of baseballs is in between the two cases, but I think it's closer to a solid. Most of the balls are locked in place by the weight of the balls above, making a lattice. If you pop a small hole in the side of the bin, the balls won't flow out, they're jammed.