Intuition behind hydrostatic force/pressure being applied laterally 
I can understand how the downward vertical force is being applied: due to the weight of the water column directly above it. However, I cannot understand how the lateral and the upward forces act. How can I understand these forces intuitively?
 A: This does look strange from the described point of view: the force from above can easily be viewed as due to the mass of water column above the object, while forces from other sides seems to correspond to no such obvious sources.
Actually, all the four forces depicted in the figure act in one way: through constant bombardment of molecules on the surface of the object.
On the microscopic level, the waters are made of water molecules, and they are all constantly moving, rattling, colliding with each other. The pressure on the object is due to the many many moving molecules hitting on the surface and bounce off. Namely, it is an averaging effect, though due to the enormously large number of water molecules, it just appears as a static pressure on the macroscopic level.
Note this understanding does also apply to the force from above. The object's surface has direct contact with only the molecules in its immediate vicinity and does not know the column of water above it.
In fact, it is only lucky that in the picture depicted in the question one can get correctly the pressure from above by considering it as due to the water column above. If however, for example, the object is in the center bottom of a U shaped tube that is filled with water, the water physically right above the object could be very little, but actually the pressure can be very high due to the high water levels at the water surface at the two ends of the tube.
A: In the left diagram the piston is held with the spring at natural length. It's then released and settles down to the diagram on the right.  The spring is then under compression and shows that there is a sideways force due to the  weight of the column of red liquid.

Similarly here

Again the spring becomes compressed and shows that there is a force acting upwards on the piston, even though there is no liquid above it.
A: *

*Consider the water molecules (or other very tiny chunks of water).


*Think of pressure not as the water molecules pushing on things, but as them being pushed on, having forces on them - coming from every angle. Each molecule looks like a ball with forces pushing on it from all six directions.


*Imagine for intuition that every water molecule is not moving*, so THERE IS NO NET FORCE ON A WATER MOLECULE. Sum of the forces is zero.


*Think of the object as stacked on the pile of water molecules below it. The molecules have to push back or be moved by the other forces on them. You could say, “No they could move away and let it sink.” which happens if denser than water. But the molecules beside that water under it... they have forces on them too, so just moving out of the way can’t happen**.
——————————————
Subscript Notes:
*The water may be churning about slightly, but none of the water molecules are moved consistently in one direction and out of the way to a different spot leaving an empty space. There is no net force moving chunks of water around, to other spots. So the intuition is still valid.
**Why do the horizontal forces on our molecule equal the vertical forces? This is because it’s a liquid, like a pile of infinitesimal pieces of sand, with each particle: 1. perfectly smooth (water friction resists only under net velocity, which is viscosity, but statically it’s the intuitive equivalent of perfectly smooth) and 2. not stacked perfectly on top of the others creating a variety of force vectors, and experiencing electrical attraction and repulsion as dipoles. The only way to avoid net motion is all forces on it, in all directions, being equal on average.
*** Sum of the forces, including gravity, is zero on each molecule. Means force from below is tiny bit more than from above, so pressure increases continuously with depth.
A: Imagine a little cube of fluid just next to the submerged object you've drawn. As you surmise, the weight of all the fluid in a column directly above that little cube presses down on top of the cube. By Newton's third law (equal and opposite reactions) the little cube pushes up equally hard on the column--that's why the column doesn't fall downward.
In spite of the force pressing down on its top, the cube does not accelerate: clearly there is a counterbalancing upward force  on the cube.  This makes sense: if you append the cube onto the column, and consider a new cube just below it, the picture is just the same, but with the weight of the column increased by one cube's worth.  The original cube experiences an upward force on its bottom that is equal in magnitude to its weight plus the downward force on its top.  So much for an intuitive explanation for the upward force. What about the sideways ones?
The fluid is static--the cube is not being squashed into a pancake even though it has no rigidity and there are forces pressing downward on its top and upward on its bottom.  There must be sideways forces present to prevent this squashing.  Where do these forces come from? Well, the four neighboring cubes which sit adjacent to the four vertical faces of our original cube also want to squash out into pancakes due to the weight of the columns of fluid sitting on their heads.  They end up pressing against each other and the end result is that there's just enough horizontal force that none of them squashes.
A: Here, perform a simple experiment:

*

*Take a small can.

*Suck out the air from it.

*Place the can in a container filled with water.

What you should find (if the conditions are favourable, that is to say, if the thickness of the can is small enough) is that the can will be crushed, from all directions, including laterally.
This clearly shows that a lateral pair of force is being applied on the container by the bottle.
Now usually, we don't encounter stuff with less air pressure inside the water right? That's why this lateral force feels so unintuitive to you. It is always there, but the two opposite components of force usually cancel each other out, except in cases when there's a considerable vacuum/hollow in between them.
The lateral force is simply a resultant of the constant bombardments of the water molecules against the surface of the object.
For the upward component, that can be thought of as a resistance by the water molecules against the downward force (weight) of the object and the column of water above it. The molecules are resisting a force which wants to cause their displacement.
Hope this answers your question!
