Exerting Pressure I dip my finger into a container of water that is resting on a scale. Is it true that the force exerted on the scale by the container will increase because the finger creates a downward force?
 A: Yes, kind of. The scale will show something, but ultimately it will not be any downward force of your finger because that is not needed. If you apply a downward force then your finger would be accelerating downward.
When you first dip your finger into the water your finger will push down on the water a little and this force will be transmitted to the scale. However, this is a transient force arising from the inertia and viscosity of the water. Imagine slapping the water with an open hand. The impact that makes your hand hurt must also be transmitted to the scale.
Back to dipping your finger: When the water has moved out of the way and your finger is suspended motionless in the water, It will feel somewhat weightless to you (buoyancy) because your finger's density is almost identical to the density of water. The weight of the submerged portion of your finger will be transmitted to the scale. Imagine in the extreme case that the container is large enough for you to jump in all the way.  The water container's weight will increase by your weight.
If you want to consider the difference in density, or are dipping something of vastly different density, then the more accurate way to look at it is that the force registered on the scale will increase by the weight of the water that is displaced by the submerged portion of the object, regardless of its density. Here are two examples:


*

*Imagine forcing a helium balloon under water. You have to push down to keep it under water. Your downward pressure will register on the scale even though the balloon is lighter than air. The downward pressure needed will match the weight of the water that is displaced by the balloon. (The balloon will also shrink as it is compressed by water pressure. The pressure and shrinkage increase with depth, so it will be a little easier to hold it deep than to hold it shallow.)

*Now imagine dipping a heavy metal sinker into the water and holding it up with a string so that it does not touch the bottom of the container.  The force you feel on the string will be a little lighter when the sinker is submerged, and the amount it will be lighter by will match the weight of the displaced water. This is because the sinker provides its own downward force to keep it submerged, so that portion of its weight will no longer be transferred up to you via the string. The scale will register only the addition of the weight of displaced water. The rest of the weight of the sinker will be carried by you via the string and will not show on the scale.  (If the string is cut and the sinker drops to the bottom of the container, then the scale will show the entire weight of the sinker regardless of density.)
A: The surrounding water doesn't know that it is your finger that is filling the displaced space. It thinks that there is still water present there, or, more precisely, it develops a hydrostatic pressure distribution that is the same as if water was present in the submerged space occupied by your finger. This includes the pressure at the very base of the container, where the pressure is now higher (because your finger has raised the level in the container). Therefore, the reading on the scale will increase by the weight of a volume of water equal to the submerged volume of your finger.
A: It will increase as you say, though why there's a downward force from your finger is a more interesting question.
First, read the question dmckee linked and its answer.  This describes the effect of buoyancy on an object that is not connected to anything.  That basic experiment is simpler than your case, because the wooden block from the question is not attached to anything but your finger is attached to your hand, arm, etc.  Be comfortable with the base case first.
Now in that case, buoyancy causes the water to push up on the block and thus the block pushes down on the water.  We can do a free body diagram on the block and see there are exactly two forces: buoyancy pushing up and gravity pulling down.  Because the block is not moving, we know these forces are in equilibrium, so we know that buoyancy must be pushing up with a force equal to that of the weight of the block.  Thus we know that the block must push down on the water with a force equal to its weight, which would register on your scale.
In your case, the free body diagram has one additional element: the force exerted by your hand on the finger.  However, the rules are the same.  We know that the finger will displace a volume of water, so buoyancy will push up with a force equal to the weight of the displaced water.  On it's own, this is enough to show why the scale changes.  The finger will always push down on the water with a force equal to the weight of that displaced water, which will increase the force the water applies to the scale.
The more interesting question is what about the force on the finger.  Unlike the simple block case, the finger isn't at equilibrium with the water.  As it turns out, the human body is slightly less dense than water (by about 1.5%).  This means that no matter how far you push the finger into the water, the force of buoyancy will always be more than the pull of gravity.  The finger will want to rise up to the surface.  You have to add a little force with your arm to push the finger down to keep it down.  That is why your hand will have to apply a downward force.
If instead of a finger, you held a block of lead and dipped it into the water, the scale would still increase equal to the weight of the displaced water.  But now your block weighs more than that water did.  Instead of wanting to float, it wants to sink.  You will need to use your hand to pull up on the block in order to provide the forces needed to help buoyancy oppose the force of gravity on the lead block.
So while you had to push down on the finger, you have to pull up on the lead block.  But in both cases the scale will read a higher weight equal to the weight of the water displaced by the finger or block.  The results from the linked question still hold, even when you're holding into the object.
