If I touch the "+" side of a high voltage battery, will I get shocked? Voltage is electrical potential difference between two points. When these two points are connected using a conducting material, current flows. So far so good?
Let's think of a high voltage battery (I don't know - 50V for example). My body has some electrical potential, which is, I assume, lower than the electrical potential of one of the poles of the battery. So if I touch only one of these poles, will I get shocked?
 A: 
So if I touch only one of these poles, will I get shocked?

No.
If two conductive bodies at different potentials make contact, current will flow which will attempt to equalize their potentials.
Because you are only touching one terminal of the battery, all of the charge that may flow into you will have the effect of bringing your potential closer to that of the battery terminal. (That is, none of the charge is flowing back into the battery's other terminal.
It takes surprisingly little charge to equalize your potential to that of the battery's terminal. So although a very small charge will enter or leave your body, it is not enough to give you a shock.
A: DISCLAIMER: Working with electricity and touching live voltages can be dangerous and even fatal. I'm not a trained electrician, just a physicist who knows that typically current doesn't flow unless there is a complete circuit.

The above diagram models everything we need.

*

*The voltage source (AC or DC) is modeled at left as an ideal voltage source $V$ with internal resistance $R_{\text{int}}$. The positive and negative terminals are notated.

*The human is indicated at right as resistor $R_{\text{human}}$ with right and left hands labeled. Each hand can touch the respective terminal as modeled with the two switches.

*I've notated a reference point. This point can be any point in space but it is typically conventionally taken to be the earth. We can do so here. There is a capacitor between both the voltage source and the human and earth. The charge on these capacitors, $Q=CV$ represents stray charge collected on either the voltage source or the human. The voltage across these capacitors represents the integrated electric field between either voltage supply and the reference point. That is, we should understand a static charged body to differ, in a circuit sense, from an electrically neutral body in the sense that the former will have electric fields lines running to the reference point giving the body a voltage relative to the reference. In the problem, as specified, these charges have not been specified. I believe we can take these two capacitances to be roughly equal in the simplest abstraction since the two bodies are close to each other in space.

*The connection between the left hand and the negative terminal has more structure. I include the stray capacitance $C_{\text{stray}}$ between the human and the negative terminal here. I also include a double Zener diode to represent the dielectric breakdown of air. That is, if the voltage between the left hand and the negative terminal (voltage across $C_{\text{stray}}$) becomes too large then the Zener diode breaks down and conducts current despite no physical connection being made. Technically the breakdown voltage for these diodes would be dependent on the distance between the left hand and the terminal, with the breakdown decreasing as the hand gets closer.

We can now analyze this circuit. Recall $C \propto \frac{1}{d}$, so as two objects get further apart their mutual capacitance increases.

*

*First, in the most naive abstraction (which may suffice to explain most cases) we can ignore all of the capacitances and replace them with open circuits. This is equivalent to $C\rightarrow 0$. We'll also ignore dielectric breakdown. We can even ignore the battery internal resistance if we want. In this case when the right hand touches the positive terminal no current flows because there is simply no complete circuit. In the words of Dave Jones, the resistor representing the human is just "flapping in the breeze". In this case literally nothing happens when the person touches the positive terminal.

*Let's add back in the capacitors now but let's assume the two bodies are electrically neutral. This means there is no voltage across the capacitors to the reference point and we can neglect that part of the circuit. In this case, if the right hand touches the + terminal, and we are below dielectric breakdown, then the complete circuit will include $V$, $R=R_{\text{in}} + R_{\text{human}}$ and $C_{\text{stray}}$. This is a simple $RC$ circuit with time constant $\tau = RC_{\text{stray}}$. If the circuit is DC then the human will charge up over a time of about $\tau$ to a voltage $Q$ such that the entire voltage drop $V$ occurs across $C_{\text{stray}}$. If the circuit is AC then there will be an AC voltage across $R_{\text{human}}$ given by (using a voltage divider equation)
\begin{align}
V \frac{R_{\text{human}}}{R+\frac{1}{iC_{\text{stray}}}}.
\end{align}
This voltage is negligible if the stray capacitance is very small (objects far apart) so that the capacitive impedance dominates the resistive impedance. This voltage gets larger if the capacitance is larger (objects close) so that the resistive  impedance dominates.

*Now suppose the bodies have a large relative charging between them. This means that the charge on the two capacitors to the reference point differs. In general, this means there is a voltage difference between the two bodies as well. Let's suppose the left hand approaches the negative terminal. Eventually the voltage difference will overcome the dielectric breakdown voltage of the double Zener combination and a spark will occur as current passes through the air to equilibrate the voltages on the two capacitors to the reference point. After this initial breakdown, we can say the bodies are charge equilibrated and we can neglect the reference point/earth part of the circuit again and proceed as above. The analysis is pretty much the same in the case of the right hand and the upper terminal, one would just need to include more circuit components that aren't shown above.

So to answer the OPs question, will you get a shock if you touch one terminal of a voltage supply? The answer turns out to be complicated.

*

*If there is a large charge imbalance for some reason, then yes, you will get a shock. This is true of touching any charged object (or touching any conductive object if you yourself are charged).

If the object and yourself are not charged? Then the situation is tricky and depends particularly on the magnitude of $V$ and the resistances and capacitances.

*

*My feeling is that, for DC voltages that you'll find in every day life, the transient charge reconfiguration from touching a terminal will not be perceptible to the human touch. There may be a large current, but it will be for a very short time $\tau$ such that the total charge and energy flow etc. is not significant. Perhaps if you were working with really high, like 100 kV level voltages, you would get into dangerous territory even from the current due to this charge redistribution due to the stray capacitance between you and the negative terminal.

*For AC voltages there will always be some voltage across the human which again depends on the magnitudes of the different components. My feeling is that the experience @Ján Lalinský described in one of their comments in another answer was due to this AC voltage, which, again, arises because of stray capacitance between the left hand and the negative terminal of the voltage supply.

A: No, you won't get shocked by one pole of a battery, not even if you are grounded. This is because even though your body is conductive and connected (usually with some non-zero resistance) to the ground, touching only one pole will cause only transient, very quick and very small redistribution of electric charge to get the battery pole at the same potential your body and the ground is, and then current stops. This current is short and small because capacitance of the battery pole is small - a very small charge transport is enough to establish the equilibrium potential distribution where both the pole and your body have the same potential.
Same analysis holds true if you touch only one terminal of a standard DC 30 Volt lab voltage source. Of course, verifying this can be dangerous, if something in the source is wrong. But healthy, well built voltage source can't push harmful current into you if you only touch one terminal adn the other terminal is well isolated.
With AC, things are little different. You can get shocked by touching one terminal of AC voltage source, depending on its "strength". In some special cases you don't, like when touching only one terminal of secondary winding (low voltage side) of a small 50/60Hz transformer, but do not rely on this in general. The general rule is AC will shock you and can kill you. This is because with AC, most sources out there are strong, and with AC there is no static equilibrium so potential of the thing you are touching will likely be oscillating in time with dangerous amplitude.
A: In my opinion a lot of these answers are overly complicated (I'm an EE).
Is the negative if the battery connected to a ground, or a metal frame, like it is in a car? Is the negative hooked up to nothing? If so, then touching the positive terminal will not shock you, even if you are touching a ground plane. There's no current path to the neutral of the battery.
If there is ANY possibility of a current path back to the negative terminal of the battery then you may get shocked. Same for AC. It gets a little more complicated there due to the splitting of neutral/ground, which is typically bonded at your service panel, but the concept is exactly the same.
A: You can look at the terminals as providing pressure to make a current flow. If this pressure is not enough (and the battery can deliver a steady supply of electrons), there will be no significant flow of electrons be produced in your skin, as the skin's resistance lies between 1, when wet, and 100 (kOhm), when dry. So the skin protects you from electrons flowing inside your body, because the resistance of your inner body is about 500(Ohm).
If the voltage of the pole goes up the pressure will be increased and the current produced in the skin will be higher. The flow of current is short though. It will stop when the voltage on your skin is the same as that of the pole you touch. A current of 10(mA) will cause a notable shock, but the effect is gone if the current has flown. That's why it's called a shock. Once you've made contact no further damage will be done.
The electrons are pushed into (or pulled from, on the other terminal) your skin, after which the skin polarizes to stop the inflow of electrons.
If the 220(V) AC power supply at our homes were a DC supply, the effect would be very different if you touched a wire of the supply. You would feel a momentary flow of electricity. A true shock. If you touch the 220 AC wire, the current will go to and fro, inducing an AC in your body. When you keep in touch with the wire, the "shock" will be continuous. It doesn't stop.
So touching a 220(V) AC current is much more dangerous than touching the pole of a battery that provides a 220 voltage. There will flow a small initial current that is reduced to zero. It doesn't flow continuously because your body develops gets polarized and this will stop the flow. The current induced by an AC supply will flow continuously to and fro (the polarization of your body following behind, so it can't stop the current from flowing), which can burn the skin or stop your heart, depending on where the wire touches your body.
A burned skin will reduce its resistance, thereby increasing the current inside your body. That's why no batteries are used for the electric chair. It could do the job, but then the battery has to provide a very high voltage. The shock will be indeed a shock as opposed to the continuous shocking of a person being electrocuted.
If you put on plastic shoes, the current will not be able to flow because the resistance will be too big. You will feel no shock if you touch the pole of a battery. Your body gets polarized only (causing no harm).
Note that if the battery has very small power, there can't be many electrons flowing. The push (pressure, voltage) is there but there aren't many electrons to push. So touching a 100 000 (V) pole can be done safely as long as the current that flows, as a result, is small.
Then why is your tongue continuously tingling when it touches a pole of a 9(V) battery? Firstly, the tongue is wet, so the current will be higher. You are moving the pole too, and this causes the tingling. If the pole was completely fixed on your tongue, no tingling would be felt. But this aside.
