If you hold a compass needle vertical does it point down or up differently on which hemisphere you are? Usually our compass is hold horizontally, and in the northern hemisphere it will point in the direction of to the north of the earth (actually to the South pole of the Earth's 'magnet').
But looking closer at some points, the magnetic field is not only horizontally aligned but also vertically.  So in some point of view, the direction is not the north or the south of the Earth, but in or out of the earth. But does this influence a compass the same as when it is held vertically?
 A: The north end of the compass needle is pulled down towards Earth when you hold it in the normal horizontal position in the northern hemisphere. So much so, in fact,  that the south end needs to be slightly heavier to balance it.   If you bring that same compass to Australia, the south end, already weighted, will be pulled down even further, perhap even dragging on the base of the compass. So you need to reverse the weights in any compass used in the southern hemisphere.

A chart of magnetic dip contour lines along which the dip measured at the Earth's surface is equal. These are called   isoclinic lines. 
Magnetic inclination, or dip angle, is the angle that the Earth's magnetic field makes with the horizontal.  If you hold the compass vertically in the northern hemisphere, the North Pole end should be pulled downwards. 

Image Source: How Magnets Work
A dip needle is just like a conventional compass, but instead of holding it horizontally, it is held vertically. It is a magnetic needle used for navigational purposes just like a compass, but is used predominantly when traveling around the north and south poles. Instead of measuring horizontal magnetic deflection, the dip needle measures vertical magnetic inclination. When over the equator, the magnetic field of Earth is parallel to the surface of the Earth. At the poles, or near them, a conventional compass is very unreliable.
A: A perfectly balanced compass needle (capable of rotating in any direction) would indeed point "into the earth" almost everywhere. A real field compass is actually compensated with a small imbalance of the mass so that gravity counters this desire to dip. For this reason high end compasses are sold "for a particular latitude range" - a compass that is tuned for Scandinavia would not sit horizontally in Patagonia.
This is explained on the Suunto web page: 

Any practical magnetic compass must show the horizontal component of the local magnetic field line to be precise. To accomplish this without having to manually compensate for the natural tilt in the magnetic lines, compass manufacturers usually use the weight of the needle or card to account for the vertical component of the magnetic field. This is referred to as magnetic zone balancing.

