What exactly is a hyperfine level as mentioned in the definition of a second? I have a quite simple question: What exactly is a hyperfine level (or a hyperfine structure)?
Maybe my question sounds (or is) stupid $-$ I am not an expert in physics $-$ but I didn't find any description in the Internet which I understood. (Or more precisely: most of the descriptions said that it has something to do with the interactions of the electrons with the nuclear spin - but either they are not more precise or they are too technical for me.)
What I know is, that there are energy levels of electrons in the atoms. (Something like electron shells, or better: orbitals which represent energy levels or frequencies or so.) I also know that there are four quantum numbers which the spin being one of them.
Are the hyperfine levels somehow related to this Pauli principle?
I.e. do electrons in a specific orbital just change their spin due to magnetic influence or resonance or whatever? If so, does an electron with one spin "direction" have more energy than an electron with the opposite spin direction? (For me, that doesn't make sense...)
Or does an electron change its orbital or shell in this process? (But what does that nuclear spin mean here, then? And why do they then talk about hyperfine structure? Because it sounds better?)
 A: This is a bit of toy model, but it will likely answer your question.
All of this comes originally from spectroscopy. When you heat a substance it begins to give off light. When you pass the resulting light through a prism, you see a series of thin lines with a characteristic distribution that's unique to that element.
It wasn't known at the time, of course, but this was due to the energy levels you refer to in your question. Specifically, each of the lines in the spectrum corresponds to the light given off when an electron transitions between two of the possible levels. Since those levels are defined largely by the structure of the nucleus, each element has its own structure.
A wrinkle was noted almost from the start: when the substance was placed in a strong magnetic field, the spectrum changed - subtly. The overall structure was the same, but now where there used to be one line there were two, very closely spaced. The spectrum now had a "fine structure". Over time we learned that this was due to the effects of the magnetic field on the energy levels, causing there to be two possible energy states depending on the spin of the electrons. There's two because there's two spins.
The hyperfine structure is similar in concept, but due to a different mechanism. In this case it is not an external field that causes the lines to split, but the internal state of the nucleus itself. Depending on the energy state of the nucleus, the internal components take on different "arrangements" (for lack of a better term) and this causes slight changes in the allowed energy levels for the electrons. In comparison to the fine structure's two levels, hyperfine had four levels, and required the introduction of a quadrupole "spin" to explain it.
As others have stated, this is a very weak effect compared to the fine structure, which is one of the reasons it took Michaelson to notice it (the guy was a god of spectroscopy).
Now I'll bet the original thing that prompted your question was this:

The second is the duration of 9 192 631 770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the cesium 133 atom.

So what they are saying is this: at the lowest possible electron energy level in cesium 133, the "ground state", you don't actually have a single line, but two VERY closely spaced ones. When an electron moves from one to the other it will give off a very specific frequency of light.
Now you could probably do the same thing with any other two levels, but those are too easy to change. For instance, if you defined the second based on two normal levels of the cesium 133 atom, say the ground and first excited state, well then the frequency changes if there's a magnet nearby. By defining the standard on the hyperfine levels, they're using a state that is much more resistant to environmental effects.
A: 
What I know is, that there are energy levels of electrons in the atoms. (Something like electron shells, or better: orbitals which represent energy levels or frequencies or so.) I also know that there are four quantum numbers which the spin being one of them.

Yep, that is all correct. In quantum mechanics, the energy levels of confined systems are generally quantized to discrete energy levels. In addition to that, angular momentum comes in 'chunks' of size $\hbar$ (or possibly $\hbar/2$), with the electron spin being one such example.
Caesium is a big atom, with 55 electrons, but when the atom is in the electronic ground state, almost all of those electrons are contained in full shells in which each electron has an orbital angular momentum and a spin which are exactly cancelled out by another electron in that same shell. That's almost all electrons, though: caesium, being in Group I of the periodic table, has a single electron sitting in an $s$ orbital, with zero orbital angular momentum (i.e. the electron shell's orbital is spherically symmetric) and spin-1/2 electron spin.
Now, here's the thing:

*

*The caesium-133 atom has a nucleus, and that nucleus also has spin (which, as it happens is spin $7/2\ \hbar$, i.e. bigger than the electron spin).


*Moreover, the spins of both the nucleus and the electron are associated with magnetic dipole moments which point in the same direction as the spins. Basically, the both act like tiny bar magnets, attracting and repelling each other depending on their relative orientation, and those forces also come with an interaction energy which depends on their relative orientation.


*And, to top it off, because of the weird ways in which angular momentum is handled in quantum mechanics, the relative orientation of the two spins is strongly constrained:

*

*they can be aligned parallel to each other, giving a total angular momentum of $4\hbar$,

*or they can be aligned anti-parallel to each other, so the electron spin 'cancels out' part of the nuclear spin to give a total angular momentum of $3\hbar$,

but that's it.


*Finally, since those two states have different relative orientations between the spins (and their associated magnetic dipole moments), the magnetic interaction energy is different between the two levels.
The SI second is tuned to this energy difference: the photon energy $h\nu$ of a radiofrequency photon at a frequency of exactly $\nu=9,192,631,770 \:\rm Hz$ is precisely resonant with the energy difference between those two energy levels, and absorbing that photon will take the electron spin from being anti-parallel to the nuclear spin and flip it so that it points in the same direction as the nucleus.

Finally, as for the term "hyperfine structure":

*

*We use the term "structure" because these small energy differences between levels which are very close to each other will most often manifest themselves in optical transitions, so that if you are e.g. exciting a caesium atom from the ground state by moving the lone $6s$ electron to a $7p$ orbital, and you have a very stable laser and a clean experiment, then the line will resolve into two individual components which correspond to the two energy levels described above. (With additional complications coming e.g. from the multiple spin orientations of the $7p$ orbital, but you get the picture.) Basically, the line acquires structure, which is where the term comes from.


*The term "hyperfine structure" is there to distinguish the mechanism from that of fine structure, which is a similar separation of orbitals into sub-levels caused by relativistic effects as well as magnetic interactions between the electron spin and the orbital angular momentum of the state.
For almost all atoms, those interactions are stronger than the magnetic interactions with the nucleus, so that the optical transition will first split into components caused by the relativistic effects (hence "fine structure") and, if you look extremely closely and with a very stable laser, each of those fine-structure components will split into sub-lines that are even closer together (hence "hyperfine structure").
A: First, you mention electron spin. The fine structure defines the spectral splitting of atoms due to relativistic effects that correct the non-relativistic Shrodinger equations.
the structure of line spectra is defined by the non-relativistic QM of non-relativistic electrons with no spin. But a more accurate form is when you take into account the relativistic and spin effects. The scale of the fine structure splitting levels compared to the hole energy levels is (Z*a)^2, where Z is the atomic number and a=1/137.
This takes into effect:


*

*Kinetic energy

*spin-orbit coupling

*zitterbewegung of electron
In the case of fine structure, the splitting is because of the magnetic moments caused of magnetic moments because of electrons spin and orbital angular momentum.

Now you are asking about hyperfine structure. This effect is of orders of magnitude smaller then fine structure splitting, is because of the interactions of the nucleus with internally generated magnetic and electric fields.
The theory comes from electromagnetism, consisting of the interactions of the nulear multipole moments (excluding electric monopole) with internally generated fields.
So it consists of:


*

*magnetic dipole: electron orbital angular momentum, electron spin, Fermi contact

*electric quadropole
Now due to the accuracy of the hyperfine structure atomic clocks, the definition of the second is given as 9,192,631,770 cycles of the hyperfine structure transition frequency of caesium-133 atoms.
A: In addition to all the other very good answers here, I would like to clarify why the spin changes the energy level of an electron.
Magnetic Moment
The electron spin corresponds to a magnetic moment - in a very simplified (and inaccurate!) picture, the electron can be seen as spinning. It behaves as a spinning, charged ball, which in turn means that there is a charge density moving: that's a circulating current! This produces a magnetic field perpendicular to the plane in which the current flows.

The magnetic field of a small current loop looks like that of a small bar magnet. Imagine placing a bar magnet inside of an external, homogeneous magnetic field. If there is an angle between the direction of the magnetic moment of the magnet and the field lines of the external field, there will be a torque acting on the magnet. This torque rotates the magnet until the lines are aligned. This reduces the energy of the system.
Why? It's analogous to the case of a ball rolling down a hill. The potential energy of the ball on top of the hill is higher than at the bottom. Similarly, the magnetic moment has a higher potential energy if it is not aligned with the external field lines.

Gross Structure
The states of the electrons in an atom are defined by the quantum numbers $n, m, l, s$. The Schrödinger equation delivers an expression for the energy of an electron with quantum number $n$ in an atom with proton number $Z$: $$E_n = -\frac{Z^2}{n^2} E_R$$ where $E_R$ is the Rydberg energy (13.6 eV).
Fine Structure
This is all nice, but real atoms do not behave much like this simplified model - it does not take into account that electrons and protons are electrically charged, and that all constituents have a spin (neutrons, protons and electrons are all fermions with spin $1/2$).
A first correction is to not use the Schrödinger equation, but the Dirac equation. It treats particles with spin and also takes into account relativistic effects. Here it turns out that most levels from the gross structure split into two. This has mainly to do with the fact that their spin $\vec S$ and orbital angular momentum $\vec L$ (from the movement around the nucleus) are coupled and allow only certain energy values.
Hyperfine Structure
Finally, the electron spin also interacts with the nuclear spin. Protons and neutrons are also spin-1/2 particles. As discussed previously, the interaction between these magnetic moments favors certain arrangements of electron and nuclear spin, which results in discrete energy levels.

