In chemical compounds, where does the “magic” come from in atomic “magic numbers”?

It is well known that atoms with a full electron shell are more stable, it is one of the first facts taught in a middle school Chemistry course:

"An element whose atoms have no electrons outside filled energy levels is particularly stable chemically. Such elements are called noble gases"

These numbers (for stable nuclei) are 2 (He), 10 (Ne), 18(Ar), 36(Kr), 54(Xe) and 86(Rn). Each correspond to a full shell of electrons with similar energies that start from $$ns$$ and end in $$np$$, just before the $$(n+1)s$$ shell is filled ($$n$$ being the principal quantum number and $$s, p$$ being the first two quantum numbers of the orbital angular momentum, i.e., $$\ell=0,1$$). We also know that the valence of an atom is the number of electrons more or fewer than the number for a noble gas.

What I do not understand is the reason for this:

"Stable chemical compounds are held together by the Coulomb attraction and are typically formed from atoms whose valence add up to zero."

What is so special about both atoms having noble gas configuration?

For the last electron in the shell we have that the potential behaves like $$−Ze^{2}/r$$ near the nucleus (whose charge is $$+Ze$$), and like $$−e^{2}/r$$ outside the atom, where the nuclear charge is screened by the negative charge of Z − 1 electrons, then the potential away from the atom (with $$\text{no. of electrons}=Z$$) should be zero. And for example, taking away an electron would increase the energy, as it would require work in order to take it out of the potential. There would also be no attraction to bind an extra electron outside the atom.

As far as I understand the mechanism through which these atoms would bind together is to first ionize and then let Coulomb attraction join them. For concreteness let us assume $$NaCl$$:

$$Na+Cl\xrightarrow{E_{\text{ionization}}}Na^{+}+Cl^{-}\xrightarrow[\text{interaction}]{\text{Coulomb}}NaCl$$

I know that for these elements the ionization energy may be small, but the energy for the non-ions should still be lower and thus they should be more stable, but this is not the case. My question would boil down to: Why do they stay together? Why not just snatch away the electron from the other ion and go their own ways?

It is always good to remember the historical context. The statement

An element whose atoms have no electrons outside filled energy levels is particularly stable chemically. Such elements are called noble gases.

confuses the phenomenon with the conclusion. The story was the following:

1. Mendeleev knew nothing about the group of stable elements in 1869, the Nobel gases were simply not discovered at that time.
2. The discovery of the noble gases aided in the development of a general understanding of atomic structure. In 1895, French chemist Henri Moissan attempted to form a reaction between fluorine, the most electronegative element, and argon, but failed. Learning from these experiments, Danish physicist Niels Bohr proposed in 1913 that the electrons in atoms are arranged in shells surrounding the nucleus, and that for all noble gases except helium the outermost shell always contains eight electrons.
3. In 1916, Gilbert N. Lewis formulated the octet rule, which concluded an octet of electrons in the outer shell was the most stable arrangement for any atom; this arrangement caused them to be unreactive with other elements since they did not require any more electrons to complete their outer shell.
(Point 2 and 3 are quotes from Wikipedia about Nobel gases)

=> It was and is until today a empirical fact, that elements with 2, 8 and again 8 electrons fill shells in a way, that these shells are particularly stable chemically.

Stable chemical compounds formed from atoms whose valence add up to zero. What is so special about both atoms having noble gas configuration?

Let us remember the historical context again:

1. The (Bohr’s) model's key success lay in explaining the Rydberg formula for the spectral emission lines of atomic hydrogen. While the Rydberg formula had been known experimentally, it did not gain a theoretical underpinning until the Bohr model was introduced.
2. The electron is able to revolve in certain stable orbits around the nucleus without radiating any energy contrary to what classical electromagnetism suggests.
3. The Bohr model gives almost exact results only for a system where two charged points orbit each other at speeds much less than that of light. This involves one-electron systems such as the hydrogen atom, singly ionized helium, and doubly ionized lithium.
4. Several enhancements to the Bohr model were proposed, most notably the Sommerfeld model or Bohr–Sommerfeld model, which suggested that electrons travel in elliptical orbits around a nucleus instead of the Bohr model's circular orbits.
5. In the end, the model was replaced by the modern quantum mechanical treatment of the hydrogen atom, which was first given by Wolfgang Pauli in 1925, using Heisenberg's matrix mechanics. The current picture of the hydrogen atom is based on the atomic orbitals of wave mechanics which Erwin Schrödinger developed in 1926.
(All points are quotes from Wikipedia about Bohr model)

In a nutshell (again from Wikipedia):

• The emission spectra of excited electrons in atoms are the experimental facts.
• Bohr's condition, that the angular momentum is an integer multiple of ħ ...
• ... was later reinterpreted in 1924 by de Broglie as a standing wave condition: the electron is described by a wave and a whole number of wavelengths must fit along the circumference of the electron's orbit.
• In modern quantum mechanics, the electron in hydrogen is a spherical cloud of probability that grows denser near the nucleus. The rate-constant of probability-decay in hydrogen is equal to the inverse of the Bohr radius

=> All explanations and conclusions are based on the chemical stability of Nobel gases. There is no explanation why Nobel gas atoms have 2, 8 and again 8 electrons in a stable arrangement; except for the fact that there are solutions for partial differential equations which can describe the electron excitations.

Quantum mechanics solves such partial differential equations for the spherical probability for electrons by the help of spherical harmonics.

Despite their name, spherical harmonics take their simplest form in Cartesian coordinates. This leads to spherical probabilities with 2 electrons for the s-shell and 6 electrons in the p-shell

But spherical harmonics have also another solutions. More general spherical harmonics of degree ℓ are not necessarily those of the Laplace basis $$Y_{\ell }^{m}$$, and their nodal sets can be of a fairly general kind. (Wikipedia)

One solution correspondent with the cubical atom, proposed by Lewis:

The cubical atom was an early atomic model in which electrons were positioned at the eight corners of a cube in a non-polar atom or molecule. This theory was developed in 1902 by Gilbert N. Lewis and published in 1916 in the article "The Atom and the Molecule" and used to account for the phenomenon of valency... The figure below shows structural representations for elements of the second row of the periodic table.

The corresponding spherical harmonics is the following:

Each of the eight segments of the sphere topologically corresponds to the eight edges of a cube.

Up to this point, I have only described what science says. Since your question didn't receive enough attention, I feel free to add my own thoughts to tell you why 2 and 8 electrons in a shell are perfect balanced around a nucleus.

Based on Lewis cubical distribution of electrons and remembering, that electrons have a magnetic dipole moment, it is possibly to bring the eight outer electrons of Ne and Ar into a perfect equilibration. This is the case for 4 electrons, pointing with their north poles to the nucleus and the other 4 electrons with their south poles. For He the two electrons are directed to each other antiparallel. Did you see the correspondence to Paulis exclusion principle?

• The "cubical atom" described here is fringe science and it has no relationship with reality beyond wishful thinking -- as has been pointed out to you repeatedly. Continued posting of misinformation, as in this post, constitutes abuse of this site. – Emilio Pisanty Jun 25 at 6:21
• @EmilioPisanty Can you, as a scientist, point out to me, who is not anchored in the scientific world, an inconsistency in the above? I know your answer, that this is not your task. But we both wasting time, you, reading my posts, and I, being wrong, but not seeing this. – HolgerFiedler Jun 25 at 7:15
• The onus is on you. Can you provide a quantitative calculation based on this "model" that can derive the absorption and emission spectra (say, of the lithium-through-neon first row of the periodic table, which you mention explicitly here) and which matches experiment? (These have been known for some 50+ years within normal QM; references are listed in the relevant results pages of the NIST ASD.) Unless you can, there is absolutely no point in discussing this model. – Emilio Pisanty Jun 25 at 7:35
• In other words, your "model" cannot be said to be inconsistent with experimental results because it doesn't even try to produce calculations and predictions to the stage where they could be compared with experiment. It isn't a model at all (in the scientific sense) ─ based on the evidence you've put forward thus far, it is no more than children's squiggles. – Emilio Pisanty Jun 25 at 7:39
• @EmilioPisanty That you are right and I know this. So, I’m not even able to calculate the octopol of a parallelopiped and hope that one day somebody like the idea and will calculate it for 8 magnetic dipole moments. – HolgerFiedler Jun 25 at 18:19