Can Coulomb be derived directly from Gauss? This relates closely to questions such as
Deriving Coulomb's Law from Gauss's Law
and
Does the Coulomb's law include more information than the Gauss's Law in electrostatics?
My question concerns the nature of isotropy in field theory generally, and how determinism impacts on this.
Here is the background. It is well known that a simple argument involving an integral and spherical symmetry allows one to use Gauss' law in electrostatics
$$
\nabla \cdot {\bf E} = \rho / \epsilon_0
$$
to deduce Coulomb's law for the field of a point charge,
$$
{\bf E} = q \hat{\bf r}/(4 \pi \epsilon_0 r^2).
$$
It is often claimed, however, that in this area Coulomb's law contains
more information than Gauss' law. The argument is that, given Coulomb's
law one can deduce Gauss, but given Gauss alone one cannot deduce Coulomb.
This claim is based on the idea that spherical symmetry is a further
assumption. It is claimed that, given a static point charge as source, one
has no way of knowing, in the absence of further field equations such as
the other Maxwell equations, whether or not the field will have a non-zero
curl.
Let me give a related example to convey the idea. If one wishes to know
the magnetic field of a long straight current-carrying wire, one can
use Ampere's Law to find the component in the direction around the wire.
One finds $B_\phi = \mu_0 I / (2 \pi r)$. However this calculation tells
one nothing at all about the radial component $B_r$. In order to establish
that $B_r = 0$ in this example, one needs further information, such as
Gauss' law for magnetic fields ($\nabla \cdot {\bf B} = 0$). So it is
clear that Ampere's law provides less information about static magnetic fields than does the Biot-Savart law.
However I think the reasoning for electrostatics is different.
For a static point charge there is spherical not just axial symmetry,
which is significant
because of the hairy ball theorem: a continuous vector field tangent to a
sphere must have at least one point where the vector is zero. So if the
field itself has spherical symmetry then its component tangent to the sphere
must be everywhere zero. This suffices to complete the argument from Gauss to Coulomb.
Before asking my specific question, I need to clear away a couple of issues. Coulomb's law is often
quoted for a "point charge" but one does not need to insist that the charge distribution is
strictly point-like, only that its radius is small compared to all other distances under
discussion. Also, one should not allow this charge to have anything beyond monopole moment
(no dipole etc.) since otherwise Coulomb's law would not apply. Finally, the charge is static.
It follows that the charge distribution we are talking about is isotropic.
Hence the nub of the question is:

*

*In classical physics, can an isotropic source give a non-isotropic field?

Or, to put it another way:


*Suppose we have a continuous vector field $F$ satisfying $\nabla \cdot {\bf F} = \rho$,
and the physical property $\rho$ has physical effects via its impact on the field, which
in turn can exert a force on other objects having non-zero $\rho$. Suppose further that
$\rho$ has the form of a uniform spherical ball, and otherwise
there is empty space everywhere else (in particular, no other source of the field
$\bf F$). In this scenario can the direction of $\bf F$ in principle be
other than radial towards or away from the centre of the ball?

I think my wish to answer "no" to these questions is because I am tacitly bringing in
some considerations which arise from the fact that we are talking about physics not
pure mathematics. In pure mathematics one could introduce a field component with
non-zero curl without changing $\nabla \cdot {\bf E}$.
But in physics the notion of "source" is connected
to the notion of cause and effect, and in classical physics at least we normally
assume a one-to-one correspondence between cause and effect. If we did not then
we might be implicitly introducing information in a way that did not satisfy the
Second Law of thermodynamics. Also, the very definition of the physical property
$\rho$ suggests to me that it would not make sense to speak of a spherically symmetric
$\rho$ without an isotropic associated field, because the field and its effects are
what indicates the presence of $\rho$ in the first place. What can a statement that
$\rho$ is isotropic mean other than that its field is isotropic?
Therefore if anyone wishes to answer "yes" to the above questions (1, 2)
then please could they discuss this further issue about cause and effect and one-to-one mapping.
If anyone wishes to answer "no" to the above questions, then please could they also state
explicitly whether or not it follows, in the context of deterministic physics at least, that Coulomb's law and Gauss' law are equivalent (in electrostatics in three dimensions) because each can be derived from the other?
 A: You say that the crux of your question is:

In classical physics, can an isotropic source give a non-isotropic field?

You are asking a question about what symmetries are present in the equations of motion (EOM) of physics.
If an equation of motion relating the source to the field has $SO(3)$ symmetry, it implies the following: whenever the source has $SO(3)$ symmetry, the field does also. You can replace "$SO(3)$" in the previous sentence with another symmetry group of your choice, but the particular scenario you're asking about a point charge has to do with $SO(3)$.
It appears that the laws that govern the electromagnetic interaction in our universe do have $SO(3)$ symmetry, and it further appears that, in our universe, all physical laws have $SO(3)$ symmetry. If that's the case, then an "isotropic" source (in the sense of $SO(3)$) cannot give rise to an anisotropic field. However, we can never completely rule out the possibility that some new fundamental interaction of nature, which doesn't have $SO(3)$ symmetry, could be discovered in the future.
A: 
So if the field itself has spherical symmetry then its component tangent to the sphere
must be everywhere zero. This suffices to complete the argument from Gauss to Coulomb.

Notice "if the field itself has spherical symmetry".
We can't derive this spherical symmetry merely from the relation $\nabla \cdot \mathbf E = \rho$ and the assumption that $\rho$ has spherical symmetry. As you wrote, curl of the field isn't implied. The field can have a transversal component, e.g. a field component $C(R)\mathbf{e}_\phi$ with rotational symmetry around axis $z$ and some convenient function $C(R)$.
We need more assumptions - such as our experience with electric fields of charged spheres. We just do not observe such "uninduced" field components (showing a preference for direction in space that the source does not have) in real world. If we generalize and make the assumption they don't exist, then we can derive Coulomb's law from the Gauss law. If we do not, we can't. Coulomb's law contains this assumption, Gauss' law does not.



*Suppose we have a continuous vector field $F$ satisfying $\nabla \cdot {\bf F} = \rho$,
and the physical property $\rho$ has physical effects via its impact on the field, which
in turn can exert a force on other objects having non-zero $\rho$. Suppose further that
$\rho$ has the form of a uniform spherical ball, and otherwise
there is empty space everywhere else (in particular, no other source of the field
$\bf F$). In this scenario can the direction of $\bf F$ in principle be
other than radial towards or away from the centre of the ball?


Yes in principle. Not known in practice.
If we observed deviation of the field from spherical symmetry, we would tend to expand the notion of the source and seek something else that is asymmetric that can fit that role. For example, effective gravity field on Earth is not perfectly radial and also gravity is weaker on the Equator. We managed to find plausible additional source - rotation of the Earth and Earth oblateness. But if we found asymmetric gravity field on a perfectly spherical planet that does not rotate and there is no other visible source, we could, in accordance with scientific methodology, conclude that Newton's gravity law is not universal and in some cases, the gravity field can have transverse components. However, many people would not like that, and would try to introduce invisible gravity source that would save the old law and explain the asymmetry.
A: The Green's function of the Poisson equation is the Coulomb law (for potential). This indeed neglects the rotational part of the field, but the same can be said about Coulomb law itself - it provides only static potential. If we want to discuss non-static effects, we need to consider a generalization of the Coulomb law to moving charges - Liénard-Wiechert potential.
(Perhaps this is not a full answer, but I hope this gives an additional perspective, and it is too answer-like, to put in comments.)
A: 
However I think the reasoning for electrostatics is different

Maybe that is the point. When Ampére tried to derive the laws for the forces between elementary parts of wires with electric currents, (as discussed here), the existing successful model were that of Newton and Coulomb, of forces at a distance. And he tried the same approach.
The evolution of the electromagnetic theory brought the concept of fields, and that force at a distance (for electrodynamics, between conducting wires) was explained as a consequence of a magnetic field, with zero divergence. The electrostatic force law from Coulomb, on the other hand, became a consequence of an electric field with zero curl.
We have now an intuitive understanding of an electric field as the result of several sources, each one with spherical symmetry. And a magnetic field as a result of linear sources such as conducting wires or magnet dipoles.
A: About your question 1:

can an isotropic source give a non-isotropic field?

Yes, a spherical, static source can give a non-isotropic field. Let's leave apart the possibility of non-spherically symmetric boundary conditions. Still, some additional isotropy assumption is required, in addition to the symmetry of the source. It is enough to consider the case of a point-like charge at the boundary between two half-spaces with a different dielectric constant. Therefore, the symmetry "of the space" is important as the symmetry of the source. But of course, it is independent of it.
I also have some objections to your considerations about cause-effect and the Second Law of Thermodynamics.
Cause-effect, in Physics, is usually a concept applicable to time-like separated events. In Electrostatics, time does not play a role, and source and field are time-independent. Gauss' law is a relation between two physical quantities. Moreover, cause-effect has nothing to do with the uniqueness of the field-source relation.
The considerations about the Second Law of Thermodynamics, if true, would be applicable to any other physical situation. But they would make it difficult to justify phenomena like the spontaneous breaking of symmetry.
Notice that the above considerations do not use the additional freedom of the curl of the vector field. However, it is not easy to dismiss this issue.
Knowing the correct expression of Maxwell's equations may be misguiding because one can take for granted something requiring independent physical assumptions, i.e., what are the sources of an electrostatic field.
Let's assume that we do not know anything about magnetic fields and the Maxwell equation for the curl of the electric field. Mathematics would tell us that the knowledge of
$$
\nabla \cdot {\bf E} = \frac{\rho}{\varepsilon_0}
$$
equation plus an appropriate asymptotic behavior is not enough to obtain a unique vector field ${\bf E}$. We need an expression for $ \nabla \times {\bf E}   $. Only Physics, not Math, can provide us with such an additional equation. The non-trivial observation is that we could imagine a world such that $ \nabla \times {\bf E} = {\bf W}  $, where ${\bf W}  $ is another source of the electrostatic field ${\bf E}$, in addition to the usual charge density.
Excluding the presence of the source term ${\bf W}  $ is Physics, and it is not included in Gauss' Law, independently of the symmetry of $\rho$. Coulomb's Law instead directly excludes the presence of a source of the electric field different from $\rho$.
