Can artificial neural networks be effective theories? Wikipedia describes "effective theories" as follows.

In science, an effective theory is a scientific theory which proposes to describe a certain set of observations, but explicitly without the claim or implication that the mechanism employed in the theory has a direct counterpart in the actual causes of the observed phenomena to which the theory is fitted. That means, the theory proposes to model a certain effect, without proposing to adequately model any of the causes which contribute to the effect.

While Wikipedia starts with the context 'in science', I have only seen this language used in Physics so I am asking here on Physics.SE.
Theorems such as the Universal Function Approximation Theorem show that certain classes of artificial neural networks can approximate Lebesque measurable functions to arbitrary precision (up to measure zero) with sufficient width and depth of the layers of the network. This has led me to tell a joke to people that "Neural networks will never be a Theory of Everything, but they are already a Theory of 'Anything'".
Tying the notion of effective theories together with neural networks, do neural networks actually count as effective theories? Or is there a nuance here in which they do not qualify?

Presumably one could ask a similar question about polynomials in the context of the Stone-Weierstrass theorem, but I suspect the answer will be similar in character.
 A: I don't think this changes the spirit of the question you're asking, but I'll start with a clarification because I think Wikipedia's definition is unclear (I don't know what "actual causes" means). The way most physicists use the term, "effective theory" doesn't refer to a single theory by itself. It's a relationship between two different theories, one of which may be unknown. One theory can be an "effective theory" of another theory (possibly unknown) by reproducing some of that other theory's predictions using less-detailed postulates.
With that clarification, the answer is pretty clearly yes: a neutal network could indeed be crafted to reproduce many of the predictions of some other theory, say QED, so a neural network could indeed be an "effective theory." Most physicists wouldn't be happy with that kind of theory, not even as an "effective theory," because it's not very efficient: every parameter characterizing every neuron, not to mention the topology of the network, is essentially a separate postulate. That's a lot of postulates. But aesthetic criteria aside, the answer is yes: a neural network can be a (very inefficient) effective theory.
Actually, maybe I'm not being fair by assuming that it would be inefficient. Suppose we trained a neural net to reproduce a jillion different experimental results. In order for the training to succeed, the neural-net architecture that we started with needs to be sufficiently expressive. If we're lucky enough to choose an architecture that is just barely expressive enough, then we would indeed have an efficient theory, whether or not we call it "effective." In practice, though, choosing an architecture that is just barely expressive enough seems just as difficult as coming up with an efficient theory the traditional way — by chipping away at the problem over many decades, one publication at a time.
