13
$\begingroup$

For any set of data points, you can comprise a 100% correlated and fitted curve using a sum of sloped lines all multiplied by their respective Heaviside step functions to form a zig-zag shaped curve. And yet, we do not use those models. This asks the question: how does the academic science community of physicists know when to disregard a model that has a higher correlation than another possible model?

$\endgroup$
  • 14
    $\begingroup$ To penalize the use of an excessive number of parameters, researchers can apply the adjusted R squared or Akaike information criterion techniques, for example. To avoid overfitting to a particular data set, they can perform cross-validation, saving a portion of their data to evaluate a fitted model $\endgroup$ – Chemomechanics Jul 31 '17 at 19:59
  • 2
    $\begingroup$ I agree with @Chemomechanics. I think cross-validation is the key here. You can fit a model perfectly with one dataset, but as soon as you collect another, it's completely off. It's the 3-step training, validation and prediction that scientists use, I suppose. $\endgroup$ – Myridium Jul 31 '17 at 20:01
  • 2
    $\begingroup$ I want to flag this as off-topic because it belongs on Cross Validated, but that is not an option through the app (which is all I have right now). I'm going to flag it for Math instead (since that is an option), but it really should be on CV. $\endgroup$ – Geoffrey Jul 31 '17 at 21:56
  • 2
    $\begingroup$ @Geoffrey The topic isn't just math itself, it is how physicists choose to use it for scientific purposes. $\endgroup$ – user165197 Aug 1 '17 at 1:17
  • 1
    $\begingroup$ @DaneJoe CV isn't about math either, it's about statistical models which is the core of the question. Not that it's off-topic here though. $\endgroup$ – Dmitry Grigoryev Aug 1 '17 at 9:17
22
$\begingroup$

For any set of data points, you can comprise a 100% interpolated and fitted curve using a sum of sloped lines all multiplied by their respective Heaviside step functions to form a zig-zag shaped curve. And yet, we do not use those models.

Physics from the time of Newton to now is the discipline where mathematical differential equations are used , whose solutions fit the data points and are predictive of new data. In order to do this , a subset of the possible mathematical solutions is picked by use of postulates/laws/principles , as strong as axioms as far as fitting the data.

What you describe is a random fit to a given data curve, and no possibility of predicting behavior for new boundary conditions and systems. It is not a model.

how does the academic science community of physicists know when to disregard a model that has a higher correlation than another possible model?

If one has a complete set of functions, like Fourier or Bessel functions one can always fit data curves. this is not a physics model, it is just an algorithm for recording data. The physics model has to predict what coefficients the functions that are used for a fit will have.

$\endgroup$
  • 1
    $\begingroup$ Newton probably said it first: "Hypotheses non fingo" (very loose translation - I don't just make theoretical stuff up to fit whatever I measured). Of course William of Occam (the guy with the razor!) had the same general idea, before "modern science" got started properly. $\endgroup$ – alephzero Aug 1 '17 at 4:02
12
$\begingroup$

“Higher correlation” is meaningless beyond a certain point; namely, when both models are already so good that every data point agrees with the model prediction within the bounds of its measurement uncertainty. Then the model which gets even closer, “unnecessarily close” to the individual measurement expectation-values isn't the better model. In fact it's considered inferior, namely overfitted: some of the features in the model will be attributable to random measurement errors rather than the “inherent” system behaviour.

Now, that's not necessarily a problem. In some applications, particularly in engineering, it's perfectly fine to just measure the particular system that you're interested in and record the curve in the way you described. Then you have a very good model for that one system, and can make all the predictions as to how that system will behave in such and such state. It's a bit of an “expensive” model, in the sense that you need to store a sizable chunk of data instead of just some short formula, but hey – if you're an engineer and your customers are happy then who cares?

Things are a bit different in physics though. Here, we seek to make all laws as general as they can possibly be. Most things we can measure in the lab aren't really what we want to describe, but more of a simplified prototype to the stuff we're really interested in (but can't measure directly). So basically what we want to do is extrapolate lab results to the real world. Now, extrapolation is a dangerous business
By the third trimester, there will be hundreds of babies inside of you
...in particular, different models that fit the same data may give rise to wildly different extrapolations. So how can we know which such model is actually going to give correct predictions?

The answer is, we can't – if we knew the correct values that are to be predicted, we wouldn't need to extrapolate in the first place! However, we can apply Occam's razor and that tends to be really quite effective at this business. Basically the idea is: a complicated model that requires lots of fitting parameters is not a good bet for extrapolation (if you can find one such model, it's likely also possible to find a similar- but differently structured model that behaves completely different as soon as you leave the original measurement domain). OTOH, a simple, elegant model given by e.g. as simple differential equation has good chances of being the only such simple model that's actually able to fit the data, hence you can be optimistic that its extrapolations will also be close to a more accurate, wider expectation window.

$\endgroup$
6
$\begingroup$

Two thoughts come to mind:

  1. A theory has to describe more than one data set. If you fine tune your "theory" to fit one experiment perfectly, it most probably will fail on another one. In particular, if you consider different types of experiments (say, you measure the fall of an apple and the motion of the moon).

  2. A theory should have some beauty. This means, there should be some simple concept behind it, so you have the feeling that it "explains" things rather than just giving an incomprehensive formula which fits to the experiment. Of course, no theory can explain everything, every theory needs some axioms, some ansatz. But the better theory explains more observations with less assumptions.

$\endgroup$
  • 2
    $\begingroup$ A theory should have some beauty? There's a fraught hypothesis. What's the simple concept behind magneto-hydrodynamics? If you're looking for explaining more observations with less assumptions, perhaps efficient is a better word. $\endgroup$ – user121330 Jul 31 '17 at 22:46
  • $\begingroup$ And #1 is not a perfect guide either. If you are a cosmologist, you only get one universe to compare with your model. $\endgroup$ – Rococo Aug 1 '17 at 0:33
  • 4
    $\begingroup$ @user121330: The simple concept behind magnetohydrodynamics is that it is based on electrodynamics and hydrodynamics which are both conceptually simple. There is a difference between complexity arising from the fact that several/many simple things are considered and one thing which is inherently complex. For example, quantum chromodynamics has some inherent complexity because it is non-abelian. Quantum electrodynamics is abelian and thus has less inherent complexity but it can get as complex as you want if you apply it to complex systems. But you are right, efficient might be the better word. $\endgroup$ – Photon Aug 1 '17 at 6:56
  • $\begingroup$ @Rococo: But I can perform different experiments in this universe. For example, if I am interested in the matter distribution of the universe, I can observe SNIa luminosities or the CMB anisotropy power spectrum and get independent statements about the universe: th.physik.uni-frankfurt.de/~scherer/Blogging/… $\endgroup$ – Photon Aug 1 '17 at 6:58
  • $\begingroup$ also even with 1 experiment you can fit to different subsets of data $\endgroup$ – jk. Aug 1 '17 at 9:29
3
$\begingroup$

Your question is essentially about the Problem of Induction. This is a philosophical question about how we can generalize and form a theory from a limited set of data points. There are an infinite number of curves that fit the data (and even more infinite if we allow for margins of error), so how do we know which it is?

The general answer is that we do additional experiments. A theory is only useful if it makes predictions, rather than merely describing the given data. So after forming a theory that describes the data, we do more experiments, and see if those results are consistent with the theory. If they aren't, we know that the theory was wrong and it can be discarded.

This is why theories that can't be tested like this are often called non-scientific; a popular example is the Multiverse -- by some definitions there's no way to tell whether or not other universes exist (if it could be detected, we would consider it part of our universe). A fuzzy area is theories that could in principle be tested, but such tests are not feasible with any technology we expect to be available; some scientists call String Theory "not even wrong" because of this.

The scientific method can never fully "prove" a theory, it's always possible that some new data point will come along that doesn't fit the theory, and it will have to be revised or discarded. For example, Mercury's orbit doesn't fit Newtonian mechanics; Einstein's General Theory of Relativity solved this (astronomers previously suspected that there was another planet perturbing Mercury's orbit). But every time we get new data that conforms to a model, we increase our confidence in it; Einstein became world famous when observations of the Sun bending the path of light from distant stars during an eclipse exactly matched the prediction of General Relativity.

So this brings us to your question. Each model makes predictions, and presumably the different models make different predictions. So we do additional experiments, to see which one's predictions are confirmed, and we then discard the other one.

$\endgroup$
-3
$\begingroup$

This is not a question that has a simple answer. The short answer is "experience" but if you want something beyond that, and are ready to read some real esoteric stuff, I would suggest you look at the philosophers of science Popper, Polanyi, and Kuhn.

$\endgroup$
  • 4
    $\begingroup$ The question does have a simple answer (as given by annav, or Photon for example). You want a function that has at least the potential to extrapolate new values with sufficient correctness - a trivial bunch of lines connecting the dots will never do that. $\endgroup$ – AnoE Jul 31 '17 at 21:47
  • 2
    $\begingroup$ I'm not surprised that this is an unpopular sentiment, but it does have some truth to it. There are useful heuristics and rules of thumb, but if deciding the correct model for a given data set did not involve some subjective judgement, much of the "creativity" in physics would have been automated already. $\endgroup$ – Rococo Aug 1 '17 at 0:31
  • $\begingroup$ @AnoE Sorry, but volumes have been written on this one question. Absolute volumes. If you have the definitive, systematic answer, you are welcome to publish it and prove that it is correct. But if you have that answer, congratulations. You are the only one who does. $\endgroup$ – bob.sacamento Aug 1 '17 at 14:35
  • 2
    $\begingroup$ The particular question is about the trivial "zig-zag-line" "model". A valid answer can at least point out why (trivially, and simply) this particular zig-zag is not a good model, and leave the OP with pointers to where all of this turns to get difficult. This particular answer has nothing of help whatsoever, it does not need "experience" to see that a zig-zag-line connecting the data dots has no power whatsoever... It's not about whether this answer is "right" or "wrong", but if it is helpful for the question at hand. $\endgroup$ – AnoE Aug 1 '17 at 15:37

protected by Qmechanic Aug 1 '17 at 3:27

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?