2
$\begingroup$

Is the Least Squares Fit always the accepted Fit in an experiment? Suppose you have $N$ data points and a function $f$ with some parameters. There is a one Least Square sum, it may be obtained for more than one set of parameters but we can find that most optimal fit. Is it true that sometimes that Least Square fit is discarded because the expected values for the parameters are assumed to be different?

Why is it necessary to give starting values to the algorithms? Why can't the computer find those parameters on its own?

Improve this question
$\endgroup$
3
  • 4
    $\begingroup$ 1) Data analysis is a massive topic in physics. For example, the particle data group has many useful reviews on how to analyze particle physics data. It is not so simple as "compute the least squares fit." 2) There is not, in general a unique local minima to least squares problems, and finding the global minima can be extremely difficult. 3) Your final question about the algorithm is entirely distinct from the rest of the question and should be asked as a question on its own, there there are many sources describing least squares. $\endgroup$
    – Richard Myers
    Commented May 9, 2021 at 3:23
  • $\begingroup$ I wasn't sure how to ask this question. The University expects us to apply these statistical tools and we never really studied them. I heard that some of these questions are ill-posed and are open to interpretation. I typically study from basics and gradually build up to be able to handle more complicated problems. THEY want us to follow the opposite strategy: start with difficult problems and hopefully in time you'll understand the basics (incidentally). I don't agree. $\endgroup$
    – Some Student
    Commented May 9, 2021 at 3:39
  • $\begingroup$ The computer needs starting values because it is probably using an iterative approach, and it needs to know what values to start iterating with. $\endgroup$
    – David White
    Commented May 9, 2021 at 14:29

4 Answers 4

Reset to default
4
$\begingroup$

No, it’s not. Sometimes it’s much more complicated than least squares, and sometimes it’s a by-hand “guide for the eye.” Not everybody has the expertise to apply a fancier data analysis, and they don’t let perfect be enemy of good.

Least squares is just sort of default for a typical situation, where you have a curve and a model, and you know nothing about the fitting parameters a priori. Then you play around with the parameters to give the computer a decent guess and report what least squares tells you. Of course, you also have to make sure you’re not over- or under-fitting.

The key here, and with all of science, is that you honestly and thoroughly report what you did. That will help your readers decide whether whatever choice you made was reasonable.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I think I can see my problem. I just iterated over a wide interval of parameters and picked the ones that gave me the least squares sum but in fact I should be looking for local minima instead and chose the one that is closest to expected values. $\endgroup$
    – Some Student
    Commented May 9, 2021 at 4:15
1
$\begingroup$

"Is it true that sometimes that Least Square fit is discarded because the expected values for the parameters are assumed to be different?" - well, yes, sometimes you get unrealistic values. This just means that you ended up in a local minimum of your functional (depending on your fit parameters) and the choice of the starting values was not optimal. If you want to end up in the global minimum, it might be that you have to start iterations already quite close to it, or at least, closer to it than to other local minima. This is why your initial parameter choice might be important (you might be lucky and then it is not important though). For example, if you are trying to fit an oscillatory function, it is always a good idea to have a good guess of the frequency before starting the iterations.

Actually, I vaguely remember when I was a student, we had a course on data analysis etc, where the least square fitting procedure was introduced, and it was somehow recommended to use it when your fit function is linear, and if not, try your best to modify the parameters and variables to obtain a linear function nonetheless. In that case, I suppose there are no additional local minima.

Improve this answer
$\endgroup$
1
$\begingroup$

A least squares fit is the best you can do if the data are normally (gaussian) distributed about a theoretical relation, such as a curve or a surface. Very often this is a good assumption.

Improve this answer
$\endgroup$
2
  • $\begingroup$ What does "distributed about a theoretical relation" mean? How should the uncertainties be calculated? And what if the distribution is Poisson? $\endgroup$
    – Some Student
    Commented May 9, 2021 at 19:24
  • $\begingroup$ Is this what you meant? ibb.co/w02PMmq The thick line is a fitted curve and the thin lines separate regions containing 68 % data points for $\sigma\ 1$, 95 % for $\sigma\ 2$ and so on. $\endgroup$
    – Some Student
    Commented May 9, 2021 at 19:41
1
$\begingroup$

If you read statistics, you probably know that there is no single "best" method to determine the propagation parameter you are interested in. Every method has its pros and cons -- so mathematically speaking there is no best solution for "fitting data". In addition, you probably know that there is not a single but at least two different type of least square fits, if we take into account that the input parameters have a random error as well.

However, I would agree that at least in 80% of physics papers in which the authors have to fit a dataset they use a least square fit. I reckon that there are at least four parts why this is so:

  1. We do not teach statistics in its full depth, so many physicists are not particularly sensitive to the advantages and shortcomings of other methods.
  2. In many papers we are merely interested in the magnitude of the effect and if the uncertainty is a major concern we rather gather more data points.
  3. In many papers we are interested in the average value of an effect, and we do not have huge outliers. In this scenario the least square fit is known to perform well.
  4. The least square fit is particularly simple to perform using many software packages.

The last point is so important (at least to me) that I'm almost always fitting my dataset using a least square routine. Even if I know in advance that I will not trust its result, the fit result is still informative: E.g. if it allows me to how influential different input parameters and how they change my result, I am interested in a rough model to optimise my input parameters for the next experimental run. Just keep in mind: All models are wrong, but some are still useful.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.