Is the Least Squares Fit always the accepted Fit in Physics? Is the Least Squares Fit always the accepted Fit in an experiment? Suppose you have $N$ data points and a function $f$ with some parameters. There is a one Least Square sum, it may be obtained for more than one set of parameters but we can find that most optimal fit. Is it true that sometimes that Least Square fit is discarded because the expected values for the parameters are assumed to be different?
Why is it necessary to give starting values to the algorithms? Why can't the computer find those parameters on its own?
 A: No, it’s not. Sometimes it’s much more complicated than least squares, and sometimes it’s a by-hand “guide for the eye.” Not everybody has the expertise to apply a fancier data analysis, and they don’t let perfect be enemy of good.
Least squares is just sort of default for a typical situation, where you have a curve and a model, and you know nothing about the fitting parameters a priori. Then you play around with the parameters to give the computer a decent guess and report what least squares tells you. Of course, you also have to make sure you’re not over- or under-fitting.
The key here, and with all of science, is that you honestly and thoroughly report what you did. That will help your readers decide whether whatever choice you made was reasonable.
A: "Is it true that sometimes that Least Square fit is discarded because the expected values for the parameters are assumed to be different?" - well, yes, sometimes you get unrealistic values. This just means that you ended up in a local minimum of your functional (depending on your fit parameters) and the choice of the starting values was not optimal. If you want to end up in the global minimum, it might be that you have to start iterations already quite close to it, or at least, closer to it than to other local minima. This is why your initial parameter choice might be important (you might be lucky and then it is not important though). For example, if you are trying to fit an oscillatory function, it is always a good idea to have a good guess of the frequency before starting the iterations.
Actually, I vaguely remember when I was a student, we had a course on data analysis etc, where the least square fitting procedure was introduced, and it was somehow recommended to use it when your fit function is linear, and if not, try your best to modify the parameters and variables to obtain a linear function nonetheless. In that case, I suppose there are no additional local minima.
A: A least squares fit is the best you can do if the data are normally (gaussian) distributed about a theoretical relation, such as a curve or a surface. Very often this is a good assumption.
A: If you read statistics, you probably know that there is no single "best" method to determine the propagation parameter you are interested in. Every method has its pros and cons -- so mathematically speaking there is no best solution for "fitting data". In addition, you probably know that there is not a single but at least two different type of least square fits, if we take into account that the input parameters have a random error as well.
However, I would agree that at least  in 80% of physics papers in which the authors have to fit a dataset they use a least square fit. I reckon that there are at least four parts why this is so:

*

*We do not teach statistics in its full depth, so many physicists are not particularly sensitive to the advantages and shortcomings of other methods.

*In many papers we are merely interested in the magnitude of the effect and if the uncertainty is a major concern we rather gather more data points.

*In many papers we are interested in the average value of an effect, and we do not have huge outliers. In this scenario the least square fit is known to perform well.

*The least square fit is particularly simple to perform using many software packages.

The last point is so important (at least to me) that I'm almost always fitting my dataset using a least square routine. Even if I know in advance that I will not trust its result, the fit result is still informative: E.g. if it allows me to how influential different input parameters and how they change my result, I am interested in a rough model to optimise my input parameters for the next experimental run. Just keep in mind: All models are wrong, but some are still useful.
