1
$\begingroup$

I have measured a quantity $b$ as a function of two variables, let's say $x$ and $y$. Then I fitted a 2D polynomial of the shape $$b(x,y) = \sum c_{j,i} x^i y^j$$ to the data (which is of 2nd order).

To estimate the goodness of the fit, I calculated the $\chi^2$ value as $$\chi^2 = \sum_i\frac{\left(O_i-E_i\right)^2}{E_i},$$ where $O_i$ are the observed (measured) values and $E_i$ are the expected (fitted) values.

I have 30 points in parameter space, that is 30 different combinations of $x$ and $y$ and the resulting $b$ values. If I understand it correctly, the degrees of freedom are $$\mbox{DF} = 30 - 1 = 29.$$

Using some calculator or tabulated values and using a significance level of 0.05 (a confidence-level of 95 %), I allow for $\chi^2 \lesssim 18$ to say "That fit is good."

Is that a reasonable statement to estimate the goodness of a 2D fit or is that completely non-sense ?

$\endgroup$
2
$\begingroup$

Statistical tests of this kind are quite hard to interpret. What the p value really says is "this thing you did is a better fit than a random line you could have drawn", or to interpret the p value more formally: "there is a probability p that the amount of variation you explained with this curve is due to chance". So if you drew 1/p random curves, one of them would work as well as this one...

Much better is a plot of residuals - for each data point, look at the difference between the value predicted by your fit, and the actual value observed. If that difference is small (comparable to your measurement error), you really have a good fit (model of your data). If the difference is much larger than your measurement error, or if the residuals are not normally distributed, or if the residuals are much bigger in one area than another, it is a sign that your fit is "not very good".

There are methods to estimate the confidence interval on your fit parameters - I often find that a more useful indication than the p value.

$\endgroup$
  • $\begingroup$ "a better fit than a random line", ahh, yes, I know what you mean. Also, the second paragraph is quite a useful suggestion, thank you! Concerning the "methods to estimate the confidence interval on your fit parameters", is it possible to just give me a keyword I could then use as a further-reading starting point...? $\endgroup$ – Alf Sep 27 '17 at 12:45
  • 1
    $\begingroup$ You could start with this question. In general, the crossvalidated.stackexchange is an excellent forum for this kind of questions - you will get to interact with actual statisticians who know what they are talking about (instead of this physicist who thinks he does...) :-) $\endgroup$ – Floris Sep 27 '17 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.