1
$\begingroup$

I know how to make a minimum-chi square fit when I have two variables both affected by statistical error. But how do I perform the fit when one of them has no statistical error, just the uncertainty of the instrument (for example say I am using a ruler which carries a constant uncertainty of 1mm) and according to my model this uncertainty cannot be neglected?

I use Root by Cern to make fits. I assume it follows this procedure to minimize $\chi^2$ of the fit when both the $x$ and $y$ variables are affected by statistical error (picture taken from L. Lyons, Statistics for nuclear and particle physicists, page 139) enter image description here

This hinges on the condition that both values of $x$ and $y$ are gaussian distributed, so that you can evaluate the $\chi^2$ value corresponding to a given choice of the fit parameters. But what happens when you have no info on the statistics of $x$ and you just know that each measurement is sistematically affected by some fixed uncertainty of the measuring instrument?

Improve this question
$\endgroup$
7
  • $\begingroup$ What you are describing is a systematic error called bias. This is handled differently from a random error. I'm sure you will find several post discussing this topic here on Physics SE. $\endgroup$
    – Semoi
    Jul 10, 2022 at 7:32
  • $\begingroup$ @Semoi thx, can you help me find one of them? I have been searching a lot but was not able to find $\endgroup$
    – Salvatore Manfredi D
    Jul 11, 2022 at 17:35
  • $\begingroup$ If the $1\ {\rm mm}$ here refers to the fact that the tick marks on the ruler are spaced by $1\ {\rm mm}$, then I would refer to that uncertainty as statistical. When you use the ruler, you make a judgment call as to the digit below $1\ {\rm mm}$, corresponding to your best guess as to where the object lies between the lines. Your guess has some randomness, but on average shouldn't systematically over or under estimate the length of the object. An example of a systematic error would be if the room was very hot and caused your ruler to expand so the tick marks weren't $1\ {\rm mm}$ apart. $\endgroup$
    – Andrew
    Jul 12, 2022 at 23:51
  • $\begingroup$ @Andrew the ruler was just an example. Say you are taking data from a voltage meter which yields numbers that do not oscillate. So for example at the ends of a simple resistor you measure 1.02 V. You use a different instrument to evaluate the current, and measurements fluctuate around the value 1.00 A. You take many measurements and evaluate the average and standard deviations. You repeat this for many values of voltage. How do you fit data to obtain the Ohm's law? Suppose you cannot neglect the uncertainty on the voltage. How do you optimize the chi squared of the fit? $\endgroup$
    – Salvatore Manfredi D
    Jul 14, 2022 at 8:03
  • 3
    $\begingroup$ “I use Root by Cern … I assume”: Professional advice: don’t assume. Root does a lot of great things, but the exact relationship between what the methods do versus what they are called can be a little surprising. $\endgroup$
    – rob ♦
    Jul 15, 2022 at 17:32

2 Answers 2

Reset to default
1
$\begingroup$

how do I perform the fit when one of them has no statistical error, just the uncertainty of the instrument

The best resource for learning about how to handle analyzing and reporting uncertainty in science is the NIST technical note 1297 on uncertainty: https://www.nist.gov/pml/nist-technical-note-1297

In there they classify uncertainty in two broad category. One is uncertainty which is evaluated based on statistical methods and the other is uncertainty which is evaluated based on non-statistical methods. In your case, for the specific measurement of interest the uncertainty would be the second type.

The best place to obtain the non-statistical uncertainty is in the manufacturer's documentation for the measuring device. If there is no documentation available or if the documentation doesn't specifically list the uncertainty, then it might be reasonable to assume that the uncertainty is based on the smallest digit of a digital display or the smallest tick mark of a non-digital display. If so, then you would model a reading of 1.02 V as coming from a uniform distribution from 1.015 V to 1.025 V. The standard uncertainty would then be equal to the standard deviation of a uniform distribution with a width of 0.01 V.

When you have both types of uncertainty, statistical and non-statistical uncertainties, then the combined standard uncertainty is the square root of the sum of squares of the individual uncertainties. This is basically the propagation of errors formula.

Improve this answer
$\endgroup$
9
  • $\begingroup$ The link is broken (for me at least it gives a 404 not found error) $\endgroup$
    – hft
    Jul 15, 2022 at 17:38
  • 1
    $\begingroup$ @SalvatoreManfrediD that sounds more like a statistics question than a physics question. But to my knowledge the chi square is non parametric. It shouldn’t matter at all that your uncertainty is a rectangular distribution instead of a Gaussian $\endgroup$
    – Dale
    Jul 15, 2022 at 22:31
  • 1
    $\begingroup$ @SalvatoreManfrediD The $\chi^2$ test does assume that the residuals are Gaussian distributed. However you're entering into a messy area of experimental physics. Ideally you would be able to calculate the exact distribution of the residuals under a null hypothesis and then test it. Usually this is impossible. So in practice you do what you can with the information you have. The link Dale posted has good advice. "Quick and dirty" methods used include (a) neglect if it is small, (b) try to empirically measure the distribution, (c) assume it's Gaussian (and, inflate the error bars to be safe). $\endgroup$
    – Andrew
    Jul 16, 2022 at 0:44
  • 1
    $\begingroup$ More rigorous (and more difficult) methods would include (a) model the non-statistical source of uncertainty and subtract it or (b) do a Bayesian analysis and marginalize over the uncertainty parameter (using some parametric model for the uncertainty, such as a uniform distribution in some interval, which need not be Gaussian). $\endgroup$
    – Andrew
    Jul 16, 2022 at 0:48
  • 1
    $\begingroup$ @SalvatoreManfrediD I think you will need to ask on the statistics SE. I think it is called Cross Validated. They should know. My understanding is that chi square is so robust against different distributions that it is considered non-parametric. But I am not a statistician $\endgroup$
    – Dale
    Jul 16, 2022 at 1:18
1
+50
$\begingroup$

This is not a trivial problem: indeed, $\chi^2$ goodness-of-fit tests assume Gaussian distribution of the residual errors, and would be affected by the rounding error (since it renders these distribution non-Gaussian). A brief google search for "Goodness of fit rounding" shows that there is no generally accepted procedure for dealing with such situations, e.g., see
About the effect of rounding on the properties of tests for testing statistical hypotheses
Sensitivity of goodness-of-fit statistics to rainfall data rounding off

Parametric approach
One could design a statistical model for the rounding error (e.g., the values are uniformly distributed between two scale marks and rounded with probability $p$ to upper value and with probability $q=1-p$ to the lower one), which would give the general distribution of residuals for which one can construct a statistical test (likely more complicated than the $\chi^2$, but doable with modern computers.)

Non-parametric approach
One could look for non-parametric tests for goodness-of-fit, that is the tests that do not assume a specific underlying distribution. I would not be surprised, if some people would still argue that the Gaussian assumption is not necessary for using the least squares test. E.g., see What are some of the most common misconceptions about linear regression?

Practical approach
Posting a question on Cross Validated (the Stack Exchange forum for statisticians) is likely to generate more qualified answers than in PSE.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.