# What should be the real error band of a fit function?

This is a question concerning data analysis. It could relate to particle/nuclear physics or other field. Please note that this question has nothing to do with technical problems, and is more of a theoretical question.

The question is, what's the precise definition of a error band of a fit function (I will present two approaches later), and what's the right way to draw it? I have already made much of the sense on this topic, so I will brief here.

My previous question was here: How to calculate the uncertainty of a fit function?

Let's call the method of drawing a statistical fit band and Method A:

A fit usually is defined by $$\chi^2:=\sum_i(\frac{f(x_i)-y_i}{\sigma_i^{stat}})^2.$$ After a fit on a dataset with statistical uncertainties, we can extract the covariance matrix of the fit parameters (no matter what analysis tool you use, for example, CERN ROOT). Let's call the eigenvectors of the covariance matrix are $$\vec{v}_i$$ and eigenvalues $$\sigma^2_i$$, due to semi-definitiveness. Then we can generate Gaussian variables $$\epsilon_i$$ and make the fit-parameters shift by the combination $$\sum_i \epsilon_i \sigma_i \vec{v}_i$$. For each random-generation, we have a fit-function curve. Then we can Monte-Carlo curves to get a fit-band. See picture below.

On the other hand, we have Method B (data-shuffling method): It's usually applied to systematic uncertainties. Before fit, we simply shift our data by $$y_i\rightarrow y_i+\epsilon \sigma_i^{sys}$$, where $$\epsilon$$ is a Gaussian random variable. We can do this many many times, then get a bund of curve. See the picture below is the 1-$$\sigma$$ band of those curves.

If we add them in quadrature (original data also shown), we have:

So far so good.

Method C: On the other hand, we could use modified-least-$$\chi^2$$ method to handle systematic uncertainty: $$(\chi^2)^\prime:=\sum_i(\frac{f(x_i)-y_i-\epsilon \sigma_i^{sys}}{\sigma_i^{stat}})^2+\epsilon^2\ \text{ for one dataset}$$

By minimizing it, we have the best fit parameter and the best $$\epsilon$$, tell us how datasets should shift. The band will be defined from the error of parameters such that $$\Delta \chi^2=1$$. Use method A again, we have

See? The "total" error band by method C is even narrower than the pure statistical blue band in Fig 1. Not to mention comparing with Fig 3.

This is sensible, though. Because method C gives the best parameters that globally minimize $$(\chi^2)^\prime$$, while method A (the usual fit) only minimize the $$\chi^2$$ in the partial parameter space (no $$\epsilon$$).

Now, should fit error band given by method A+B (Fig 3) the true band? Or should it be method C (Fig 4)?

I think they represent two different point of view. Method B is like SMEARING the fit curve due to the shift of data by systematic uncertainties. So, combined with pure statistical band, the total $$\Delta\chi^2$$ should be 1+something>1. While in method C $$\Delta\chi^2=1$$ exactly, the narrow band simply says: "OK, here is the fit CURVE (a line) you want. But I'm not 100% sure of it, so I draw a little uncertainty band. This band is not as wide as method A+B because it should be those data points that are uncertain, not my function curve."

So can you tell me which band is the right one? Method A+B or method C?

It took me a while to put together those words. This is really important for my research. Hope someone of relevant field could answer me.