How to fit the data for the asymptotic limit?

Suppose we have a pretty good experiment data with very small random error. But our theory is limited, which makes the systematic error significant. Sometimes we do have a nice theory for the weak field or strong field limit, but the experiment will go beyond that. How can we fit the data for the limiting case?

To be more specific, let me assume that a real system is governed by the following relation: $$y_{\rm exact}=\sqrt{ax^2+bx^4}$$. But the perturbation method only gives $$y_{\rm theory}=kx$$, which agrees with the exact relation for $$x\rightarrow0$$. Now I got some data looks like the red crosses in the figure, what I expect to have is the blue dashed line. If I compromise to fit with the first five data points, I will get the magenta line. To make a comparison, the slope for the blue line is $$k=1$$, but the slop of the magenta line is $$k=1.144$$. There is an error of $$14.4\%$$!

I do think it is a waste if we just ignore the rest of the data, and limiting to the first several data will also expose us the scope of random errors. Is there any method to improve this? And maybe there is another greedy question, that can we acquire some modification to the existing model from those deviated data points? Thanks.

• If you comparing to an expansion around $x = 0$ why would you expect it to be correct in regions of the parameter space "far" from the center of the fit? When you make use of approximations (as we so often do) then it is your responsibility to insure that you don't try to apply those results in regimes where the approximation is not reasonable. – dmckee --- ex-moderator kitten Aug 12 '19 at 14:56
• in your hypothetical, what’s the complete state of knowledge of our prediction for $y$? We know a pertubative treatment gives $y=kx$, but do we know anything about the theoretical error on that prediction or the $x$ where the perturbative treatment breaks down? – innisfree Aug 30 '19 at 14:26
• ... To solve your problem, you need to consider all relevant information - which includes the data you observed but also what you knew about your theory and its prediction – innisfree Aug 30 '19 at 14:34

With real world data, I expect that the best you can do is fit the red crosses for the whole range of the data to ensure that you get the best estimate of the real physics relationship that is involved. If you know the physics that corresponds to this data, you can use that to fit the data to the correct functional form. After that is done, take the first derivative of your mathematical relationship, calculate the slope at approximately x=0.1, and extrapolate that slope to get the desired linear tangent by using the point-slope form of a line.

Suppose we have a pretty good experiment data with very small random error. But our theory is limited, which makes the systematic error significant. Sometimes we do have a nice theory for the weak field or strong field limit, but the experiment will go beyond that. How can we fit the data for the limiting case?

First of all and speaking as an experimentalist, I don't think that you're properly using the term "systematic error" as it applies to experimental data. You're using the term "systematic error" to describe the how the experimental data points differ from one's expectations based on some approximate theory, but that's not how the term "systematic error" is used by experimentalists. When experimentalists speak of "systematic errors", they mean errors that are introduced into an experimental measurement due to something like a defect in the experiment or an incorrect experimental parameter which systematically affects all of the measurements made by the experiment.

If you have some theory which you know is an approximation to the true theory, you can simply take your approximate theory and do a weighted fit of it to the data points based on where you expect your approximate theory to be most accurate. For your example, if your perturbation theory expansion about the point $$x=0$$ gives $$y_{theory}=kx$$, then you should give data points near $$x=0$$ more weight in the fitting process. In the limit that you believe that the random errors associated with the measured data are negligibly small (and assuming that you know that $$y=0$$ when $$x=0$$ for your example), then you could use just the first data point at $$x≈0.1$$ and the point (x,y)=(0,0) for the linear fit.

In general, determining how to precisely weight the data points when fitting to an approximate function can be very difficult because, after all, you don't know what the true theoretical function is (If you did know what it is, then you wouldn't have to concern yourself about how to fit the approximate function to the data - you could forget all about the approximate function and simply fit the true theoretical function to all of the data).

• The usage is actually common in hep-ph & hep-exp, though sometimes a distinction is made between systematic and theoretical errors. – innisfree Aug 30 '19 at 14:21

With data obtained by scanning of the graph joint to the question, the result of least square regression is shown in the next figure :

The fitting is very good : Root Mean Square Error = 0.0028

This is significantly better than with the quadratic function :

Root Mean Square Error = 0.0090 , next figure.

CONCLUSION :

In order to get the slope of the tangent at the origin, the proposed process is simple :

• Fit a convenient function to the experimental curve. If you have to chose between several kind of functions, compute the least square error for each and keep the one with the best fit.

• Compute the derivative of the function and the value of the derivative at origin.