0
$\begingroup$

Look, for example, fig 5a in the following paper.

My understanding was that, to calculate the 'expected' line, we would calculate the predicted signal+background cross section $\sigma_{s+b}$ (since the cross section is the test statistic here), define a critical region by chosing a confidence level (or, equivalently, a rate of type II error in this case), let's say, 95% so that

$$ \tag{A} 0.05=\int_0^{\sigma^{95\%}_{\text{exp}}(m_S)}d\sigma \:f(\sigma(m_S)) $$

where $f$ is, in theoretical research, our favorite PDF for the background+signal hypothesis -- usually Poisson -- correctly centered at our calculated value and with a standard deviation dictated by the chosen PDF model itself. This could also be derived from a set of pseudoexperiments.

In this way we have a $\sigma^{95\%}_{\text{exp}}(m)$ for each value of the unknown degree of freedom $m$ that parametrizes it.

Then, the green and yellow bands are understandably obtained.

But what about the observed line???

I understand how the observed line varies with $m$ on graphs that plot the 95% confidence level on the ratio $\sigma^{95\%}_{\text{obs}}/\sigma_{s+b}(m)$, because here the dependence comes from the denominator. These are the graphs that exclude the theoretical point in space of parameters when $\sigma^{95\%}_{\text{obs}}/\sigma_{s+b}(m)<0$.

But what about in graphs like the example I cite that put an upper limit directly on the cross sections?

My only guess is that it is the same as before in eq. (A), but, now, with the PDF centered at (and standard deviation derived from this median)

$$ \sigma_{b+s}(m)-\sigma_{b}+\sigma_{\text{obs}}, $$

where and $\sigma_{b+s}(m)$ and $\sigma_{b}$ are calculated and $\sigma_{\text{obs}}$ is measured.

However, this must not be the case because than the dependence in $m$ would again come only from $\sigma_{b+s}(m)$ which is calculated and would not cause irregular curves as is always seen in these graphs.

I hope I could make my question clear and appreciate any insights.

$\endgroup$
1
  • $\begingroup$ I will write fully if I have time. For now: your construction isn’t quite right. The 0.05 probability lies in the sample space. Your formula looks more like a credible region $\endgroup$
    – innisfree
    Commented Dec 27, 2023 at 10:25

1 Answer 1

0
$\begingroup$

Might this very simplistic answer help: The sensitivity is the expected limit, in the absence of a signal, based on the background model (the null hypothesis). You can calculate that using a Monte Carlo realization of your experiment, repeating that many times to get the statistical variation, and show the median (centroid of the Brazil flag) and width (green and yellow at 1 and 2 sigma) of that. Now, to get the observed limit, you repeat the exact same process, but a single time using real data instead of many times using simulated data. Le voila, your observed limit (the solid line), and how it compares to the distribution of expected limits (the Brazil flag).

$\endgroup$
4
  • $\begingroup$ Thank you for the answer! I don't think that the expected limits shown in graphs like the one I show are derived from the positive 2 sigma deviation of the background alone. I believe they are derived from the negative 2 sigma deviation of the background plus signal hypothesis, i.e., the curve is the contour where the alternative (background plus signal) hypothesis would give a measurement as low as that only 5% of times, as I described in the question. Do you disagree? $\endgroup$
    – GaloisFan
    Commented Feb 25, 2022 at 22:58
  • $\begingroup$ The sensitivity should be calculated under the null hypothesis, whatever that is. If you search for the signal, that would be background only with zero signal. If you search for BSM physics on top of a SM signal, then that would be background plus SM signal. $\endgroup$
    – rfl
    Commented Feb 26, 2022 at 7:13
  • $\begingroup$ I understand what you call sensitivity, but I disagree that the "expected" line is derived from 95% confidence on maximum cross section of background alone -- this could not possibly depend upon a new physics parameter (such as the scalar resonance mass in the graph above). The expected line is derived from an application of eq. A above for each $m_S$. $\endgroup$
    – GaloisFan
    Commented Feb 27, 2022 at 1:57
  • $\begingroup$ Oh but it does depend on the new physics parameter! Depending on that parameter you would expect a different signal, e.g. at different energy in your detector. Depending on how your background is distributed and things like acceptances/efficiencies of your analysis, you will be more or less sensitive to that new physics, depending on where it would be expected. But you don't assume that any such signal is in your data in order to calculate that sensitivity. $\endgroup$
    – rfl
    Commented Feb 27, 2022 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.