# What is the difference between data-driven and semi-data-driven techniques?

I keep hearing the terms data-driven and semi-data-driven techniques(especially for background estimation) in many CERN analysis talks. I looked for their difference but didn't find any documentation about their difference. Can anyone give me the answer or point me to resources about this and also tell me what the alternatives of these two techniques are?

• The alternative could be called "simulation-driven". The terms just mean whether you rely on data or simulation (or a mix) to tell you something. Oct 19, 2017 at 16:20
• and simulations have generators which use specific theoretical formulas. Oct 20, 2017 at 7:20

Let's say we want to search for some new particle beyond the Standard Model (SM) but that there SM physics and/or detector glitches which can be confused with it: this is the background. We need to know the background precisely enough in order to establish a signal/background for the new particle. One technique is that of control samples. We select two region of phase space: the control sample region (CR) which is chosen so as to be rich in background (as predicted by the new physics and SM), and the signal region (SR) where the actual measurement of new physics will take place. The CR does usually not contain only the background we are interested in but also some other contributions. Here is then what a semi-data driven analysis would look like: in the following $N$ will denote a count; $BG$ and $\require{cancel}\cancel{BG}$ will refer respectively to the specific background we want to know and any other background; $data$ and $MC$ will obviously refer to measured and Monte-Carlo simulated counts (@annav comment reminded me that I need a caveat here: instead of a Monte-Carlo we could have the fit of a function to data in the CR, and this function is then used to extrapolate to the SR; i.e. a Monte-Carlo is just an advanced way to do that but not one that conceptually change the problem).

$$N_{BG}^{SR} = \left(N_{BG,\,data}^{CR} - N_{\cancel{BG},\,MC}^{CR}\right)\frac{N_{BG,\,MC}^{SR}}{N_{BG,\,MC}^{CR}}$$

Let's walk through it: in the first parenthesis, we evaluate the selected background by removing from data the simulated background we are not interested in. Then we divide by the simulated background in the CR to get a calibration of the MC. Finally we can safely multiply by the simulation of the background in SR. This is a typical semi-data-driven analysis. This is not completely data-driven because of using the MC to evaluate $\cancel{BG}$ in this case: the Monte-Carlo here would contain theoretical inputs and this term $N_{\cancel{BG},\,MC}^{CR}$ is an absolute contribution, as opposed to a term appearing in a ratio with an experimental count, as the other MC terms. If that contribution was negligible or could be measured too, then it would be data-driven. The extrapolation of the background from the CR to the SR is an orthogonal issue that is, anyway, always present.

I should add that this is only possible because of the extraordinary progresses in the full simulation of the detector (using a software known as GEANT 4). Tevatron could not proceed along the same lines for example. Indeed, as you see in my terse exposition, it all hinges on how accurate the MC can be.

• you have not addressed the simple "data driven" analysis and the difference with the "semi data driven " which is the question of the op. Oct 20, 2017 at 10:49
• I would say that only data driven analysis is used, for example, if the background to a resonance is taken from the data before and after the resonance region. So the difference is that the "semi" uses theoretical information in the MC.What says you? Oct 20, 2017 at 10:51
• yes, good point, I lost track of the question in my hurry!
– user154997
Oct 20, 2017 at 11:04
• Your example is a good one where no MC is needed for the extrapolation actually.
– user154997
Oct 20, 2017 at 11:08