# Cosmology / Interpretation between credibility/confidence_level with bayesian/frequentist approaches

I try to understand the following article :

testing general relativity from curvature and energy contents at cosmological scale

I don't understand the title of figure 1 : where it is indicated the prior values for $$\omega_{b}, \omega_{\text{cdm}}, \text{h}, ...$$ : what do authors mean by "prior ?

1) Does this term "prior"refer to the bayesian formula :

$$\begin{equation} \text{posterior}= \dfrac{\text{likelihood}\,\times\,\text{prior}}{\text{evidence}}\quad(1) \end{equation}$$

which, I think, corresponds to the formula :

$$\begin{equation} p(\theta|d)={\dfrac{p(d|\theta)p(\theta )}{p(d)}}\quad(2) \end{equation}$$

where $$\theta$$ is the parameter to estimate and $$d$$ represent the data

???

So, if this is the case, the prior of parameter $$\theta_{i}$$ would represent the probability $$p(\theta_{i})$$, wouldn't it ?

2.1) On the figure 3 : I don't understand how to get this figure.

Given Likelihood is proportional to posterior (is it right from above equation $$(1)$$ ?), I have to know the theorical model to compute Likelihood ?

I mean, to get $$p(\theta|d)$$, I have to generate the probability $$p(d|\theta)$$ assuming I know the value of $$\theta$$ parameters, don't I ?

There seems here a paradox : I compute the posterior $$p(\theta|d)$$ to estimate $$\theta$$ parameter on one side but I have to know precisely the probability $$p(\theta)$$ on the other side.

2.2) Moreover, how to compute on this figure the Likelihood of red and black curves which corresponds respectively with parameter $$w$$ free and $$\Omega_{k},\Omega_{dyn}$$ with also free ?

I don't know which theorical model (I suppose this is a PDF (probability function)) to use ?

3) Finally, I have a last question about Confidence level (CL with frequentist approach) and Credibility level (Bayesian approach) :

How to make the link between these 2 notions (if this is possible) ? the first is an interval on a random variable and the second is an interval about a parameter, so at first sight, this would't have the same signification.

However, I often see the notion of "Confidence level" for estimation of a parameter, like for example the contours on figure 4 of the article cited above, i.e o this figure : Any help or explanations are welcome, I am very interested in understanding all these concepts of statistics.

Regards

Let's say you have conducted an experiment, and from it obtained a data distribution. However, raw data by itself is not exactly interesting; instead, you want to be able to extract some meaningful (i.e. statistical) result from the data.

The process by which this is done is known as statistical inference: given some set of data, $$\boldsymbol{X}$$, one wishes to test how well it can be described by a model, $$M(\boldsymbol{X};\boldsymbol{\Theta})$$, where $$\boldsymbol\Theta$$ are the parameters of the model, whose true values are unknown a priori. To extract likely values of the model parameters given the observed data, one defines the likelihood function: $$\begin{equation} L(\boldsymbol{\Theta};\boldsymbol{X}) \equiv M(\boldsymbol{X};\boldsymbol{\Theta}), \end{equation}$$ which differs from the model in that the data is fixed, while model parameters are treated as random variables.

A common method of inference is that of maximum likelihood estimation (MLE), which states that the set of parameter values, $$\boldsymbol{\hat\Theta}$$, that result in maximisation of the likelihood function provides an estimate of the true parameter values. Confidence regions can then be constructed around the $$\boldsymbol{\hat\Theta}$$ using the likelihood ratio test.

Another method is known as Bayesian inference, which makes use of Bayes' theorem: $$\begin{equation} P(\boldsymbol{\Theta}|\boldsymbol{X},M) = \frac{P(\boldsymbol{X}|\boldsymbol{\Theta},M) P(\boldsymbol{\Theta}|M)}{P(\boldsymbol{X}|M)}\,, \end{equation}$$ where $$P(\boldsymbol{\Theta}|\boldsymbol{X},M)$$ is the posterior probability distribution, $$P(\boldsymbol{X}|\boldsymbol{\Theta},M)=L(\boldsymbol{\Theta};\boldsymbol{X})$$ is the likelihood function, $$P(\boldsymbol{\Theta}|M)=\pi(\boldsymbol{\Theta})$$ is the prior probability distribution, and $$P(\boldsymbol{X}|M)=\mathcal{Z}$$ is the Bayesian evidence, which normalizes the posterior distribution. Maximum a posteriori (MAP) estimation prescribes a method of estimating model parameters through maximisation of the posterior probability. However, note that it is equivalent to just maximise the likelihood times prior, $$\begin{equation} L(\boldsymbol{\Theta})\pi(\boldsymbol{\Theta}), \end{equation}$$ since constant factors (such as the evidence) do not shift the location of the maximum in parameter space. So in a practical application of MAP for parameter estimation purposes, one only needs to define the likelihood function and prior distributions: the former according to whichever theoretical model under test, and the latter defining the search region in parameter space.

what do authors mean by "prior"? Does this term "prior" refer to the bayesian formula...So, if this is the case, the prior of parameter $$\theta_i$$ would represent the probability $$p(\theta_i)$$, wouldn't it ?

Yes, they are referring to the prior that appears in Bayes' theorem, though note that this is a probability distribution.

In this case, what they seem to mean is that they have chosen $$\delta$$-distributions as priors for the $$\omega_b$$, $$\omega_{cdm}$$, $$h$$, $$A_s$$, $$n_s$$ and $$\tau_{reio}$$ parameters. Put more simply, what they show here is the model distribution when all parameters are fixed to certain values.

Given Likelihood is proportional to posterior (is it right from above equation (1) ?), I have to know the theorical model to compute Likelihood ?

Yes, you need to have a theoretical model before the likelihood function can be defined.

I mean, to get $$p(\theta|d)$$, I have to generate the probability $$p(d|\theta)$$ assuming I know the value of $$\theta$$ parameters, don't I ?

There seems here a paradox : I compute the posterior $$p(\theta|d)$$ to estimate $$\theta$$ parameter on one side but I have to know precisely the probability $$p(\theta)$$ on the other side.

To calculate a value for the posterior distribution (or more accurately, posterior times evidence) at a particular point in parameter space, you would evaluate $$L(\boldsymbol{\Theta})\pi(\boldsymbol{\Theta})$$ at that point. Doing this for sufficiently many points inside the prior volume will then yield a distribution for the posterior probability, from which an estimate for the parameters can be extracted.

Finally, I have a last question about Confidence level (CL with frequentist approach) and Credibility level (Bayesian approach) :

How to make the link between these 2 notions (if this is possible) ? the first is an interval on a random variable and the second is an interval about a parameter, so at first sight, this would't have the same signification.

However, I often see the notion of "Confidence level" for estimation of a parameter, like for example the contours on figure 4 of the article cited above, i.e o this figure

Confidence regions are constructed using the likelihood distribution, while credible regions are based on the posterior distribution. The main difference (other than how they are obtained) lies in their interpretation.

I'm not an expert in the philosophy of frequentist vs Bayesian statistics, but as I understand it, a frequentist believes that there is a "true value" for the parameters of a model (assuming that the model describes our universe), while a Bayesian does not believe in the existence of a single "true value", instead treating parameters as random variables. Hence, a confidence interval is a random interval (dependent on a particular dataset) which can be said to enclose the true parameter value with some probability, while a credible region is the region about a random variable (i.e. the parameter).

To present a simple example, consider $$5\pm1$$. A frequentist would say that the $$\pm1$$ is random; while a Bayesian thinks the $$5$$ to be the random variable.

But at the end of the day, they are both meant to convey an estimation of the parameters of some theoretical model, e.g. in the figure you presented.

• Thanks for this detailled answer. Just last things : 1) Don't you want to write $\begin{equation} L(\boldsymbol{\boldsymbol{X};\Theta}) \equiv M(\boldsymbol{\Theta};\boldsymbol{X}), \end{equation}$ instead of \begin{equation} L(\boldsymbol{\Theta};\boldsymbol{X}) \equiv M(\boldsymbol{X};\boldsymbol{\Theta}), \end{equation} ? 2) What doesss happen when we take a Gaussian distribution for the prior ? Does the posterior remain gaussian in this case ? Regards, I will award you for your good answer. Nov 23 '19 at 11:54
• @youpilat13 No, it is correct as written, the model parameters are treated as variables in the likelihood function, while the data is the input. For your second question, one can't really say what the posterior distribution will look like because it will depend largely on the likelihood distribution as well. Nov 23 '19 at 12:56
• sorry to bore you again but what's the difference between statistical inference ( maximum likelihood estimation (MLE) ??) and bayesian inference (Maximum a posteriori (MAP) ??) both medthods are using the max/min of Likelihood except the MAP which maximizes/minimizes the product of likelihood by a non-uniform prior ? Nov 23 '19 at 20:52
• Yes, in practice, the difference between MLE and MAP is whether or not prior distributions are incorporated in the analysis. Nov 24 '19 at 3:35