4
$\begingroup$

I am curently working on Forecast in cosmology and I didn't grasp very well different details. Forecast allows, wiht Fisher's formalism, to compute constraints on cosmological parameters.

I have 2 issues of understanding :

1) Here below a table containing all standard deviation of these parameters, for different cases :

table of constraints

In the caption of Table 16, I don't understand what the term "Marginalized 1$\sigma$ erros" means. Why do they say "Marginalized", it could be simply formulated as "the constraints got with a 1$\sigma$ confidence level" or ""errors with a 1$\sigma$ C.L (68% of probability to be in the interval of values)", couldn't it ?

If it would have been writen "marginalized 2$\sigma$ error, the first value in Table 16 would have been equal to $(\Omega_{m,0})_{2\sigma}=0.032 = (\Omega_{m,0})_{1\sigma} =0.016 \text{x} 2$, wouldn't it have been ?

I would like to understand this vocabulary which is very specific of this Forecast field.

2.1) Here below a figure representing the correlations (by drawing contours at 1$\sigma$ and 2$\sigma$ C.L (confidence levels)) between the different cosmological paramters :

correlation picture

I have understood, the right diagonal (with Gaussian shapes) represent the posterior distribution, i.e $\text{Probability(parameters|data)}$ or the probability to get an interval of values for each parameter, knowing the data.

But how to justify that I have these posterior distribution on this descending diagonal ?

I know the relation : $\text{posterior}= \dfrac{\text{likelihood}\,\times\,\text{prior}}{\text{evidence}}$ or the equivalent :

$p(\theta|d)={\dfrac{p(d|\theta)p(\theta)}{p(d)}}$ with $\theta$ the parameters and $d$ the data.

We can use Fisher's formalism assuming likelihood is Gaussian, and the posterior are obtained by inverting the Fisher's matrix.

So I wonder what the others cases (except this diagonal) are plotted and mosty what they represent in the formula above, especially towards posterior distribution :

$\text{posterior}= \dfrac{\text{likelihood}\,\times\,\text{prior}}{\text{evidence}}$

It seems all this cases looks like "joint distribution" but I can't get to recall what this joint distribution corresponds to, and its link with posterior distribution.

2.2) Finally, a last question, in the caption of figure 9, it is also noted "marginalized contours" : there too, why using the term "marginalized" ??

Any help is welcome, I would be very grateful.

If someone thinks this post should be moved to mathematic exchange forum, don't hesitate to do it. I posted here since there is a physical context but I may be wrong.

ps : GC represents Galaxy clustring probe, WL the weak lensing, GC$_{\text{ph}}$ the photometric proble and XC the cross-correlations.

UPDATE 1: I follow my reasoning to try to better undertsand :

1) Concerning the descending diagonal on the right , does it represent a distribution in a frequentist sense or bayesian sense ?

We agree this diagonal represent the posterior of each parameter, i.e the probability (by integrating the surface of PDF) to have the parameter into a range (i.e, bounds of integration) knowing the data, don't we ? In my case, the data come from CAMB code which produces matter power spectrum.

2) So, assuming the fact that posteriors are represented on this diagonal, I don't understand how to introduce the notion of integration over all others parameters ?

Could one take a concrete example of integration on a posterior distribution ? (I make confusions between PDF of a random variable and the posterior of a parameter which allows to estimate this parameter).

3) When we talk about frequentist way, I understand that marginalization is done by : $$f(x)=\int_{0}^{+\infty}f(x,y)\,\text{d}y$$ but I can't get to do the same wih posterior $p(\theta|d)$ (where $p(\theta|d)=\dfrac{p(d|\theta)p(\theta)}{p(d)}$).

4) I think that in frequentist way, we manipulate PDF of a random variable whereas in Bayesian approach, we manipulate probability of parameters of a PDF, i.e parameters of a model given by a PDF too. I don't know precisely what is the meaning of the concept "marginalisation of a paramater".

That's why I would like to get a simple and concrete example of marginalization (in Bayesian approach) by integrating over all others parameters with the definition of posterior probability that I have written above ?

Sorry if the answers about these questions may be evident but this is a new field for me.

UPDATE 2: I keep on doing investigations on this topic to explain better my problem :

Like I said into 4) section above, I begin to grasp the differences between frequentist/bayesian approaches.

If I take the notations $\theta_{1}$ and $\theta_{2}$ parameters and $d$ for the data, maybe I could write to express "the operation of marginalisation" :

$$p(d|\theta_{1})= {\large\int}_{0}^{+\infty}\,p(d|(\theta_{1},\theta_{2})) \text{d}\theta_{2}={\large\int}_{0}^{+\infty}\dfrac{p((\theta_{1},\theta_{2})|d))\,p(d)}{p((\theta_{1},\theta_{2})}\text{d}\theta_{2}\quad(1)$$

and finally get :

$$p(d|\theta_{1})= {\large\int}_{0}^{+\infty}\,\dfrac{p((\theta_{1},\theta_{2})|d)\,p(d)}{p((\theta_{1},\theta_{2})} \text{d}\theta_{2}$$

which implies :

$$p(\theta_{1}|d)= \bigg[{\large\int}_{0}^{+\infty}\,\dfrac{p((\theta_{1},\theta_{2})|d)\,p(d)}{p((\theta_{1},\theta_{2})} \text{d}\theta_{2}\bigg]\,\dfrac{p(\theta_{1})}{p(d)}\quad(2)$$

In practise, The factor of $(1)$ $\big(p(d|\theta_{1})\big)$ is commonly taken as the likelihood function : this assumes a theoritical model given by parameter $\theta_{1}$ from which we can produce data with the theoritical likelihood and that's why I write $p(d|\theta_{1})=\text{probability of having data given a model}$ for the distribution of $\theta_{1}$ (which is a parameter in Bayesian approach and not a random variable like in classical Frequentist approach, is it right ??)

But to finish about these formulas : relations $(1)$ and $(2)$ seem to be difficult to compute since I don't know if I can take a uniform distribution for the factor $p(\theta_{1},\theta_{2})$ : is it really the case ?

Any remark is welcome.

$\endgroup$
  • $\begingroup$ Take this as a hint to know what to look for: I think I remember that marginalized means that you find that dependent (from the values of other parameters) parameter after you integrate over all the parameters spaces of all the other parameters. I may be wrong but the answer should be related to this way of doing things: like you find the values of all the other parameters independently of the value of the one you care about and then from those values you derive the value of other parameter $\endgroup$ – Run like hell Jun 29 '19 at 10:20
  • $\begingroup$ I'll need to really understand this by September, iif you don''t come to answer I may answer by then. If you do, please answer your own question with the right answer so that we all can see it. I put the question in my favourites so I may check it from time to time $\endgroup$ – Run like hell Jun 29 '19 at 10:22
  • $\begingroup$ @Runlikehell thanks for you remarks, I have added an UPDATE 1 to try to better epxlain my issues, if this can help you ... Regards $\endgroup$ – youpilat13 Jun 29 '19 at 17:13
2
+50
$\begingroup$

So take a look at the definition of the Fisher Information: $$ \mathcal{I}(\theta) = E\left[ \left. \left( \frac{\partial}{\partial \theta} \log f(X;\theta) \right)^2 \right| \theta \right] = \int \left( \frac{\partial}{\partial \theta} \log f(x;\theta) \right)^2 f(x;\theta) dx $$ Nowhere in the above will you find any $p(\theta)$. That is because the Fisher information describes the likelihood function, not the posterior distribution. Now it happens that the likelihood function is equal to the posterior distribution if you assume a uniform prior $p(\theta) = \text{constant}$. One of the main reasons people care about the Fisher Information is that it gives a lower bound on the parameter error via the Cramer-Rao bound, which says that the variance of an unbiased estimator of $\theta$ must be at least $\frac{1}{\mathcal{I}(\theta)}$. Similarly, the inverse of the Fisher Information Matrix $F^{-1}$ is often referred to as a covariance matrix for the parameters $\theta$, but that is only true in the bayesian sense under the assumptions of a uniform prior and gaussian likelihood.

Under these assumptions, the "covariance matrix" $\Sigma = F^{-1}$ defines a multivariate normal distribution $\mathcal{N}(\mu, \Sigma)$. This is the probability density function from which the confidence intervals are inferred, and also the one that is marginalized, simply by integrating over some of the parameters, as you have already mentioned. The nice thing is that to marginalize a multivariate normal distribution, all you have to do is remove the rows and columns that correspond to the unwanted "nuisance" parameters from the covariance matrix.

When you marginalize over everything but one parameter, you just get a variance (a single diagonal element of the covariance matrix), and this can be represented by a gaussian curve with that variance, which is what we see on the plots along the diagonal in your figure. When you marginalize over all but two parameters, you get a two-by-two matrix. Assuming the parameters are nondegenerate, the matrix will be nonsingular and have two real, positive eigenvalues and associated eigenvectors. The eigenvectors are the semi-minor and semi-major axes of a confidence ellipse and the eigenvalues are the lengths of these axes. It is these confidence ellipses that we see on the other plots.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks, 1) You said that Similarly, the inverse of the Fisher Information Matrix 𝐹−1 is often referred to as a covariance matrix for the parameters 𝜃, but that is only true in the bayesian sense under the assumptions of a uniform prior and gaussian likelihood. I would like to know what does happen when prior is not uniform, I mean for example wit a gaussian prior : what are the consequences on posterior ? 2) Secondly :** Are all Likelihoods are always gaussian** ? $\endgroup$ – youpilat13 Nov 19 '19 at 23:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.