$\chi^{2}=\sum_{i=1}^{n}\left(\frac{x_{i}-\mu}{\sigma}\right)^{2}$ OR $ \chi^{2}=\sum_{i=1}^{n}\left(\frac{x_{i}-\mu_{i}}{\sigma_{i}}\right)^{2}$

Question

By using a general Likelihood computed from a theoretical model and $\lambda_{i}, \lambda_{j}$ as cosmological parameters, We have the following definition of an element $(i, j)$ of Fisher matrix $F$ : $$ F_{i j}=\left\langle-\frac{\partial^{2} \ln (\mathcal{L})}{\partial \lambda_{i} \lambda_{i}}\right\rangle=\left\langle\frac{\partial \ln (\mathcal{L})}{\partial \lambda_{i}} \frac{\partial \ln (\mathcal{L})}{\partial \lambda_{j}}\right\rangle $$

We make here a strong assumption by considering that all Likelihoods are Gaussian, relating this latter and the $\chi^{2}$ by:

$$\chi^{2}=\sum_{i=1}^{n}\left(\frac{x_{i}-\mu}{\sigma}\right)^{2}\quad(1)$$

$$\Rightarrow \ln (\mathcal{L})=-\frac{1}{2} \chi^{2}+K$$

with $K$ a constant, where one has taken general notations with data vector :

$$\mathbf{X} \equiv \left\{x_{1}, . ., x_{n}\right\}$$.

We Considering a model with

$$\mu=\overline{\mathbf{X}}\quad(2)$$

from the maximum likelihood estimator where the mean of data is represented by vector $\overline{\mathbf{X}}$.

Thus, we can write : $$ -\frac{\partial \ln \mathcal{L}}{\partial \lambda_{i}}=-\sum_{k=1}^{n} \frac{\left(x_{k}-\mu\right)}{\sigma} \frac{\partial \mu}{\partial \lambda_{i}} $$

so :

$$ \begin{aligned} F_{i j} &=\sum_{k=1}^{n} \sum_{k^{\prime}=1}^{n}\left\langle\frac{\left(x_{k}-\mu\right)\left(x_{k^{\prime}}-\mu\right)}{\sigma^{4}} \frac{\partial \mu}{\partial \lambda_{i}} \frac{\partial \mu}{\partial \lambda_{j}}\right\rangle \\ &=\sum_{k=1}^{n} \sum_{k^{\prime}=1}^{n} \delta_{k k^{\prime}} \frac{1}{\sigma^{4}}\left\langle\left(x_{k}-\mu\right)\left(x_{k^{\prime}}-\mu\right)\right\rangle \frac{\partial \mu}{\partial \lambda_{i}} \frac{\partial \mu}{\partial \lambda_{j}} \end{aligned} $$ since $: \delta_{k k^{\prime}}\left\langle\left(x_{k}-\mu\right)\left(x_{k^{\prime}}-\mu\right)\right\rangle=\sigma^{2}$ Following :

$$ F_{i j}=\sum_{k=1}^{n} \frac{1}{\sigma^{2}}\left\langle\frac{\partial \mu}{\partial \lambda_{i}} \frac{\partial \mu}{\partial \lambda_{j}}\right\rangle=\sum_{k=1}^{n} \frac{1}{\sigma^{2}} \frac{\partial \mu}{\partial \lambda_{i}} \frac{\partial \mu}{\partial \lambda_{j}}\quad(3) $$

Question : I don't know if I have to consider a unique value for $\mu$ and $\sigma$ like I did in eq$(1)$ or maybe should I rather write eq$(1)$ like this :

$$\chi^{2}=\sum_{i=1}^{n}\left(\frac{x_{i}-\mu_{i}}{\sigma_{i}}\right)^{2}\quad(4)$$ ?

and then consider eq$(2)$ with the $\mu$ as a vector of different means :

$$\mu=\overline{\mathbf{X}}\quad(5)$$

with

$$\overline{\mathbf{X}} \equiv \left\{\bar{x}_{1}, \bar{x}_{2}, . ., \bar{x}_{n}\right\}=\left\{\mu_{1}, \mu_{2}, . ., \mu_{n}\right\}\quad(6)$$

and not : $$\overline{\mathbf{X}} \equiv \left\{\mu, \mu, . ., \mu\right\}$$

In the case of expression $(6)$, the final expression of Fisher element $F_{ij}$ would be :

$$F_{i j}=\sum_{k=1}^{n} \frac{1}{\sigma_{k}^{2}}\left\langle\frac{\partial \mu_{k}}{\partial \lambda_{i}} \frac{\partial \mu_{k}}{\partial \lambda_{j}}\right\rangle=\sum_{k=1}^{n} \frac{1}{\sigma_{k}^{2}} \frac{\partial \mu_{k}}{\partial \lambda_{i}} \frac{\partial \mu_{k}}{\partial \lambda_{j}}\quad(7)$$

As you can see, I make confusions in $\chi^2$ definition between the expected value of a model and its generalization when we consider a vector of data, which seems to assume that the means are different and not equal to a same single value $\mu$.

This is the same issue about a unique $\sigma$ instead of $\sigma_k$ : indeed, $\sigma_k$ would mean that I have multiple measures for the same point $k$.

1. If have had only one measure for each point point $k$, I think that correct expression is eq$(3)$. The same thing for single $\mu$. Associated expression of $\chi^2$ would be eq(1) in this case.

2. If have had multiple measures for each point point $k$, I think that correct expression is eq$(7)$ since I can define a $\sigma_k$ from these multiple data. $\sigma_k$ is the error on each point $k$, i.e on each multiple measure for point $k$. The same thing for multiple distincts $\mu_k$ which means that we would have an expected value for each point $k$. Associated expression of $\chi^2$ would be eq(4) in this case.

So finally, is my reasoning on point 1. and 2. correct ?

The appropriate expression equation$(3)$ or equation$(7)$ depends on these 2 cases 1. and 2. , doesn't it ?

Answer 1

The notation in this question is very hard to follow. In defining the Fisher matrix it seems the indices $i$ and $j$ label parameters, but in the $\chi^2$ likelihood the index $i$ is labeling data samples. And it is not at all clear what the underlying data or model are supposed to be.

Nevertheless, given a choice between Eq 3 and 7, I think the answer is pretty clearly that Eq 3 is correct. Eq 3 represents a case where you use $n$ data points to measure 1 parameter $\mu$ and its uncertainty $\sigma$. In Eq 7, you are fitting $n$ parameters, plus uncertainty, to $n$ data samples (I'm assuming each data sample $x_i$ is a single number). I can't imagine a scenario where this is a meaningful thing to do. You would surely be overfitting your data set. I also hope you agree it would be hard to trust an uncertainty estimate of a parameter based on fitting that parameter to a single data point.

---

As I understand the edit to the question, the data are estimators $\hat{a}_{\ell m}^A$, where $A\in \{G, W\}$, which should be unbiased estimators of the cosmological parameters $a_{\ell m}$. The model parameters are $a_{\ell m}$ and $C_{\ell}^{A B}$. Then I would expect the log likelihood to have the form \begin{equation} \ln \mathcal{L} = -\frac{1}{2} \sum_{A=\{G, W\}} \sum_{B=\{G, W\}}\sum_{\ell=\ell_{\rm min}}^{\ell_{\rm max}} \sum_{m=-\ell}^{\ell} \left(a_{\ell m} - \hat{a}_{\ell m}^A\right) C^{-1}_{AB, \ell} \left(a_{\ell m} - \hat{a}_{\ell m}^B \right) \end{equation}

I'm inspired to make this guess based on the way the question was phrased. However, physically, I would have thought the $a_{\ell m}$ are not interesting physical parameters, since they are essentially random and not predicted by the theory. So I would actually have thought a likelihood of the following form would make more sense \begin{equation} \ln \mathcal{L} = -\frac{1}{2} \sum_{A=\{G, W\}} \sum_{B=\{G, W\}}\sum_{\ell=\ell_{\rm min}}^{\ell_{\rm max}} \sum_{m=-\ell}^{\ell} \left(\hat{a}_{\ell m}^A\right) C^{-1}_{AB, \ell} \left(\hat{a}_{\ell m}^B \right) \end{equation}