Cosmology context - MCMC code : Recomputation of covariance matrix after each point accepted I am working on a MCMC code (basically with Metropolis-Hastings) and I would like to understand different important points.
We always mention the covariance matrix which is used in the computation of accepted/rejected points.

*

*Initialy, it seems this matrix should be built like an Idendity matrix but with a diagonal of $\sigma_{i}^{2}$ (all covariances terms are null at the beginning). But what I don't understand is that I have only prior intervals on each parameter, not their initial variance : how can I manage this and build correctly the initial covariance matrix.


*After, I make a proposal on one of a parameter that will be accepted or rejected.


*If accepted, Why have I got to recompute this covariance matrix ? since the variances are changing at every point accepted (I mean, the histogram evolves during the computation, even after one more point) ?. Especially, how to compute the covariance terms from multiples single histogram ?


*In this case, once covariance matrix is recomputed, which parameter have I to do a proposal on : should I take the parameter that has the smallest variance or the biggest variance in the diagonal of current covariance matrix ?


*I understand I can compute the $\chi^2$ from current covariance matrix, the goal being to have the smallest $\chi^2$ possible. But this computation of this $\chi^2$ is for every point accepted I guess, insn't it ? (not all points proposed).
As you can see, it is a little haze for the moment for coding this MCMC computation.
Any clarifications/suggestions/remarks are welcome.
 A: I am having trouble interpreting exactly what you are asking, but I will answer based on my understanding of your question.
In the Metropolis-Hastings algorithm, you have a generating function $g(\theta' | \theta)$ which randomly produces a new candidate position given your current position in parameter space. Here, $\theta$ is the parameter that your walker is currently at, and $\theta '$ is the candidate position that you will decide to step to based on the MH criteria. If you are running a higher dimensional chain, these $\theta$'s could be vectors of parameters and everything I say holds in the multivariate case. $g(\theta' | \theta)$ can be thought of as a probability distribution that you draw from in order to generate your next move.
In principle, this function could be anything. In many applications, people choose a Gaussian with some covariance, which I understand to be the covariance you are talking about in your question. In the one dimensional case, this means that your generating function looks like
$$
g(\theta' | \theta) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp\left(-\frac{(\theta' - \theta)^2}{2\sigma^2} \right)
$$
That is, you center a Gaussian distribution at your current location, and randomly sample from it to pick your next move. In the multivariate case, this becomes
$$
g(\vec{\theta}' | \vec{\theta}) = \frac{1}{\sqrt{2\pi |\det\Sigma|}} \exp\left(-\frac{1}{2}(\vec{\theta}' - \vec{\theta})^T \Sigma^{-1} (\vec{\theta}' - \vec{\theta}) \right)
$$
where $\Sigma$ is the covariance matrix for your generating function.
Choosing the covariance that you use for your generating function is in some sense an art, and there is no right answer. Ideally, you can choose a generating function that preferentially moves you to regions of higher likelihood. The reason for this is economical. Due to the way the MH algorithm works, if you often generate candidate steps that move you to regions of low likelihood, many of these steps will end up being rejected and you will spend a long time in the same location. In some sense these are wasted moves in your chain and we would like to minimize this.
One way to do so would be to pick a covariance matrix that has a similar shape as to the covariance of the posterior distribution. The issue of course is that we don't know the shape of the posterior beforehand - that is what we are running the MCMC to estimate! So instead what we can do is run an iterative process, where we first run a shorter chain with a generating function covariance matrix that is chosen somewhat arbitrarily. Often times this is just an isotropic matrix (diagonal) with the size of your entries chose to try and ensure good coverage of the space.
With this preliminary chain run, you can roughly estimate the shape of the posterior by histograming your chain, and from this create a generating function covariance matrix that is more tuned to the problem at hand. This process can be repeated iteratively if needed.
This more or less addresses your first question. As for the others, I am not exactly sure what you mean as what you are describing doesn't seem to be a standard MH implementation. A few points in particular:

*

*You should not have to recompute covariances as the chain runs. You compute the covariance at the end of the chain by treating the chain as a set of samples from the posterior distribution.


*While you can make proposals separately for each parameter, you can also make a joint proposal where you choose a whole set of new candidate parameters at once. My impression is that this is more common.


*You compute the $\chi^2$ for every proposed point (framed differently, you compute the likelihood). The ratio of likelihoods for the two points enters into the algorithm at decision stage, where you calculate the probability of accepting the move. Moves to higher likelihood are always accepted, whereas moves to lower likelihood are sometimes accepted with a probability proportional to the ratio of likelihoods. This is done to ensure detailed balance, which guarantees that the chain will sample the posterior given enough time.
I might recommend finding some good sources to understand the algorithm better. Numerical Recipes has a good and concise discussion to start.
