# Change of variables in calculating the integral of multivariable differential entropy

I have already asked this question in math.SX but here might be more proper. So I decided to put a copy here and delete the one which is not the one that got an answer:

I know that for one dimensional differential entropy of a density function $p(x)$, one has the following formula by change of variables:

$$H(p)=\int p(x)\log(p(x))dx=\int\limits_{0}^{1}\frac{d}{dp}[\log F^{-1}(p)]dp$$.

Where $F$ is the cumulative distribution function of $p$. To extend the idea for multivariate density functions my idea was to try to integrate on level sets of $p(x)$ to simulate somehow what happens in one dimension.

As a results I took a look at this question and topics relevant to pullback method in differential geometry and co-area formulation but couldn't come up with anything by myself. Does anybody know of any method to get a similar formula for differential entropy when $p$ is a multivariable density function?

EDIT: As it has been clarified by "NowIGetToLearnWhatAHeadIs" CDF does not have a unique generalisation. What I thought to be the natural one is:

$$F(x_1,...,x_n)=\int\limits_{-\infty}^{x_1}\int\limits_{-\infty}^{x_2}...\int\limits_{-\infty}^{x_n}p(x_1,...,x_n)dx_ndx_{n-1}...d_2d_1$$

Its unclear to me what you are actually asking. There isn't a generalization of cumulative distribution function to higher dimensions that I know of.

Here is one thing I thought of though. I am not sure if you already found this formula or if this is something you are looking for.

So we have some measure space $X$ with pdf $p$. Then we can define a function $\lambda$ by

$\lambda(p) = \mu(\{x \in X | p(x) < p\}).$

Then we obtain the expression for the entropy $-H = \int p \log p\, \, dx = \int_0^\infty p \log p \lambda'(p) d p .$

Basically this just says that you can look at all the places in $X$ where $p(x)$ is between $p_0$ and $p_0 + dp$. Each of these places gives a contribution of $p_0 \log p_0$ to the integral.

The total contribution is then the contribution from each point times the measure of the set of points contributing; that is, $(p_0 \log p_0) \mu(\{x \in X | p_0 < p(x) < p_0 + dp\})=(p_0 \log p_0) (\lambda(p_0 + dp) -\lambda(p_0 ) ) =(p_0 \log p_0)\lambda'(p_0 ) dp.$

Thus the value of the integral is the sum of the contributions form each interval $(p_0, p_0+dp)$. That is, the integral has the value given above: $\int_0^\infty p \log p \lambda'(p) d p .$

• Thank you very much for this interesting reply. What I see as the natural extension of CDF for high dimensions is: $$F(x_1,...,x_n)=\int\limits_{-\infty}^{x_1}\int\limits_{-\infty}^{x_2}...\int \limits_{-\infty}^{x_n}p(x_1,...,x_n)dx_ndx_{n-1}...d_2d_1$$ What seems a bit strange to me is the fact that in your formulation one cannot get the one dimensional case that I have written above; first of all, $\lambda(p)$ does not coincide with 1-dim CDF and secondly there is no log-derivative expression as it is the case in the formula that I have written. Jun 11, 2014 at 14:09