1
$\begingroup$

I need to show that $-\sum_i{p_i \log{p_i}}$ is maximal iff $p_i=p_j$ for all $i\neq j$ using the convexity inequality:

$\phi (\frac{\sum{a_i}}{N})\leq \frac{\sum{\phi (a_i)}}{N}$

I tried expanding with cross-entropy and KL-divergence

$-\sum_i{\frac{1}{N} \log{\frac{1}{N}}}=-\sum_i{\frac{1}{N} \log{p_i}}-\sum_i{\frac{1}{N} \log{\frac{N}{p_i}}}$

But I got only trivial answers like $\log N \leq \log N$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.