Quantcast
Channel: Active questions tagged real-analysis - Mathematics Stack Exchange
Viewing all articles
Browse latest Browse all 9146

Inequality involving entropies: $\left \|p -\frac{1}{n} e \right \|^2_2\ge\left \|\frac{-p\log p}{H(p)} -\frac{1}{n} e \right \|^2_2$

$
0
0

For a given probability vector $p=(p_1,\dots,p_n)$ with $p_1,\dots,p_n > 0, \sum_{i=1}^n p_i=1$, I want to prove the following inequality

$$\left \|p -\frac{1}{n} e \right \|^2_2=\sum_{i=1}^n \left (p_i -\frac{1}{n} \right )^2\ge\left \|-\frac{1}{H(p)}p \log p -\frac{1}{n} e \right \|^2_2=\sum_{i=1}^n \left (\frac{-p_i \log p_i}{-\sum_{i=1}^n p_i\log p_i} -\frac{1}{n} \right )^2,$$

where the equality holds only if all the probabilities $p_i,i=1,\dots,n$ are equal. For $n=2$, the difference of right and left sides is plotted below[1] for $(p,1-p)$:

$\hspace{3cm}$

which shows the inequality holds for $n=2$. I came across with this inequality in my answer to this question [2]. I guess the above inequality holds for any $n\ge2$ (and any $l_p$ norm) based on my numerical experiments for $n=2$.


Viewing all articles
Browse latest Browse all 9146

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>