3
$\begingroup$

Suppose $p$ and $q$ are discrete distributions over $k$ outcomes, $q$ is given, for which $p$ does the following hold?

\begin{equation*} p_1 \log q_1 + \ldots + p_k \log q_k \ge q_1 \log q_1 + \ldots + q_k \log q_k\end{equation*}

2 Answers 2

4

p such that:

$$\prod_i q_i^{p_i-q_i} \ge 1$$

I arrive at this point with the following calculations:

First of all I rewrite the equation:

$$\sum_i p_i \log q_i \ge \sum_i q_i \log q_i$$

And

$$\sum_i (p_i \log q_i - q_i \log q_i) = \sum_i (\log q_i^{p_i-q_i}) = \log \prod_i q_i^{p_i-q_i} \ge 0$$

The most simple deduction about p is that

$$p_i \ge q_i , \forall i$$

with

$$q_i \ge 1$$

  • 0
    Welcome to math.SE! We support TeX equations here, but they have to be surrounded by dollar signs. See http://meta.math.stackexchange.com/questions/107/what-should-go-in-the-math-stackexchange-faq/117#117 for detail.2010-08-20
0

$p$ and $q$ are distributions, so $q_i\ge1$ is not possible. Further, since both sum to 1, one distribution cannot be strictly smaller than the other. You could investigate how it behaves for a two-state space before drawing more general conclusions.