1
$\begingroup$

I'm taking a course on machine learning and came across a model answer which I can't wrap my head around.

The whole model answer is the 2nd exercise here: https://noppa.tkk.fi/noppa/kurssi/t-61.3050/viikkoharjoitukset/T-61_3050_proposed_solutions.pdf

The part I'm specifically having trouble understanding is the last 2 lines of the 2nd page, ie this: http://i.stack.imgur.com/SZEPC.png

How does the left side of the upper equation become just the sum of indicator variables? How does the right side simplify to just lambda*p_i?

Any help is appreciated.

1 Answers 1

2

I say there be typos. Particular the suddenly appearing subscript $k$ does not make sense.

Straightforward differentiation gives: $${\partial \over {\partial p_i}} L(p|X)={1 \over p_i}\sum_{t=1}^N x^t_i + \lambda$$ Setting this equal to zero gives us: $$\sum_{t=1}^N x^t_i + \lambda p_i=0$$ Proceeding from which and reading the subscript $k$ in your text as $i$ leads to the same ultimate answer.

  • 0
    Thanks for the answer, but it's still a bit unclear to me how differentiating with respect to p_i, (sum(i=1...K)(p_i) equals one?2010-10-15
  • 1
    $\partial \log p_j/\partial p_i=0$ when $j \neq i$, so all terms other than the i-th disappear.2010-10-15
  • 0
    Ahh now I see it, thank you.2010-10-15