Looking for a proof of the expected value of the score function equating zero, I came to this document that was recommended in another answer.
Considering that we have a sample of $n$ $x_i$ values, I cannot figure out why the expected value turns out into an integral instead of a summation: what is the curve of which we're taking the area below it? In my mind I can just see some specific points in a graph, with no area under it, since we have a finite and discrete number of datapoints.
I do understand that the integral is crucial for the proof as to be interchanged with the derivative and then using the pdf probabilities for equating it to 1. But I would not know how to apply all this into a pmf or discrete case.
Thanks in advance
Best Answer
$X_i$ is continuous a random variable, with pdf $f_{X_i}(x_i;theta)$, and the expectation requires an integral. The integral limits contain the domain of $X_i$. Not $i$ from $1$ to $n$. The $n$ samples you have are just realizations of $X_i$, i.e. $X_1,X_2,…,X_n$. You're not integrating/summing across these variables. You're integrating for a particular $i$, let's say $X_2$, and obtain an expression for the expected value of interest.
Similar Posts:
- Solved – Derivation of EPE in “The elements of statistical learning ”
- Solved – Derivation of EPE in “The elements of statistical learning ”
- Solved – Law of iterated expectations
- Solved – Total area under any probability density function
- Solved – Second order moment of multivariate Gaussian (bishop p. 83)