Solved – How to sample from a log transformed distribution using uniform distribution

I am transforming an unscaled density function to log scale to avoid underflow issues.

BI was performing integration on this function on a grid of values before I used the log transormation, to build a grid based cumulative distribution function. Then, using a draw from uniform[0,1], I was choosing the the largest point on the grid which had a cumulative probability value smaller than the draw from the uniform. This worked fine as long as the univariate density function could be integrated.

With the transform to log-scale, I can't really get my head around this mechanism. The joint likelihood is so small that I can't back transform the density, so I have to perform the same uniform distribution based sampling on the log scale. Is this an established practice? Feedback and pointers would be appreciated.

As I understand it, you've generally discretized to create a set of $n$ points, $x_1, dots, x_n$, with probability $p_1, dots, p_n$, and you then calculate the cumulative probabilities, say $c_i = sum_{j=1}^i p_j$. So you can draw $U sim Uniform(0,1)$ and then take $X = x_{i^*}$ where $i^* = min_i {i:c_i ge U}$, or something like that.

But your current problem is that the $p_i$ are so small that you want to just work with $a_i = log p_i$.

One approach would be to sort the $a_i$ from largest to smallest and then calculate partial sums using something like the following addlog function, which calculates $log(f + g)$ on the basis of $a = log(f)$ and $b = log(g)$.

addlog(a, b, THRESH=200.0) {   if(b > a + THRESH) return(b);   else if(a > b + THRESH) return(a);   else return(a + log1p(exp(b-a))); } 

where log1p(x) returns log(1+x).

But, really, I would think that you should focus on the $x_i$ for which $p_i$ is large enough that you don't need to worry about underflow, and neglect the $x_i$ with exceedingly small $p_i$. If all of the $p_i$ are small, then it seems that you should grid more coarsely. In most applications, it should be sufficient to discretize to 1000 or so values, I would think.

Similar Posts:

Rate this post

Leave a Comment