Solved – Can we change the acceptance rate in random walk Metropolis algorithm by changing the parameter of the proposal distribution

Can we change the acceptance rate in the random walk Metropolis algorithm by changing the parameter of the proposal distribution?

Let the target distribution be $pi$.
Let $p(x_2 | x_1)$ be the proposal density for a new state $x_2$ at current state $x_1$. The acceptance rate is
alpha = min(1, frac{pi(x_2) p(x_1|x_2)}{pi(x_1) p(x_2|x_1)})

If I am correct, in the random walk Metropolis algorithm, the proposal density is symmetric in the sense that $p(x_2 | x_1) = p(x_1 | x_2)$, so the acceptance rate doesn't depend on the proposal density, but only on the target distribution $pi$ to be sampled. So changing the parameter of the proposal distribution will not change the acceptance rate $alpha$.

For example, if the proposal distribution, at current state $x_1$, is a Gaussian distribution centered at the current state with a constant variance, i.e. $N(x_1, sigma^2)$, which is by the way symmetric in the above sense, will changing the variance $sigma^2$ of the Gaussian proposal distribution not change the acceptance rate $alpha$?


If your proposal has very a low variance, then your proposed new state will be very similar to the current state, so $frac{pi(x_2)}{pi(x_1)}$ will be close to 1 (in the limit, with 0 variance, the proposal and current state will be same and you'll have it exactly equal to 1), so acceptance rate will be close to 100%.

If your proposal has high variance, however, $frac{pi(x_2)}{pi(x_1)}$ will (at least sometimes) be way smaller than 1, so your acceptance rate will get closer and closer to 0%. So the acceptance rate decreases as the variance of the proposal increases.

The problem with very low variances (which will get you higher acceptance rate) is that they take longer to explore the posterior space by never moving far away from the current state. Adaptive MCMC methods like Haario et Al. try to handle such problem by changing the variance matrix of the proposal on the fly.

To tune your acceptance rate you could just try increase and decrease the variance, a somewhat trial and error approach. But depending on the geometry of the posterior, the acceptance rate might change drastically during the sampling process. Also, for multiparameter models the proposal covariance matrix has many variance and covariance terms and such method gets impractical.

There is more sofiscated methods to handle this like the adaptative metropolis method outlined in the link above, or you might want to take a look at other methods like those listed here. You may also try software like Jags and Stan if Metropolis doesn't work for your problem.

Similar Posts:

Rate this post

Leave a Comment