A simple Sequential Monte Carlo algorithm for posterior approximation

August 6, 2021
algorithm, SMC

Introduction # Sequential Monte Carlo (SMC) can be used to take samples from posterior distributions, as an alternative to popular methods such as Markov Chain Monte Carlo (MCMC) like Metropolis-Hastings, Gibbs Sampling or more state-of-the-art Hamiltonian Monte Carlo (HMC) and No-U-Turn Sampler (NUTS). Instead of creating a single Markov Chain, SMC works by keeping track of a large population of samples, which, in a similar spirit to Genetic Algorithms can be “selected” and “mutated” to evolve a better “fit” to the posterior distribution. ...

A most unprincipled derivation of the gamma distribution

July 13, 2021
gamma, pdf, cdf

Introduction # In this article I will derive the gamma distribution in a most unprincipled way. Why? First, there are many good resources out there explaining how to derive the gamma distribution from first principles, usually involving these idealized things called poisson processes. These are great sources and I would definitely recommend them. This one by Aerin Kim is amazing. However, more often than not, people use gamma distributions in real world problems for much more mundane reasons: It’s a well behaved yet flexible positive continuous distribution. ...