The reparameterization trick is what makes Variational Autoencoders (VAEs) trainable with standard stochastic gradient descent. It works by moving randomness outside the computation graph and turns an awkward gradient of an expectation into an ordinary chain-rule derivative. A VAE is a generative model.

An encoder maps input data x to a distribution over a latent variable z, while a decoder maps a sampled z back to a reconstruction of x. What makes it trainable is its objective, the ELBO (Evidence Lower Bound), i.e., a tractable stand-in for the true data likelihood, made up of a reconstruction term and a term that regularizes the latent distribution toward a simple prior. To train a VAE involves maximizing this ELBO using gradient descent, and in order to do that the gradient of an expectation must be computed; more precisely, the gradient of the expected reconstruction quality with respect to randomly sampled latent variables z must be computed.

This article walks through that problem, the two main families of gradient estimators used to solve it, and why the pathwise gradients obtained through reparameterization tend to have dramatically lower variance than the alternative. A lower-variance gradient estimator is not just a theoretical nicety. In practice, it directly translates to more stable training curves, fewer wild swings in the loss, faster convergence, and better final models.

This is especially critical for complex models like VAEs, Bayesian neural networks, and continuous-control Reinforcement Learning agents, where the training signal can otherwise be too noisy to be useful. The problem: Gradients of expectations -- why sampling breaks backprop Suppose we want to optimize with respect to θ. If θ only appeared inside f, this would be a standard backprop problem.

The complication is that θ parameterizes the distribution that z is drawn from -- the sampling process itself depends on θ -- so we can't just push the gradient through a fixed computation graph. Monte Carlo estimates of L(θ) are easy (draw samples, average f(z)), but Monte Carlo estimates of θ L(θ) are not automatic, because differentiating through a sampling operation isn't well defined. There are two general ways out of this: the score function estimator (REINFORCE) and the pathwise / reparameterization estimator.

Both are unbiased. They differ enormously in variance. The score function estimator (REINFORCE): flexible but high variance The classic trick here is the log-derivative identity: Substituting this into the gradient of the expectation gives which is now an expectation again, so it can be estimated by sampling z ~ p_θ and averaging f(z)·̇θ log p_θ(z).

This is the estimator behind REINFORCE in policy-gradient reinforcement learning, and it's genuinely versatile. It works for discrete z and it doesn't require reparameterizing the distribution.