01
Modeling a Distribution
A generative model attempts to capture statistical structure in observed data so that new samples can be generated from the learned model.
x ~ p_data(x)
Learning constructs an approximation p_θ(x) parameterized by θ. The quality of a generative model therefore depends on how well its learned distribution captures meaningful structure while remaining useful for sampling.
02
Sampling
Sampling transforms a learned distribution into concrete outputs. The exact procedure differs by model family: autoregressive models sample token by token, diffusion models iteratively denoise, and latent-variable models sample latent representations before decoding them.