Narrating the story of Latent Diffusion Model

Diffusion Models(DMs) are quiet popular since past some time since the quality of generated images is simply amazing beating other generative models such as GANs, VAEs, Autoregressive models and others. But the bottle neck of using DM, the sampling speed aka inference time to produce samples is worth considering. Dealing with the same and producing more high quality images, Stability AI company came up with the idea of Latent Diffusion Model which was accepted in CVPR conference 2022 and later the pre-trained model weights were released to public for constructive purposes. In this blogpost I will be talking more about LDM from architecture point of view and will mention few comparisons with DM.

The applications of image synthesis span from medical imaging, robotics, sensing, digital heritage to industrial applications. In past one decade or more, research community has focused on this problem from various perspectives and have come up with vivid architectures or problem solving styles to generate novel Image content. Latest trend has been to generate images conditioned on some extra information such as Image captions, scene description text, semantic maps, reference images or other data representation.

The techniques such as Generative Adversarial networks (GAN), Variational Auto-encoder (VAE), Auto-regressive Transformer (ART), Flow based…