Interrogating the design choices of Diffusion Architecture

If you are reading about generative modelling techniques used these days, then no doubt you would have encountered the term Diffusion models which simply to put — convert random dots to images during sampling. There have been many design choices that have been made while writing equations for forward and reverse diffusion and laying down its neural architecture for learning. Although most of the past research works have done full justification to explain each decision choice they made, but still its easy to get lost in the zoo of equations that these papers come with, hindering the attention to be paid to explanation of these choices.

In this blog, I will be jotting down few of the important doubts that may appear in your mind after reading the Diffusion model litrature. If there are any other particular questions you have in mind on related topic, please feel free to comment them to which I will try to respond as quickly as possible.

Questions with Answers

Q1. Why only the noise added at each step is gaussian. Can I use any other probability distribution of noise?

Ans- Adding gaussian noise provides an easy composition of noise among many steps and we can jump to any t’th step in forward diffusion. This makes Gaussian distribution a perfect candidate for…