Plato’s Allegory of Generative Models

This blogpost is about general understanding of generative modelling through a philosophical analogy.

Great philosopher of Greek, Plato, in his famous work “Republic” compares the effect of education and lack of it on our nature. This allegory is commonly known by the name of “Plato’s Allegory of the Cave”. Let’s see it in detail but only after discussing about generative models.

Generative Modelling

Given observed samples x from a distribution of interest, the goal of a generative model is to learn to model its true data distribution p(x). Once learned, we can generate new samples from our approximate model at will. We are able to use the learned model to evaluate the likelihood of observed or sampled data as well.

There are several well-known directions in current literature:-

• Generative Adversarial Networks (GANs) model the sampling procedure of a complex distribution, which is learned in an adversarial manner.
• Likelihood based models seeks to learn a model that assigns a high likelihood to the observed data samples. This includes autoregressive models, normalizing flows, Variational Auto-encoders (VAEs) and Diffusion Models which are kind of Hierarchal VAEs.

For many modalities, we can think of the data we observe as represented or generated by an associated unseen latent variable, which we can denote by random variable z.