Pre-training of Deep Bidirectional Transformers for Language Understanding — BERT

BERT is a language representation model which pre-trains deep bidirectional representations from text by jointly conditioning on context on an unlabeled text corpus for different tasks. It is different from context-free models (word2vec), shallowly bidirectional contextual models (ELMo) and unidirectional contextual models (OpenAI GPT).The motivation…