An Overview of Attention and Transformer Mechanisms for NLP

Nisreen explained the technical aspects of attention and self attention mechanisms, as well as explored how attention is used in the transformer architecture in order to aid in machine translation tasks.

Supplemental Resources

Neural Machine Translation by Jointly Learning to Align and Translate, Bahdanau et al.
Attention is all you Need, Vaswani et al.