Transformer Architecture Quiz

5 questions Pass: 70% +25 pts (50 with Pro)

Quiz covering Transformers and Attention

What you'll be asked

  1. What is the primary purpose of the 'Self-Attention' mechanism in a Transformer model?
  2. In the original Transformer architecture, why are 'Positional Encodings' added to the input embeddings?
  3. What is the function of the 'Feed-Forward Network' (FFN) that follows the Multi-Head Attention layer in each Transformer block?
  4. When training a Transformer, what is the role of the 'Mask' in the decoder's masked self-attention layer?
  5. How does increasing the number of 'Attention Heads' in Multi-Head Attention affect the model's learning capacity?

Answers and explanations come after you submit.