Attention Mechanisms Quiz

5 questions Pass: 70% +25 pts (50 with Pro)

Quiz covering Transformers and Attention

What you'll be asked

  1. What is the primary purpose of the 'Attention' mechanism in a Transformer model?
  2. In the context of Self-Attention, what do the Query (Q), Key (K), and Value (V) vectors represent?
  3. Why is 'Multi-Head Attention' preferred over a single 'Attention' head in Transformer architectures?
  4. What is the role of the 'Scaled Dot-Product' in the attention calculation?
  5. In a Transformer's decoder, why is 'Masked Multi-Head Attention' used instead of standard multi-head attention?

Answers and explanations come after you submit.