RLHF Fine-Tuning Quiz

5 questions Pass: 70% +25 pts (50 with Pro)

Quiz covering Advanced Training

What you'll be asked

  1. What is the primary purpose of Reinforcement Learning from Human Feedback (RLHF) in training Large Language Models?
  2. In the RLHF pipeline, what is the role of the Reward Model?
  3. Which algorithm is most commonly used to update the policy model during the RLHF phase to prevent the model from deviating too far from the original supervised fine-tuned model?
  4. Why is a KL-divergence penalty typically included in the PPO objective function during RLHF?
  5. When training a Reward Model, why is it common practice to use pairwise comparisons (A is better than B) rather than absolute scalar ratings (1-5 stars)?

Answers and explanations come after you submit.