Human Evaluation Quiz

5 questions Pass: 70% +25 pts (50 with Pro)

Quiz covering Model Evaluation

What you'll be asked

  1. What is the primary purpose of using human evaluation for foundation models?
  2. When setting up an A/B test for two different model responses, what is the best practice to minimize rater bias?
  3. Which of the following is a significant drawback of using Likert scales (e.g., 1-5 rating) in human evaluation?
  4. What is the primary benefit of using a 'pairwise comparison' approach over an absolute rating scale?
  5. In a human evaluation study, you notice 'annotator drift' occurring over time. What is the most effective way to address this?

Answers and explanations come after you submit.