Human Evaluation Quiz
5 questions Pass: 70% +25 pts (50 with Pro)
Quiz covering Model Evaluation
What you'll be asked
- What is the primary purpose of using human evaluation for foundation models?
- When setting up an A/B test for two different model responses, what is the best practice to minimize rater bias?
- Which of the following is a significant drawback of using Likert scales (e.g., 1-5 rating) in human evaluation?
- What is the primary benefit of using a 'pairwise comparison' approach over an absolute rating scale?
- In a human evaluation study, you notice 'annotator drift' occurring over time. What is the most effective way to address this?
Answers and explanations come after you submit.
Course Content