Human Feedback Collection Quiz

5 questions Pass: 70% +25 pts (50 with Pro)

Quiz covering GenAI Evaluation

What you'll be asked

  1. What is the primary purpose of collecting human feedback in GenAI evaluation?
  2. When designing a Likert scale for human evaluators, what is a best practice to minimize bias?
  3. Which of the following scenarios is best suited for a 'Pairwise Comparison' evaluation method?
  4. Why is 'Inter-Annotator Agreement' (IAA) a critical metric in human evaluation programs?
  5. In a Reinforcement Learning from Human Feedback (RLHF) pipeline, how does 'reward hacking' manifest during the human feedback collection phase?

Answers and explanations come after you submit.