LLM-as-a-Judge Techniques Quiz

5 questions Pass: 70% +25 pts (50 with Pro)

Quiz covering GenAI Evaluation

What you'll be asked

  1. What is the primary purpose of using an 'LLM-as-a-Judge' in a GenAI evaluation pipeline?
  2. When designing a prompt for an LLM-as-a-Judge, which technique is most effective at reducing positional bias (the tendency for the judge to prefer the first option)?
  3. Which of the following describes the 'Self-Preference Bias' in LLM-as-a-Judge evaluation?
  4. Why is it recommended to include a 'Chain-of-Thought' (CoT) component in the judge's system prompt?
  5. When calibrating a Judge LLM, how can you quantitatively validate its performance against human gold-standard labels?

Answers and explanations come after you submit.