LLM-as-a-Judge Techniques Quiz
5 questions Pass: 70% +25 pts (50 with Pro)
Quiz covering GenAI Evaluation
What you'll be asked
- What is the primary purpose of using an 'LLM-as-a-Judge' in a GenAI evaluation pipeline?
- When designing a prompt for an LLM-as-a-Judge, which technique is most effective at reducing positional bias (the tendency for the judge to prefer the first option)?
- Which of the following describes the 'Self-Preference Bias' in LLM-as-a-Judge evaluation?
- Why is it recommended to include a 'Chain-of-Thought' (CoT) component in the judge's system prompt?
- When calibrating a Judge LLM, how can you quantitatively validate its performance against human gold-standard labels?
Answers and explanations come after you submit.
Course Content