Evaluation Metrics Quiz
5 questions Pass: 70% +25 pts (50 with Pro)
Quiz covering Model Evaluation
What you'll be asked
- Which evaluation metric is commonly used to measure the similarity between a model-generated summary and a reference summary based on n-gram overlap?
- When evaluating a generative model using Perplexity (PPL), what does a lower score indicate?
- Why is it often insufficient to rely solely on automated metrics like BLEU or ROUGE for evaluating large language models (LLMs) in creative writing tasks?
- You are evaluating a classifier for a highly imbalanced dataset where the positive class is rare. Which metric would provide the most misleading assessment of model performance?
- In the context of 'LLM-as-a-judge', what is the primary risk associated with using a stronger model (e.g., GPT-4) to evaluate the outputs of a smaller model?
Answers and explanations come after you submit.
Course Content