Model Debugger Quiz

5 questions Pass: 70% +25 pts (50 with Pro)

Quiz covering Model Analysis

What you'll be asked

  1. Which of the following metrics is most useful for identifying if a classification model is failing on a specific minority class?
  2. You notice your model performs exceptionally well on the training set but poorly on the validation set. What is the most likely cause?
  3. When performing error analysis, you discover the model consistently misclassifies images of cats as dogs only in low-lighting conditions. What should be your next step?
  4. If a model's feature importance scores (e.g., via SHAP or Permutation Importance) reveal that a target-related variable is the strongest predictor by a massive margin, what should you immediately investigate?
  5. During model debugging, you use Partial Dependence Plots (PDPs). What specific insight do these plots provide that a global feature importance metric does not?

Answers and explanations come after you submit.