A/B Testing Models Quiz
5 questions Pass: 70% +25 pts (50 with Pro)
Quiz covering Model Analysis
What you'll be asked
- What is the primary purpose of conducting an A/B test when deploying a new machine learning model?
- In an A/B test setup, why is it critical to ensure that users are randomly assigned to either the control group or the treatment group?
- You are running an A/B test for a recommendation model. Which of the following scenarios best describes a 'novelty effect' that might skew your results?
- When evaluating the results of an A/B test, what does a p-value of 0.03 indicate, assuming a significance level (alpha) of 0.05?
- You observe a statistically significant improvement in click-through rate (CTR) for the treatment group, but notice that the model's latency increased by 200ms. What is the most appropriate next step?
Answers and explanations come after you submit.
Course Content
- Data Cleaning Techniques
- Data Cleaning Techniques Quiz5q
- Feature Scaling Normalization
- Feature Scaling Normalization Quiz5q
- Encoding Techniques
- Encoding Techniques Quiz5q
- Glue DataBrew
- Glue DataBrew Quiz5q
- SageMaker Feature Store
- SageMaker Feature Store Quiz5q
- Ground Truth Labeling
- Ground Truth Labeling Quiz5q
- SageMaker Training Jobs
- SageMaker Training Jobs Quiz5q
- Hyperparameter Tuning
- Hyperparameter Tuning Quiz5q
- Distributed Training
- Distributed Training Quiz5q
- Fine-Tuning Models
- Fine-Tuning Models Quiz5q
- Regularization Techniques
- Regularization Techniques Quiz5q
- Model Registry
- Model Registry Quiz5q
- Ensemble Methods
- Ensemble Methods Quiz5q