⚡ Free Classes and Scholarships Available for Underprivileged Students -

Automating model retraining

Automating and Orchestrating ML Pipelines

Set 1
10 Questions
No time limit
Practice Mode
0%
Score
0
Correct
0
Incorrect
10
Total Questions
Back to Topics
Question 1 of 10
Your company deployed a product recommendation model six months ago that now shows declining prediction accuracy. Users report receiving irrelevant recommendations. Model performance metrics indicate a 15% drop in precision. What is the most likely cause requiring a retraining strategy?
Explanation
Option B (Model drift due to changing user preferences and catalog updates over time) is correct because the scenario describes declining prediction accuracy and relevance issues, which are classic symptoms of model drift. When user behavior patterns change or the product catalog evolves, the model's learned patterns become stale, requiring retraining with fresh data. Option A is incorrect because infrastructure degradation affects response time, not prediction accuracy or relevance. Option D is incorrect because memory issues would cause errors or crashes, not a gradual decline in recommendation quality. Option C is incorrect because network latency affects speed, not the quality of predictions being made.
Question 2 of 10
You need to establish a retraining policy for a fraud detection model serving a financial application. The model must maintain high accuracy as fraud patterns evolve rapidly. Which approach best balances model performance with operational efficiency?
Explanation
Option A (Schedule weekly retraining with continuous monitoring of model performance metrics) is correct because it provides regular updates to capture evolving fraud patterns while monitoring allows for additional retraining if performance degrades between scheduled runs. This balances keeping current with fraud trends and operational costs. Option D is incorrect because waiting for customer complaints is reactive and allows fraud to succeed before action is taken. Option B is incorrect because daily retraining is operationally expensive and unnecessary unless data shows daily pattern changes. Option C is incorrect because quarterly retraining is too infrequent for rapidly evolving fraud patterns, allowing significant performance degradation.
Question 3 of 10
You are implementing a CI/CD pipeline for ML model deployment using Cloud Build. The pipeline must build a custom container, run validation tests, and deploy to Vertex AI. Which Cloud Build configuration component defines the container image for model serving?
Explanation
Option C (The steps section using a docker build command targeting the Dockerfile) is correct because the steps section contains the build instructions, and the docker build command creates the container image from the Dockerfile that packages the model and serving code. Option D is incorrect because substitutions define variables for parameterizing builds, not the container image itself. Option B is incorrect because timeout sets execution limits and doesn't define what to build. Option A is incorrect because the artifacts section specifies which files to save after the build completes, not how to build the container.
Question 4 of 10
Your team needs to trigger model retraining when prediction performance drops below acceptable thresholds. The solution must automatically detect performance degradation and initiate retraining workflows. Which GCP service combination best implements this requirement?
Explanation
Option D (Cloud Monitoring alerts triggering Cloud Functions to start Vertex AI pipelines) is correct because Cloud Monitoring can track model performance metrics and create alerts based on thresholds, Cloud Functions provides serverless execution to respond to alerts, and Vertex AI Pipelines orchestrates the retraining workflow. Option A is incorrect because Cloud Scheduler is time-based (not metric-based) and BigQuery queries don't execute retraining pipelines. Option B is incorrect because Cloud Logging is for logs (not metrics), and App Engine cron jobs are scheduled, not event-driven. Option C is incorrect because Cloud Tasks manages task queues and Dataflow is for data processing, neither handles model retraining orchestration based on performance metrics.
Question 5 of 10
You are designing a retraining strategy for a demand forecasting model. Historical analysis shows that forecast accuracy correlates with the amount of recent data used for training. Which retraining trigger strategy would most effectively maintain model accuracy?
Explanation
Option C (Time-based retraining monthly with sliding window of recent three months data) is correct because it ensures regular updates with fresh recent data, which the scenario states correlates with accuracy. The sliding window keeps training data current and relevant for forecasting. Option D is incorrect because total dataset growth doesn't ensure recent data is incorporated; old data accumulation would trigger retraining unnecessarily. Option A is incorrect because waiting for performance degradation is reactive and allows accuracy to decline before action is taken. Option B is incorrect because manual quarterly reviews introduce delays and don't provide the regular updates needed to maintain accuracy with recent data.
Question 6 of 10
Your ML team uses Cloud Build for CI/CD and needs to deploy models to Vertex AI only after passing validation tests on a holdout dataset. The deployment must be automated but conditional. How should you configure the Cloud Build pipeline?
Explanation
Option B (Add a validation step that runs tests and conditionally executes deployment steps) is correct because Cloud Build supports sequential steps with conditional execution based on previous step success, allowing validation tests to gate deployment. This ensures only validated models reach production automatically. Option D is incorrect because deploying before validation violates the requirement for conditional deployment and introduces risk of serving invalid models. Option C is incorrect because running validation after deployment means untested models are briefly in production, which doesn't meet the requirement. Option A is incorrect because skipping automated testing entirely removes the validation requirement and relies on detecting problems in production.
Question 7 of 10
You need to implement continuous training for a sentiment analysis model that processes customer reviews. New reviews arrive continuously throughout the day. What retraining frequency would be most appropriate for this scenario?
Explanation
Option A (Daily retraining during off-peak hours using previous day's accumulated reviews) is correct because it balances capturing evolving sentiment patterns with operational efficiency. Daily batching provides sufficient new training data while off-peak scheduling minimizes resource contention. Option B is incorrect because retraining after each review is computationally wasteful and impractical; single reviews don't provide meaningful pattern changes. Option D is incorrect because hourly retraining is unnecessarily frequent for sentiment patterns and increases costs without proportional accuracy gains. Option C is incorrect because monthly retraining is too infrequent to capture evolving customer sentiment trends effectively.
Question 8 of 10
Your organization requires that all ML model deployments follow a CI/CD process with code review, automated testing, and approval gates. You're using Cloud Build. Which approach best implements these requirements?
Explanation
Option D (Integrate Cloud Build with Cloud Source Repositories using triggers and build approvals) is correct because this combination provides version control for code review, automated build triggers for testing, and manual approval steps before deployment, fulfilling all stated requirements. Option A is incorrect because immediate automatic deployment bypasses the required approval gates. Option C is incorrect because manual deployment defeats the CI/CD automation requirement and introduces inconsistency. Option B is incorrect because scheduled copying doesn't implement code review, testing, or approval processes required by the organization.
Question 9 of 10
You need to retrain a customer churn prediction model when concept drift is detected. The system must compare current prediction distributions with training data distributions. Which metric-based approach best identifies this type of drift?
Explanation
Option D (Monitor prediction distribution divergence using statistical distance metrics like KL divergence) is correct because concept drift manifests as changes in the relationship between features and target, which appears as shifts in prediction distributions. KL divergence and similar metrics quantify these distribution changes. Option C is incorrect because accuracy alone requires labeled ground truth and may lag behind drift detection; it doesn't directly measure distribution changes. Option A is incorrect because latency measures performance speed, not concept drift or prediction quality. Option B is incorrect because prediction volume counts don't indicate whether the model's predictions remain accurate or whether distributions have shifted.
Question 10 of 10
Your team manages multiple ML models in production and needs to automate retraining across all models. Each model has different retraining requirements and schedules. Which GCP service provides the best orchestration solution?
Explanation
Option B (Vertex AI Pipelines with parameterized workflows for each model's requirements) is correct because Vertex AI Pipelines provides ML-specific orchestration with reusable components, metadata tracking, and parameterization to handle different model requirements within a unified framework. Option C is incorrect because Cloud Scheduler only handles scheduling, not orchestration, dependency management, or ML-specific features like experiment tracking. Option A is incorrect because individual Cloud Functions create management overhead and lack built-in ML pipeline features like artifact lineage and component reuse. Option D is incorrect because Dataflow is designed for data processing, not ML workflow orchestration, and doesn't provide model training orchestration capabilities.