⚡ Free Classes and Scholarships Available for Underprivileged Students -

Identifying risks to ML solutions

Monitoring ML Solutions

Set 1
10 Questions
No time limit
Practice Mode
0%
Score
0
Correct
0
Incorrect
10
Total Questions
Back to Topics
Question 1 of 10
You deployed a credit scoring model on Vertex AI that performs well on validation data but shows significantly lower approval rates for applicants from certain postal codes. You need to identify whether this indicates bias. What should you do first?
Explanation
Option D (Vertex AI Explainable AI) is correct because you need to first understand which features are driving the different outcomes across postal codes. Explainable AI provides feature attributions that reveal whether postal code or correlated features are inappropriately influencing decisions, helping identify if bias exists before taking corrective action. Option B is incorrect because prediction drift monitoring tracks changes over time but doesn't explain why certain groups receive different outcomes or whether those differences constitute bias. Option A is incorrect because retraining without first understanding the root cause could be premature and might not address the actual issue if legitimate factors correlate with postal codes. Option C is incorrect because differential privacy protects individual data during training but doesn't help identify or explain existing bias in model predictions.
Question 2 of 10
Your team is preparing a loan approval model for production deployment. During bias assessment, you discover the model has equal overall accuracy across demographic groups but different false positive rates. What does this indicate about your model's readiness?
Explanation
Option A (disparate impact requiring fairness intervention) is correct because different false positive rates across demographic groups indicate the model makes systematically different types of errors for different groups, which constitutes disparate impact. In loan approval, this means some groups are unfairly denied loans they would repay, which is both ethically problematic and potentially legally non-compliant. The model is not ready for deployment without addressing this bias. Option D is incorrect because equal overall accuracy masks the different error patterns; fairness requires examining multiple metrics including false positive and false negative rates separately. Option C is incorrect because retraining alone may not solve the problem; you need fairness-aware techniques, threshold adjustment, or other interventions beyond standard retraining. Option B is incorrect because this type of systematic bias is not acceptable variance and monitoring alone won't prevent the harm; the issue must be corrected before deployment.
Question 3 of 10
A production image classification model on Vertex AI is experiencing adversarial attacks where small input perturbations cause misclassification. You need to harden the model against these attacks. What approach should you implement?
Explanation
Option B (adversarial training with perturbed examples) is correct because adversarial attacks exploit model vulnerabilities at the algorithmic level, requiring the model itself to be hardened. Adversarial training exposes the model to perturbed inputs during training, making it more robust to these attacks. Input validation can also catch some obvious manipulations before they reach the model. Option C is incorrect because Private Service Connect controls network-level access but doesn't protect against adversarial inputs from authorized users or services. Option A is incorrect because Cloud Armor protects against network-level attacks like DDoS but cannot detect adversarial perturbations in legitimate-looking ML inputs. Option D is incorrect because while monitoring can detect attacks after they occur, it doesn't prevent the misclassifications or harden the model against the attacks themselves.
Question 4 of 10
You're deploying a healthcare diagnosis model that must comply with strict explainability requirements for regulatory approval. The model needs to provide explanations for each prediction. Which Vertex AI feature should you configure?
Explanation
Option C (Feature Attribution with sampled Shapley) is correct because Vertex AI Feature Attribution provides per-prediction explanations showing how each feature contributed to individual predictions, which is exactly what regulatory explainability requirements demand. Sampled Shapley is one of the available methods that provides theoretically grounded feature importance scores. Option A is incorrect because Model Monitoring tracks distribution changes over time but doesn't provide per-prediction explanations showing why individual predictions were made. Option D is incorrect because TensorBoard visualizes model structure and training behavior but doesn't generate explanations for individual predictions made on production data. Option B is incorrect because logging predictions enables audit trails but doesn't provide the explanations of how features influenced each prediction that regulations require.
Question 5 of 10
Your company's ML model uses customer data that includes sensitive personal information. You need to prevent the model from memorizing and potentially leaking specific training examples. What technique should you apply?
Explanation
Option D (differential privacy during training) is correct because differential privacy adds carefully calibrated noise during training to mathematically guarantee that the model cannot memorize or leak specific training examples, even if an attacker has full access to the model. This directly addresses the risk of training data memorization. Option C is incorrect because encryption protects data at rest and in transit but doesn't prevent the model from memorizing plaintext values during training; the model still sees decrypted data. Option A is incorrect because VPC Service Controls prevent unauthorized network access but don't address the fundamental issue of the model potentially memorizing training data. Option B is incorrect because DLP redaction at inference time doesn't prevent the model from having already memorized sensitive training data; the risk is in what the model learned, not just what it outputs.
Question 6 of 10
You discover that your deployed recommendation model shows lower engagement metrics for content from minority creators. You need to assess whether this represents algorithmic bias. What should you analyze first?
Explanation
Option A (evaluating engagement differences by positioning) is correct because to determine if algorithmic bias exists, you must distinguish between the model systematically ranking minority creator content lower (bias) versus users genuinely preferring other content when fairly presented. If lower engagement results from worse positioning by the algorithm rather than user preference, that indicates bias. Option B is incorrect because while training data imbalance can cause bias, it doesn't directly tell you whether the current performance difference is due to biased rankings or reflects actual user preferences. Option C is incorrect because comparing impressions to clicks doesn't reveal whether the algorithm itself is biased in how it ranks content; impressions might already reflect biased ranking decisions. Option D is incorrect because demographic features influencing scores isn't necessarily bias; the question is whether the algorithm systematically underranks quality content from certain creators regardless of features.
Question 7 of 10
Your fraud detection model on Vertex AI needs protection against model inversion attacks where attackers try to reconstruct training data from model outputs. What security measure addresses this risk?
Explanation
Option B (differential privacy and output precision limits) is correct because model inversion attacks exploit detailed model outputs to reconstruct training data. Differential privacy during training ensures the model doesn't leak specific training examples, while limiting output precision reduces the information available for reconstruction attacks. Both measures directly address the inversion attack vector. Option D is incorrect because rate limiting slows attackers but doesn't prevent model inversion; even limited queries can eventually reveal training data if the model is vulnerable. Option A is incorrect because private endpoints control who can access the model but don't prevent authorized users from performing inversion attacks using the predictions they're allowed to receive. Option C is incorrect because filtering low-confidence predictions doesn't prevent inversion attacks; attackers use the pattern of outputs across many queries, not individual confidence scores.
Question 8 of 10
Before deploying a hiring screening model, you need to assess its readiness using Google's Responsible AI principles. The model performs well technically but you haven't evaluated fairness. What should you do?
Explanation
Option C (evaluate fairness metrics on test data before deployment) is correct because Google's Responsible AI principles require proactive fairness assessment before deployment, not reactive detection after harm occurs. You should evaluate metrics like equal opportunity, demographic parity, and disparate impact across protected groups using existing test data to ensure the model is ready. Option B is incorrect because deploying first and monitoring later violates the principle of preventing harm; fairness issues should be identified and addressed before the model affects real candidates. Option D is incorrect because A/B testing exposes real people to potentially biased decisions before you've established the model is fair, which is ethically problematic in hiring contexts. Option A is incorrect because documentation alone doesn't constitute fairness assessment; you must actually measure fairness metrics, and human review doesn't replace pre-deployment evaluation.
Question 9 of 10
Your model endpoint is receiving inference requests that you suspect are probing for vulnerabilities through edge cases. You need to identify and mitigate this security risk. What approach should you implement?
Explanation
Option D (input validation and anomaly detection) is correct because probing attacks involve sending unusual or malicious inputs to find vulnerabilities. Input validation checks reject obviously invalid requests, while anomaly detection on request patterns identifies systematic probing behavior before inputs reach the model, allowing you to block attacks. Option C is incorrect because audit logging records attacks for forensic analysis but doesn't prevent or mitigate them; the model still processes potentially harmful inputs. Option B is incorrect because IAM controls who can access the endpoint but doesn't prevent authorized users or compromised credentials from sending malicious inputs to probe for vulnerabilities. Option A is incorrect because SSL encryption protects data in transit but doesn't validate or filter the content of requests; encrypted malicious inputs still reach and potentially exploit the model.
Question 10 of 10
You're using Vertex AI Explainable AI on a production model and notice that explanations for similar inputs vary significantly. You need to make explanations more stable while maintaining accuracy. What should you configure?
Explanation
Option B (increase Shapley samples) is correct because when using sampled Shapley values, increasing the number of samples reduces variance in the approximation, making explanations more stable across similar inputs. More samples provide better estimates of each feature's contribution, trading computation time for explanation consistency. Option A is incorrect because while integrated gradients may have different stability characteristics, switching methods doesn't specifically address the instability in the current approach and may introduce other trade-offs; fixing the sampling is more direct. Option C is incorrect because reducing baselines generally decreases explanation quality and can increase instability since you have fewer reference points for comparison. Option D is incorrect because decreasing path count (for integrated gradients) or samples (for Shapley) would actually increase instability and reduce explanation quality by using less information in the computation.