⚡ Free Classes and Scholarships Available for Underprivileged Students -

Tracking and running ML experiments

Collaborating within and across teams to manage data and models

Set 1
10 Questions
No time limit
Practice Mode
0%
Score
0
Correct
0
Incorrect
10
Total Questions
Back to Topics
Question 1 of 10
You are developing multiple PyTorch models for image classification and need to compare hyperparameter configurations across 50+ training runs. You want to visualize loss curves, compare metrics, and track which hyperparameters produced the best results. What should you use?
Explanation
Option A (Vertex AI Experiments) is correct because it is specifically designed to track parameters, metrics, and artifacts across multiple training runs, enabling comparison of hyperparameters and performance across experiments. It provides a unified interface to compare runs and identify the best configurations. Option B is incorrect because while TensorBoard is excellent for visualizing individual training runs and monitoring real-time metrics, it is not optimized for comparing parameters and metrics across 50+ different experiments systematically. Option C is incorrect because custom logging to Cloud Storage requires building your own comparison tools and lacks the structured experiment tracking and comparison capabilities needed. Option D is incorrect because Kubeflow Pipelines with ML Metadata is designed for tracking pipeline executions and lineage, not for comparing hyperparameter configurations across training experiments.
Question 2 of 10
Your team is training a TensorFlow model and needs to debug training performance issues, including identifying bottlenecks in the data pipeline and profiling GPU utilization. Which tool provides the most comprehensive debugging capabilities for this scenario?
Explanation
Option B (Vertex AI TensorBoard with profiling plugin) is correct because TensorBoard's profiling plugin provides detailed performance analysis including GPU utilization, data pipeline bottlenecks, operation-level profiling, and memory usage visualization, which are essential for debugging training performance issues. Option A is incorrect because Vertex AI Experiments focuses on tracking experiment parameters and metrics for comparison, not on profiling training performance or identifying bottlenecks at the operation level. Option D is incorrect because Cloud Monitoring provides infrastructure-level metrics but lacks the ML-specific profiling capabilities needed to debug data pipeline issues and operation-level performance. Option C is incorrect because Vertex AI Model Monitoring is designed for monitoring deployed models for skew and drift, not for debugging training job performance.
Question 3 of 10
You are building a text-to-image generative model and need to evaluate the quality and diversity of generated images. Your evaluation must consider both semantic alignment with text prompts and the absence of harmful or biased content. What evaluation approach should you implement?
Explanation
Option D (CLIP scores for alignment and human evaluation for safety) is correct because CLIP scores measure semantic alignment between images and text prompts effectively, while human evaluation is the most reliable method for assessing safety, bias, and harmful content in generative outputs. Option A is incorrect because BLEU scores are designed for text-to-text evaluation (comparing generated text to reference text), not for evaluating text-to-image generation quality. Option B is incorrect because perplexity is a metric for language models that measures prediction uncertainty on text sequences, not applicable to evaluating generated images. Option C is incorrect because while FID and structural similarity assess image quality and diversity, they do not evaluate semantic alignment with text prompts or identify harmful/biased content.
Question 4 of 10
Your organization runs hundreds of ML experiments monthly using various frameworks including TensorFlow, PyTorch, and scikit-learn. You need a centralized system to track all experiments, compare results, and maintain reproducibility. Which solution best meets these requirements?
Explanation
Option C (Vertex AI Experiments with framework-agnostic SDK) is correct because it provides a centralized, framework-agnostic experiment tracking system that works with TensorFlow, PyTorch, scikit-learn, and other frameworks through its SDK, enabling comparison and reproducibility across all experiments. Option D is incorrect because TensorBoard is primarily designed for TensorFlow/PyTorch visualization and doesn't provide the same level of framework-agnostic experiment tracking and comparison capabilities needed for scikit-learn and other frameworks. Option B is incorrect because using Cloud Storage with structured naming requires building custom tooling for tracking, comparison, and reproducibility, lacking the built-in experiment management features. Option A is incorrect because Kubeflow Pipelines focuses on workflow orchestration rather than experiment tracking, and would require significant custom development for centralized experiment comparison.
Question 5 of 10
You are evaluating a large language model for customer support and need to assess response quality across dimensions including accuracy, helpfulness, safety, and potential bias. Manual evaluation of all outputs is impractical. What evaluation strategy should you adopt?
Explanation
Option A (automated metrics like ROUGE combined with targeted human evaluation) is correct because it balances scalability with comprehensive quality assessment—automated metrics provide broad coverage for accuracy and helpfulness, while targeted human evaluation addresses safety, bias, and nuanced quality aspects that automated metrics cannot reliably measure. Option B is incorrect because perplexity measures only the model's confidence in predictions, not the actual quality, helpfulness, safety, or bias of generated responses. Option C is incorrect because latency and throughput are operational metrics that don't evaluate response quality, accuracy, safety, or bias. Option D is incorrect because validation loss indicates training progress but doesn't evaluate the specific quality dimensions required (helpfulness, safety, bias) for customer support responses.
Question 6 of 10
Your team needs to visualize embedding projections and analyze model architecture graphs for a complex PyTorch neural network during development. Real-time metric tracking during training is also required. Which tool provides these capabilities most effectively?
Explanation
Option B (Vertex AI TensorBoard integrated with PyTorch) is correct because TensorBoard natively supports embedding projections, computational graph visualization, and real-time metric tracking for PyTorch through the torch.utils.tensorboard module, providing all required capabilities in a single integrated tool. Option C is incorrect because Vertex AI Experiments focuses on tracking parameters and metrics across experiments but doesn't provide native support for embedding projections or architecture graph visualization. Option D is incorrect because Cloud Monitoring is designed for infrastructure and application monitoring, not for ML-specific visualizations like embeddings or neural network architectures. Option A is incorrect because Kubeflow Pipelines is designed for workflow orchestration and would require extensive custom development to provide embedding projections and architecture visualization.
Question 7 of 10
You are comparing different prompt engineering techniques for a generative AI model. You need to track which prompts, temperature settings, and output formats produce the best results across 200 test cases. What is the most appropriate tool for this task?
Explanation
Option C (Vertex AI Experiments) is correct because it is designed to track parameters (prompts, temperature, formats), log metrics, and compare results across multiple runs, making it ideal for systematic prompt engineering experimentation and comparison. Option A is incorrect because TensorBoard is optimized for visualizing training metrics and model internals, not for tracking and comparing discrete prompt variations and their associated parameters across test cases. Option B is incorrect because Vertex AI Feature Store is designed for storing, versioning, and serving features for model training and inference, not for tracking experimental results from prompt testing. Option D is incorrect because Vertex AI Pipelines is designed for orchestrating ML workflows, not for tracking and comparing experiment parameters and results across different configurations.
Question 8 of 10
Your organization is developing a code generation model and needs to evaluate output quality. The evaluation must measure functional correctness, code efficiency, and adherence to security best practices. Which evaluation approach is most appropriate?
Explanation
Option D (execute against test suites and apply static analysis) is correct because test execution validates functional correctness, while static analysis tools can assess efficiency and security practices, providing comprehensive automated evaluation of the key quality dimensions. Option A is incorrect because BLEU scores measure text similarity and don't evaluate whether code actually works, is efficient, or follows security best practices, which are the stated requirements. Option C is incorrect because confidence scores and perplexity measure the model's certainty, not the actual correctness, efficiency, or security of the generated code. Option B is incorrect because while human evaluation is valuable, relying solely on it without automation is impractical and inefficient when automated testing and static analysis can objectively measure correctness and security.
Question 9 of 10
You are training a TensorFlow model using Vertex AI Training and want to monitor training progress, compare multiple concurrent runs, and share visualizations with your team. The solution must require minimal code changes. What should you implement?
Explanation
Option C (enable Vertex AI TensorBoard in training configuration) is correct because Vertex AI Training has built-in TensorBoard integration that requires only configuration parameters, automatically captures TensorFlow metrics, supports multiple concurrent runs, and provides shareable visualizations with minimal code changes. Option D is incorrect because instrumenting code for Vertex AI Experiments requires adding explicit logging code throughout the training script, which is more code changes than simply enabling TensorBoard integration. Option A is incorrect because exporting to BigQuery and building Data Studio dashboards requires significant custom development and code changes, not the minimal changes specified. Option B is incorrect because Cloud Logging is designed for application logging and would require substantial custom code to capture training metrics and build visualization dashboards.
Question 10 of 10
You are evaluating a generative AI chatbot for factual accuracy and creativity. Some responses require strict factual correctness while others need creative interpretation. How should you design your evaluation framework to address both requirements?
Explanation
Option A (different criteria based on query intent) is correct because factual queries and creative queries have different success criteria—factual questions require accuracy verification while creative requests need evaluation for originality and usefulness, necessitating intent-based evaluation approaches. Option B is incorrect because automated fact-checking doesn't evaluate creative quality, and applying only factual evaluation to creative responses would miss the stated requirement to assess creativity. Option D is incorrect because measuring only creativity across all interactions ignores the stated requirement for strict factual correctness in some responses. Option C is incorrect because a single balanced metric cannot adequately evaluate both strict factual accuracy and creative interpretation, which are often opposing objectives requiring different evaluation approaches.