⚡ Free Classes and Scholarships Available for Underprivileged Students -

Tracking and auditing metadata

Automating and Orchestrating ML Pipelines

Set 1
10 Questions
No time limit
Practice Mode
0%
Score
0
Correct
0
Incorrect
10
Total Questions
Back to Topics
Question 1 of 10
You are training multiple versions of a classification model with different hyperparameters in Vertex AI. You need to compare metrics like precision, recall, and F1-score across all training runs to identify the best performing configuration. What should you use?
Explanation
Option B (Vertex AI Experiments) is correct because it is specifically designed for tracking, visualizing, and comparing metrics and parameters across multiple training runs. It provides built-in visualization tools to compare hyperparameters and performance metrics side-by-side, making it ideal for this use case. Option A (Vertex ML Metadata) is incorrect because while it tracks lineage and metadata about ML workflows, it focuses on artifact provenance and execution context rather than providing comparison tools for metrics across experiments. Option C (Cloud Logging) is incorrect because it's a general-purpose logging service not designed for structured ML experiment tracking and comparison; analyzing metrics would require manual log parsing and custom visualization. Option D (BigQuery) is incorrect because while you could store and query metrics, it requires significant custom development to build comparison dashboards and lacks the ML-specific features and integrations that Vertex AI Experiments provides out of the box.
Question 2 of 10
Your team needs to track the complete lineage of a deployed model, including the training dataset, preprocessing steps, training code, and hyperparameters used. The solution must integrate with Vertex AI Pipelines. Which approach should you implement?
Explanation
Option D (Vertex ML Metadata) is correct because it is the native metadata store for Vertex AI that automatically captures lineage when using Vertex AI Pipelines. It tracks artifacts, executions, and contexts, creating a complete lineage graph without requiring custom code. Option C (Cloud Storage with naming conventions) is incorrect because it only provides a file organization strategy without capturing actual lineage relationships, execution metadata, or the ability to query dependencies programmatically. Option B (BigQuery tables) is incorrect because it would require significant custom implementation to track all relationships and doesn't integrate natively with Vertex AI Pipelines to automatically capture lineage. Option A (Cloud Firestore) is incorrect because it's a general-purpose NoSQL database that would require building a complete custom lineage tracking system from scratch, duplicating functionality that Vertex ML Metadata already provides.
Question 3 of 10
You deployed a model to production six months ago. Compliance requirements now mandate that you prove which exact training dataset was used for that model. You need to implement dataset versioning going forward. What should you do?
Explanation
Option A (Vertex AI Managed Datasets with version labels) is correct because it provides native versioning capabilities integrated with Vertex ML Metadata, automatically tracking dataset versions and their relationships to training jobs and models through lineage. Option C (Cloud Storage with timestamp folders) is incorrect because while it creates separate copies, it doesn't provide formal versioning, doesn't integrate with ML Metadata for lineage tracking, and requires manual management of version relationships. Option D (BigQuery snapshots with date suffixes) is incorrect because it's limited to BigQuery data sources only, requires manual snapshot management, and doesn't automatically integrate with Vertex AI's lineage tracking system. Option B (Cloud Storage object metadata) is incorrect because object-level tagging doesn't constitute true versioning, doesn't prevent accidental overwrites or deletions, and lacks integration with Vertex AI's metadata and lineage tracking capabilities.
Question 4 of 10
Your ML pipeline generates multiple model artifacts during experimentation, including saved models, evaluation metrics, and visualizations. You need to organize and version these artifacts to enable reproducibility. Which approach provides the most comprehensive solution?
Explanation
Option C (Vertex AI Experiments) is correct because it provides a unified interface to log, version, and organize all types of ML artifacts (models, metrics, visualizations) while automatically linking them to specific experiment runs, parameters, and metadata for complete reproducibility. Option D (Cloud Storage with Git hashes) is incorrect because while it provides some traceability, it lacks structured metadata storage, doesn't capture metrics or parameters, and requires custom tooling to retrieve and compare artifacts across experiments. Option A (Cloud SQL with file references) is incorrect because it requires building a custom artifact management system, doesn't integrate with Vertex AI services, and lacks ML-specific features like automatic metric tracking and visualization. Option B (Cloud Source Repositories) is incorrect because it's designed for source code versioning, not for storing or versioning binary ML artifacts like models and large datasets, and would result in repository bloat.
Question 5 of 10
You are running a Vertex AI Pipeline that preprocesses data, trains a model, and deploys it. You need to query which preprocessing execution produced the dataset used by a specific deployed model. What should you use?
Explanation
Option D (Vertex ML Metadata) is correct because it maintains a graph of artifacts, executions, and contexts that allows querying lineage relationships directly, enabling you to trace from a deployed model back through its training dataset to the preprocessing execution that created it. Option B (Cloud Logging) is incorrect because while it captures execution logs, it doesn't maintain structured lineage relationships between artifacts, making it extremely difficult to trace dependencies without custom parsing and correlation logic. Option C (BigQuery audit logs) is incorrect because audit logs track access and administrative actions but don't contain the artifact lineage graph needed to trace which preprocessing execution produced a specific dataset used by a model. Option A (Vertex AI Experiments) is incorrect because while it tracks experiment runs and their parameters, it's focused on comparing training experiments rather than maintaining execution-level lineage across pipeline components.
Question 6 of 10
Your organization requires that all model training runs be auditable, including who initiated the training, when it occurred, and what configuration was used. You need to implement this with minimal operational overhead. What should you do?
Explanation
Option B (Vertex AI Experiments) is correct because it automatically captures run metadata including user identity, timestamps, parameters, and configurations with minimal code changes, providing a purpose-built audit trail for ML training runs. Option C (custom Cloud Logging) is incorrect because it requires significant development effort to instrument every training script, maintain consistent logging formats, and capture all relevant metadata that Vertex AI Experiments provides automatically. Option A (Cloud Audit Logs) is incorrect because while it captures API-level activity, it doesn't log the ML-specific metadata like hyperparameters, metrics, and artifacts that are needed for comprehensive training run auditability. Option D (Cloud Spanner) is incorrect because it requires building a complete custom audit system from scratch with significant operational overhead, duplicating functionality already provided by Vertex AI Experiments.
Question 7 of 10
You need to compare the performance of 50 different model variants trained with various feature sets and hyperparameters. The comparison should include custom visualizations of confusion matrices and ROC curves. Which solution provides this capability most effectively?
Explanation
Option A (Vertex AI Experiments) is correct because it natively supports logging both standard metrics and custom visualizations (including confusion matrices and ROC curves), provides built-in comparison views across experiments, and integrates directly with training code through the Vertex AI SDK. Option B (Data Studio) is incorrect because it's a general BI tool not designed for ML-specific visualizations like confusion matrices and ROC curves, and would require extensive custom development to replicate experiment comparison features. Option C (BigQuery and Jupyter) is incorrect because while it provides flexibility, it requires significant manual effort to build comparison tools, lacks centralized experiment tracking, and doesn't provide a unified interface for team collaboration. Option D (Cloud Functions and Cloud Storage) is incorrect because it would require building a complete custom visualization and comparison system from scratch, creating unnecessary operational overhead and maintenance burden.
Question 8 of 10
Your team is implementing a retraining pipeline that runs weekly. You need to ensure that for any model version in production, you can identify the exact code version, dataset version, and hyperparameters that produced it. What is the most efficient approach?
Explanation
Option C (Vertex ML Metadata within pipelines) is correct because it automatically captures and maintains the complete lineage graph linking models to their training executions, datasets, code versions, and hyperparameters when using Vertex AI Pipelines, with no additional code required. Option A (configuration files in Git) is incorrect because while it versions the configuration, it doesn't automatically link configurations to specific model artifacts or training executions, requiring manual correlation to trace a deployed model back to its training details. Option D (Firestore documents) is incorrect because it requires custom code to create and maintain all relationships manually, duplicating functionality already provided by Vertex ML Metadata, and doesn't integrate with Vertex AI services. Option B (BigQuery tables) is incorrect because while it can store the information, it requires building custom tracking logic, doesn't provide automatic lineage capture, and lacks the graph-based relationship model needed for efficient lineage queries.
Question 9 of 10
You trained multiple models using Vertex AI custom training jobs with different frameworks (TensorFlow, PyTorch, XGBoost). You need a unified way to track and compare all model versions regardless of framework. What should you implement?
Explanation
Option D (Vertex AI Model Registry) is correct because it provides framework-agnostic model versioning and management, allowing you to register, version, and track models from any framework in a single unified system integrated with Vertex ML Metadata. Option B (Cloud Storage with folder structures) is incorrect because it only provides file organization without versioning capabilities, metadata tracking, or the ability to query and compare models across frameworks in a unified way. Option A (separate BigQuery systems per framework) is incorrect because it creates fragmented tracking, requires maintaining multiple custom solutions, and makes cross-framework comparison difficult without additional integration work. Option C (framework-specific native registries) is incorrect because it creates silos, lacks unified tracking across frameworks, doesn't integrate with Vertex AI's lineage and metadata systems, and makes enterprise-wide model governance more complex.
Question 10 of 10
Your data science team needs to reproduce a model that was trained three months ago. The original training code is available in Git, but you need to identify the exact dataset version used. What metadata tracking approach would have enabled this?
Explanation
Option A (Vertex AI dataset registration with ML Metadata) is correct because it creates versioned dataset artifacts that are automatically linked to training executions through lineage tracking, enabling you to query which exact dataset version was used for any training run. Option C (checksums in environment variables) is incorrect because while checksums verify data integrity, they don't provide dataset versioning, aren't stored in a queryable metadata system, and don't automatically link to training executions for lineage tracking. Option D (wiki documentation) is incorrect because manual documentation is error-prone, not automatically maintained, doesn't integrate with training systems, and becomes outdated quickly, making it unreliable for reproducibility. Option B (dataset paths in code) is incorrect because hardcoded paths don't constitute versioning, can point to data that has changed or been deleted, and don't capture which data was actually used if the dataset was modified after the code commit.