⚡ Free Classes and Scholarships Available for Underprivileged Students -

Choosing appropriate hardware for training

Scaling Prototypes into ML Models

Set 1
10 Questions
No time limit
Practice Mode
0%
Score
0
Correct
0
Incorrect
10
Total Questions
Back to Topics
Question 1 of 10
You are training a large transformer-based language model with 10 billion parameters on Vertex AI. The model requires significant matrix multiplication operations and benefits from high memory bandwidth. Training time is critical, and you need the most cost-effective solution for this specific workload. What hardware should you select?
Explanation
Option D (TPU v4 Pods) is correct because TPUs are specifically optimized for large-scale matrix operations common in transformer models, offer superior performance for training large language models, and provide high memory bandwidth through their specialized architecture. They are the most cost-effective solution for this workload type. Option B (NVIDIA A100 GPUs) would work but is less cost-effective than TPUs for pure transformer training at this scale. Option C (NVIDIA T4 GPUs) are better suited for inference workloads and smaller training jobs, lacking the compute power needed for 10B parameter models. Option A (High-memory CPU instances) would be extremely slow for training deep learning models of this size, as CPUs lack the parallel processing capabilities required for efficient neural network training.
Question 2 of 10
Your team is implementing distributed training for a computer vision model using PyTorch on Vertex AI with 8 NVIDIA V100 GPUs. You need to synchronize gradients efficiently across all GPUs while minimizing communication overhead. Which approach should you use?
Explanation
Option B (Vertex AI Reduction Server) is correct because it is specifically designed for efficient gradient synchronization in distributed training, providing optimized all-reduce operations that minimize communication overhead between GPUs during training. It handles the complexity of gradient aggregation automatically. Option C (Cloud Storage bucket) is inappropriate for real-time gradient synchronization as it has high latency and is designed for persistent storage, not rapid inter-process communication. Option A (Pub/Sub messaging) is designed for asynchronous message passing and would introduce unnecessary latency for the tight synchronization required in distributed training. Option D (Memorystore Redis) is a caching solution not optimized for the specific all-reduce communication patterns needed in distributed GPU training.
Question 3 of 10
You need to train a custom object detection model with a dataset of 50,000 images. The model uses a ResNet-50 backbone and must be deployed within 2 weeks. Budget is limited, and you want to minimize training costs while maintaining reasonable training time. What hardware configuration should you choose?
Explanation
Option C (Single NVIDIA T4 GPU) is correct because for a moderate-sized dataset (50,000 images) with a standard architecture like ResNet-50, a single T4 GPU provides sufficient compute power at the lowest cost while still completing training in a reasonable timeframe. T4s offer good cost-performance for this scale. Option A (TPU v3 Pod) is massive overkill for this dataset size and would be extremely expensive, as TPU Pods are designed for very large-scale training jobs. Option B (8 NVIDIA A100 GPUs) would provide faster training but at significantly higher cost than necessary for this moderate workload, violating the budget constraint. Option D (CPU instance) would result in unacceptably slow training times for a neural network of this complexity, potentially taking weeks to complete.
Question 4 of 10
Your organization is training multiple small to medium-sized ML models daily using TensorFlow on Vertex AI. Models vary in architecture (CNNs, RNNs, and custom layers) and training times range from 2-6 hours each. You need flexibility to handle different frameworks and model types. What hardware should you standardize on?
Explanation
Option A (NVIDIA T4 or V100 GPUs) is correct because GPUs provide the flexibility needed to handle diverse model architectures and frameworks, including custom layers that may not be optimized for TPUs. They offer good performance for the 2-6 hour training window and support a wider range of operations. Option D (TPU v2 Pods) would be cost-prohibitive for multiple daily training jobs of this size, and TPUs have limitations with custom operations and non-standard architectures. Option C (Edge TPU devices) are designed for inference at the edge, not training, and lack the compute power needed for training neural networks. Option B (Preemptible CPU instances) would result in extremely slow training times for neural networks and frequent interruptions, making them impractical despite cost savings.
Question 5 of 10
You are implementing data parallelism for training a deep learning model using Horovod on Vertex AI with 16 GPUs. The model has large activation maps that consume significant memory. What configuration will optimize training throughput while preventing out-of-memory errors?
Explanation
Option D (Gradient compression with reduced batch size) is correct because in data parallelism with Horovod, reducing the per-GPU batch size directly addresses memory constraints from large activation maps, while gradient compression reduces communication overhead during all-reduce operations, maintaining training throughput. This approach keeps the data-parallel paradigm intact. Option A (Model parallelism) changes the fundamental training strategy and is more complex to implement; it's typically used when the model itself doesn't fit on a single device, not when activations are the issue. Option C (Asynchronous updates with parameter server) introduces staleness in gradients and is less efficient than Horovod's synchronous all-reduce approach, particularly for the stated scenario. Option B (Increased learning rate with larger batches) would worsen the memory problem rather than solve it, as larger batches require more memory for activations.
Question 6 of 10
Your team needs to train a neural network that includes custom TensorFlow operations not yet supported by TPU compilers. The model is medium-sized (500M parameters) and training must complete within the next sprint. What hardware choice balances compatibility and performance?
Explanation
Option C (NVIDIA A100 GPUs with CUDA) is correct because GPUs provide native support for custom TensorFlow operations through CUDA, eliminating compatibility issues while offering excellent performance for a 500M parameter model. This ensures training can proceed immediately without debugging compilation issues. Option D (TPU v4 with XLA) would fail because the custom operations are explicitly stated as not supported by TPU compilers, leading to compilation errors or fallback to CPU. Option A (TPU v3 with compatibility mode) still relies on XLA compilation and won't resolve the unsupported operations issue; compatibility mode doesn't add support for arbitrary custom ops. Option B (Cloud TPU VMs with custom kernels) would require significant engineering effort to implement custom kernels for TPU, exceeding the sprint timeline constraint.
Question 7 of 10
You are training a recommendation model using embeddings for 100 million users. The model requires frequent embedding lookups and updates during training. Memory bandwidth is the primary bottleneck. Which hardware configuration best addresses this specific constraint?
Explanation
Option A (TPU v4 with high-bandwidth memory) is correct because TPUs are specifically designed with high-bandwidth memory (HBM) that excels at the rapid memory access patterns required for large embedding tables. The memory bandwidth of TPUs significantly outperforms other options for embedding-intensive workloads. Option D (NVIDIA V100 GPUs with NVLink) provides good memory bandwidth, but still lower than TPU HBM, and NVLink primarily helps with multi-GPU communication rather than single-device memory bandwidth. Option C (NVIDIA T4 GPUs) have lower memory bandwidth than V100s or TPUs, and simply adding more GPUs doesn't increase per-device memory bandwidth for embedding lookups. Option B (CPU with SSD) would be extremely slow for training neural networks, as SSDs have orders of magnitude lower bandwidth than HBM or GPU memory.
Question 8 of 10
Your data science team is prototyping various model architectures including experimental attention mechanisms and sparse operations. Models change frequently during the research phase. Training jobs typically run for 4-8 hours on Vertex AI. What hardware provides the best flexibility for this exploratory phase?
Explanation
Option B (NVIDIA V100 or A100 GPUs) is correct because GPUs offer maximum flexibility for experimental architectures, supporting custom operations, sparse computations, and rapidly changing code without requiring XLA compilation or TPU-specific optimizations. They work well with standard deep learning frameworks during prototyping. Option A (TPU v3 Pods) are expensive and less flexible for experimental code, as custom operations may not compile or run efficiently on TPUs, and the Pod scale is unnecessary for prototyping. Option C (Edge TPU development boards) are designed for inference deployment testing, not training, and lack the compute power needed for model training. Option D (Preemptible TPU v2 instances) would suffer from both TPU flexibility limitations and frequent interruptions during the 4-8 hour training windows, disrupting the research workflow.
Question 9 of 10
You need to perform distributed training of a 2 billion parameter model using TensorFlow on Vertex AI. The training script uses tf.distribute.Strategy for scaling. You want to minimize code changes while achieving near-linear scaling across multiple accelerators. What should you implement?
Explanation
Option D (MirroredStrategy with Reduction Server) is correct because MirroredStrategy is designed for synchronous training across multiple GPUs with minimal code changes, and Vertex AI's Reduction Server optimizes the all-reduce operations for near-linear scaling. This combination provides the best performance with the least modification to existing code. Option B (TPUStrategy with TPU Pods) would work but requires TPU-specific code considerations and may need more changes than GPU-based MirroredStrategy, though it is a valid alternative. Option A (ParameterServerStrategy with CPU) introduces asynchronous training complexity and CPUs would be a bottleneck, not suitable for a 2B parameter model. Option C (MultiWorkerMirroredStrategy with manual aggregation) unnecessarily complicates the implementation, as Reduction Server provides optimized aggregation automatically without manual intervention.
Question 10 of 10
Your production training pipeline runs daily on Vertex AI, training a convolutional neural network for image classification. Training takes approximately 10 hours. Management wants to reduce training costs by 60% while maintaining model quality. What approach achieves this cost reduction?
Explanation
Option A (Preemptible GPU instances with checkpointing) is correct because preemptible VMs offer approximately 60-70% cost reduction compared to regular instances, and implementing checkpointing ensures training can resume after preemptions without losing progress. For a 10-hour job, this is feasible with proper checkpoint management. Option D (Reduce model size and use CPUs) would significantly degrade model quality, violating the constraint to maintain quality, and CPU training would still be slow even for smaller models. Option B (TPU Pods with reduced frequency) doesn't reduce per-run costs meaningfully (TPU Pods are expensive), and reducing frequency changes the training schedule rather than the cost per training run. Option C (Quantization-aware training) may slightly reduce training time but doesn't reduce costs by 60%; it's primarily for inference optimization and doesn't fundamentally change training costs.