GPU Computing Fundamentals for ML Engineers

The CUDA Core Architecture: From Registers to HBM

When NVIDIA engineers designed the CUDA architecture, they made a fundamental trade-off: sacrifice some generality to crush matrix multiplication. That decision built the AI infrastructure landscape we use today.

A modern GPU isn't one monolithic processor - it's a hierarchy of parallelism. At the bottom, you have CUDA cores. These are small, simple processors designed to execute the same instruction on many pieces of data simultaneously. This is the "many-core" philosophy: rather than having 8 super-smart cores (like a CPU), a GPU might have 14,000+ simple cores working in lockstep.

Here's where it gets interesting: these CUDA cores aren't scattered randomly. They're organized into Streaming Multiprocessors (SMs). Each SM contains:

• 128 FP32 CUDA cores (in Hopper architecture)
• 64 FP64 CUDA cores
• 4 fourth-generation Tensor Cores
• Registers (fast, per-thread storage)
• L1 cache (shared with all cores in the SM)
• Shared memory (fast, programmable cache)

An H100 GPU has 144 SMs, which means 144 × 128 = 18,432 total FP32 CUDA cores. Compare that to an A100 with 108 SMs = 13,824 cores. Raw core count matters, but organization matters more.

Within each SM, CUDA cores are grouped into warps - 32 cores that execute the same instruction simultaneously. A warp is the atomic scheduling unit on the GPU. If you launch 32 threads, they form one warp. If you launch 64 threads, that's two warps. This is why thread block sizes are typically powers of 32.

But here's the gotcha that catches beginners: if your 32 threads take different code paths (a branch), they all execute both paths serially. One thread's divergence stalls the entire warp. This is why modern ML code tries to keep operations uniform.

The Memory Hierarchy: Speed vs. Capacity

Now we get to the most important part: where your data lives determines your speed.

Here's the hierarchy from fastest to slowest, with approximate latencies on an H100:

Registers (per-thread):  1-2 cycles latency,  ~100KB per core
L1 Cache (per-SM):       ~30 cycles latency,  ~192KB per SM
Shared Memory (per-SM):  ~30 cycles latency,  ~192KB per SM (programmable)
L2 Cache (per-GPU):      ~200 cycles latency, ~50MB shared
HBM3 Memory (H100):      ~400-500 cycles latency, 80GB-141GB total

The H100's HBM3 (High Bandwidth Memory 3) is where the magic happens. With a 5120-bit bus width and 3.35 TB/s bandwidth, HBM3 can shovel data into those 18,000+ cores at rates that would make a CPU weep. That's 3.35 petabytes per second - enough to feed hungry tensor operations.

But here's the practical reality: HBM has ~400-500 cycle latency. You can't just grab a random byte from HBM quickly. What you can do is grab millions of bytes in parallel. This is why GPUs excel at "data-parallel" workloads - situations where many threads access nearby memory locations. Random access patterns = death by latency.

The deeper gotcha: your 80GB of HBM is shared by all 18,000+ cores. If your model weighs 50GB, you've got maybe 30GB left for activations, gradients, and intermediate tensors. This is why bigger GPUs (H100 80GB vs. RTX 4090 24GB) matter for large model training, even though both can run matrix multiply operations.

Why Memory Hierarchy Design Matters

The memory hierarchy isn't arbitrary. It reflects a fundamental trade-off in electrical engineering. Fast memory (registers, L1 cache) is small because it's expensive to build. Slow memory (HBM) is large because it's cheaper per byte but requires more time to access. GPU designers optimized for the typical ML workload: massive parallel access to large contiguous blocks of data.

This is completely different from CPU design, which optimizes for random access and complex control flow. A CPU can afford to spend hundreds of cycles waiting for data because it has sophisticated prefetching and out-of-order execution. A GPU has no such luxury. If your threads are waiting for data, they're sitting idle, wasting expensive silicon. This is why GPU programmers obsess over memory coalescing - arranging memory access so threads read contiguous locations and get maximum throughput.

GPU Memory Technologies: GDDR6 vs. HBM2e vs. HBM3

Not all GPU memory is created equal. The choice between GDDR6, HBM2e, and HBM3 is a choice between speed and capacity - and it directly affects your ability to train large models.

GDDR6 (Graphics Double Data Rate 6) is what you find in gaming GPUs like the RTX 4090. It's cheap, high-speed per-pin (20 Gbps), but it has a relatively narrow memory bus. A 384-bit bus at 20 Gbps yields approximately 960 GB/s bandwidth. That sounds huge, but wait.

HBM2e (High Bandwidth Memory 2 Enhanced) appears in the A100 GPU. It uses a wider bus - HBM stacks eight 64-bit channels, creating a 512-bit interface per stack. The A100 has four stacks totaling 2048 bits wide. At 3.2 Gbps per pin, the A100 80GB delivers approximately 2.04 TB/s - more than 2x GDDR6's bandwidth.

HBM3 (High Bandwidth Memory 3) is what's in the H100. At 3.35 TB/s (80GB) to 3.9 TB/s (94GB), it delivers another 1.65x improvement. This is the current frontier.

Here's where the practical implications hit: if you're training an LLM and your batch size requires 75GB of GPU memory, the RTX 4090 (24GB GDDR6) is immediately ruled out. The A100 80GB squeezes in. The H100 80GB also squeezes in - with breathing room. But when you shift from training to serving an LLM, things flip.

In inference, you're memory-bandwidth-bound, not memory-capacity-bound. Why? Because you're doing one token prediction at a time, loading the 50GB model weights once into cache, then doing very little computation per byte loaded. With GDDR6's 960 GB/s, you're waiting. With HBM3's 3.35 TB/s, you're feeding the cores. This is why the L40S (GDDR6-based) is a cost-effective inference GPU - you don't need that extra bandwidth as much, so you save money. But for training, HBM is nearly mandatory for large models.

PCIe vs. NVLink: When Single GPU Isn't Enough

Here's a plot twist: some of your GPU's bottleneck isn't internal - it's external. When you connect multiple GPUs, they need to talk to each other. This is where PCIe vs. NVLink becomes a critical choice.

PCIe Gen5 x16 delivers 128 GB/s of bandwidth. It's the standard interconnect. Data travels from one GPU → up through the PCIe hierarchy → through the CPU/host bridge → across the PCIe switch → down to another GPU. This adds latency and consumes CPU resources.

NVLink 4.0 (H100) delivers 900 GB/s per GPU - that's 7x PCIe Gen5. NVLink 5.0 (Blackwell) hits 1.8 TB/s per GPU - 14x PCIe Gen5.

Here's the practical translation: if you're doing distributed training with gradient synchronization across 8 GPUs, you're exchanging gradients that might represent 10+ GB/s of traffic per GPU. With PCIe, that gradient exchange is your bottleneck. With NVLink, you're nowhere near saturation - you're still waiting on compute.

In fact, empirical numbers show that NVLink enables 2-3x higher throughput for multi-GPU training compared to PCIe setups. The difference isn't theoretical - it's measured in wall-clock time.

Now, you might ask: if NVLink is so much better, why not always use it? Because NVLink requires nvidia-fabric topology (multiple GPUs in the same physical chassis, connected via NVLink switches). A PCIe-based multi-GPU setup can span racks. NVLink is cheaper per-GPU than PCIe switches, but you pay for the chassis. An H100 8-GPU NVLink node costs more upfront than an 8-GPU PCIe rack, but the training speed advantage justifies it for large models.

The deeper insight: the more you parallelize, the more interconnect bandwidth matters. Data-parallel training with 8 GPUs? NVLink becomes critical. Single-GPU inference? Interconnect doesn't matter at all.

Understanding Tensor Cores: From Volta to Hopper

Here's where GPU architecture gets truly alien. Tensor Cores aren't the same as CUDA cores. They're specialized matrix-multiply engines.

In Volta (first-gen Tensor Cores), a single Tensor Core could multiply two 4x4 FP16 matrices and accumulate the result in one operation. This was roughly 8x faster than doing the same multiply with CUDA cores. Turing improved this slightly. Ampere (A100) made Tensor Cores faster and added TF32 support. But Hopper changed the game.

Hopper's fourth-generation Tensor Cores introduced FP8 support. This is the breakthrough that enabled efficient LLM training. Here's why: a standard FP16 multiplication uses 16 bits per value. An FP8 multiplication uses 8 bits. That's half the memory bandwidth required. Hopper's Tensor Cores can do FP8 matrix multiply at twice the throughput of FP16.

Concretely: an H100 can reach 989 TFLOPS in FP8 vs. 494 TFLOPS in FP16. Same silicon, but the narrower data type lets you use 2x the compute.

But here's the gotcha (and why research engineers love this stuff): FP8 has lower precision. You lose some accuracy. NVIDIA's approach is mixed precision: do the forward pass in FP8 (fast), store the loss in higher precision, then backprop in higher precision. This maintains accuracy while gaining speed.

Let's look at the numbers across architectures:

Notice the jump from Volta to Hopper: 8x improvement in raw TFLOPS. This isn't marketing - it's empirical. An H100 can train transformers 3-6x faster than an A100, depending on model size and batch size.

Why Tensor Core Evolution Matters for Your Projects

Each generation of Tensor Cores made new things feasible. Volta enabled competitive transformer training on V100s. Ampere made A100s the standard for large models. Hopper's FP8 support made 200B+ parameter models trainable on fewer GPUs. When you choose a GPU, you're not just getting speed - you're enabling certain training strategies that weren't possible on older hardware.

GPU Selection Decision Matrix

You're building a new training cluster. Should you buy H100s? A100s? RTX 4090s? The answer depends on your workload and budget. Here's a framework:

graph TD
    A["What's your primary task?"] -->|LLM Training 50B+| B["Use H100 SXM5 or RTX 6000 Ada"]
    A -->|LLM Training 7-50B| C["A100 80GB or H100 (overkill)"]
    A -->|Fine-tuning only| D["RTX 4090 or L40S"]
    A -->|Inference only| E["L40S or RTX 4000 Ada"]
 
    B --> F["8-GPU NVLink pod<br/>900GB/s interconnect<br/>141TB/s aggregate memory bw"]
    C --> G["PCIe or NVLink<br/>128GB/s or 900GB/s<br/>~$200-300K per pod"]
    D --> H["Single or dual-GPU<br/>Cost-effective training<br/>24-48GB memory"]
    E --> I["No interconnect needed<br/>Optimize inference efficiency<br/>Lower TCO per token"]
 
    style F fill:#ffcccc
    style G fill:#ffeecc
    style H fill:#ffffcc
    style I fill:#ccffcc

Let me translate this into practical heuristics:

For research and large-scale training (50B+ parameters): Use H100 SXM5 in an NVLink pod. The 900 GB/s interconnect and 3.35 TB/s memory bandwidth let you scale to 8-16 GPUs with predictable scaling.

For mid-scale training (7-50B parameters): A100 80GB in PCIe or NVLink topology. Cheaper than H100, still plenty of memory and bandwidth. Training an LLaMA 13B fine-tune? A100 is your sweet spot.

For fine-tuning and experimentation: RTX 4090 or RTX 6000 Ada. 24-48GB GDDR6. You'll hit memory limits on massive models, but most ML work doesn't require Hugging Face's largest models. Cost-effective per TFLOPS.

For inference at scale: L40S (24GB HBM) or RTX 6000 Ada. Inference doesn't need the raw compute of H100 - it needs efficient memory bandwidth and power efficiency. The L40S (23 TFLOPS, 864 GB/s bandwidth) serves tokens faster per watt than an H100.

The real insight: you don't automatically buy the biggest GPU. You buy the smallest GPU that doesn't bottleneck your workload. Bottleneck on memory capacity? Bigger GPU. Bottleneck on bandwidth? You need higher-bandwidth memory technology. Bottleneck on compute? You're actually using the GPU efficiently.

Memory Hierarchy Visualization

Here's how the pieces fit together:

graph TB
    subgraph "Per-Core"
        REG["Registers<br/>~100KB/core<br/>1-2 cycles"]
    end
 
    subgraph "Per-SM 144x"
        L1["L1 Cache<br/>192KB<br/>~30 cycles"]
        SM["Shared Memory<br/>192KB<br/>~30 cycles<br/>Programmable"]
    end
 
    subgraph "Per-GPU"
        L2["L2 Cache<br/>50MB<br/>~200 cycles"]
        HBM["HBM3 Memory<br/>80-141GB<br/>400-500 cycles<br/>3.35-3.9 TB/s"]
    end
 
    subgraph "Multi-GPU"
        NVL["NVLink or PCIe<br/>900GB/s or 128GB/s<br/>Inter-GPU bandwidth"]
    end
 
    REG --> L1
    L1 --> SM
    L1 --> L2
    SM --> L2
    L2 --> HBM
    HBM --> NVL
 
    style REG fill:#ff9999
    style L1 fill:#ffcc99
    style SM fill:#ffcc99
    style L2 fill:#ffff99
    style HBM fill:#ccffcc
    style NVL fill:#99ccff

The key insight from this hierarchy: your code's data locality matters enormously. If your computation fits in registers, it's lightning-fast. If it requires HBM access, you're paying 400+ cycle latencies. This is why GPU kernels are written to maximize L1/shared memory reuse - pulling data into fast memory once, then doing thousands of operations on it.

Practical PyTorch Benchmarking

Now let's measure what we've been discussing. Here's a simple benchmark you can run on your own GPU:

import torch
import time
import numpy as np
 
def benchmark_memory_bandwidth(gpu_id=0):
    """
    Measure effective memory bandwidth by doing vector addition.
    Each operation: 3 GB of memory access (2 reads, 1 write)
    """
    torch.cuda.set_device(gpu_id)
 
    # Test different memory sizes
    sizes_mb = [128, 512, 2048, 8192, 32768]
    results = {}
 
    for size_mb in sizes_mb:
        # Create tensors that fit in cache vs. exceed it
        tensor_size = (size_mb * 1024 * 1024) // 4  # 4 bytes per float32
        x = torch.randn(tensor_size, device='cuda')
        y = torch.randn(tensor_size, device='cuda')
 
        # Warm up
        torch.cuda.synchronize()
        z = x + y  # Simple operation
        torch.cuda.synchronize()
 
        # Benchmark
        num_iterations = 100
        torch.cuda.synchronize()
        start = time.time()
 
        for _ in range(num_iterations):
            z = x + y
 
        torch.cuda.synchronize()
        elapsed = time.time() - start
 
        # Calculate bandwidth
        bytes_per_iter = tensor_size * 4 * 3  # 3 operations per element
        gb_per_sec = (bytes_per_iter * num_iterations) / (elapsed * 1e9)
 
        results[size_mb] = gb_per_sec
        print(f"{size_mb}MB:\t{gb_per_sec:.1f} GB/s")
 
    return results
 
def benchmark_compute_throughput(gpu_id=0):
    """
    Measure pure compute throughput with minimal memory access.
    This stresses the CUDA cores.
    """
    torch.cuda.set_device(gpu_id)
 
    # Large batch matrix multiply (minimal memory relative to compute)
    batch_size = 2048
    m, n, k = 2048, 2048, 2048
 
    x = torch.randn(batch_size, m, k, device='cuda', dtype=torch.float32)
    y = torch.randn(batch_size, k, n, device='cuda', dtype=torch.float32)
 
    torch.cuda.synchronize()
    start = time.time()
 
    num_iterations = 10
    for _ in range(num_iterations):
        z = torch.bmm(x, y)
 
    torch.cuda.synchronize()
    elapsed = time.time() - start
 
    # FLOPS: batch_size * 2 * m * n * k operations
    flops = batch_size * 2 * m * n * k * num_iterations
    tflops = flops / (elapsed * 1e12)
 
    print(f"Matrix multiply TFLOPS: {tflops:.1f}")
    print(f"Time per iteration: {(elapsed/num_iterations)*1000:.2f}ms")
 
    return tflops
 
# Run benchmarks
print("=== Memory Bandwidth Benchmark ===")
bandwidth_results = benchmark_memory_bandwidth()
 
print("\n=== Compute Throughput Benchmark ===")
compute_tflops = benchmark_compute_throughput()
 
# Compare against theoretical maximums
gpu_name = torch.cuda.get_device_name(0)
print(f"\nResults for: {gpu_name}")

When you run this on an H100, you should see:

• L1/L2 cache sizes (128MB): 4.5+ TB/s (nearly theoretical HBM bandwidth)
• Beyond cache (8GB+): 3.0+ TB/s (actual HBM bandwidth)
• Matrix multiply: 900+ TFLOPS (for FP16 with Tensor Cores)

On an A100: expect 2.0 TB/s bandwidth and 600+ TFLOPS FP16. On an RTX 4090: expect 900 GB/s bandwidth but the same CUDA core count means lower absolute TFLOPS.

These numbers tell you where your bottleneck actually is. Memory-bound? Add GPU memory or improve data locality. Compute-bound? You're actually efficiently using the GPU.

Practical Implications for Your ML Workflow

Let's ground this in real decisions:

Scenario 1: Fine-tuning BERT on your GPU

• Model: 110M parameters = 440MB weights
• Batch size: 32, sequence length: 512
• Total memory: ~6-8GB

Decision: RTX 4090 or any 16GB+ GPU works. You're not memory-limited. You'll hit compute saturation if your training loop isn't optimized, but that's code, not hardware.

Scenario 2: Training a 13B parameter LLaMA model

• Model weights: 52GB (16-bit)
• Activations for batch size 4: ~20GB
• Optimizer states: ~52GB
• Total: ~124GB

Decision: H100 80GB won't fit (need to use gradient checkpointing). H100 with 141GB (NVL variant) or multiple-GPU training is required. An A100 80GB with gradient checkpointing is marginal.

Scenario 3: Serving a 70B parameter model for inference

• Model weights: 280GB (FP8 quantized)
• Per-token activation: ~1GB
• Batch size: 16 (concurrent requests)

Decision: RTX 4090 is immediately out (24GB). L40S with 24GB is out. You need 8x RTX 6000 Ada (48GB each) or 4x H100 (80GB each). But bandwidth matters: with RTX 6000 Ada (960 GB/s), you get 160 tokens/second. With H100 (3.35 TB/s), you get 560 tokens/second. The H100 is almost 4x faster for inference, justifying the higher cost.

The Hidden Layer: Why Your Model Training Stalls

Here's wisdom you won't find in NVIDIA marketing: most AI engineers assume their GPU is the bottleneck, but it usually isn't.

Your training loop is likely bottlenecked on data loading, not GPU compute. You're starting 18,000+ GPU cores every iteration, and they're waiting for your data loader to fetch the next batch. Solution? Prefetch data, use mixed-precision training to reduce the compute time so the GPU waits less, or scale to multiple GPUs where each GPU can be busy while others load.

If you do hit GPU bottlenecks, it's usually not raw TFLOPS - it's memory bandwidth. Your model is too small to keep 18,000+ cores busy doing something meaningful. Add larger batches if you can, or accept that smaller models won't saturate the GPU.

Multi-GPU training often shows diminishing returns above 8 GPUs because communication bandwidth (NVLink or PCIe) becomes the bottleneck. You're spending time synchronizing gradients instead of computing them. This is why distributed training frameworks use gradient compression, overlapping communication with computation, and careful batch size tuning.

The deeper insight: GPUs are only as fast as the weakest link in your infrastructure. Slow storage → slow data loading → GPU stalls. Slow interconnect → slow gradient synchronization → GPU stalls. Unoptimized data types → leaving TFLOPS on the table. CPU bottleneck in data preprocessing → GPU idle waiting.

This is why real ML infrastructure requires hardware + software co-design, not just buying the biggest GPU.

Summary: Building Your GPU Strategy

You now understand:

1. CUDA architecture: SMs, warps, and the hierarchy from registers to HBM. Your data locality determines your speed.

2. Memory technologies: GDDR6 works for smaller models and inference. HBM2e and HBM3 are necessary for large-model training. Bandwidth, not capacity, is often your limiting factor.

3. Interconnects: PCIe is cheap but slow (128 GB/s). NVLink is fast (900 GB/s for H100, 1.8 TB/s for Blackwell) but expensive and requires tight coupling. Choose based on your model's communication patterns.

4. Tensor Cores: They've evolved from 8x baseline speedup to supporting FP8, which enables 16x speedup for compatible workloads. Mixed-precision training is now default, not advanced.

5. GPU selection: Isn't about buying the biggest - it's about matching GPU capabilities to your bottleneck. Memory-bound? Get more memory. Bandwidth-bound? Get HBM. Compute-bound? You're actually using the GPU efficiently.

The final wisdom: your GPU choice should come from profiling your workload, not from vendor hype. Measure memory bandwidth with simple benchmarks. Profile your training loop. Understand where the bottleneck actually is. Then buy the smallest GPU that removes that bottleneck.

Start with the PyTorch benchmark above. Profile your actual model. Compare measured bandwidth to theoretical maximums. The gaps you find are where optimization opportunities live.

The Real Cost of GPU Choices

Choosing the wrong GPU for your workload is extraordinarily expensive because the cost scales with usage. Buying an H100 when you only need an A100 means paying two hundred percent for every hour of training. If you train for one thousand hours a year, that difference compounds to hundreds of thousands of dollars in unnecessary spend. But buying an A100 when you need an H100 is worse because your model takes three times as long to train, your iteration cycle slows, and your engineers spend time waiting instead of experimenting. The time cost often exceeds the hardware cost.

This is why profiling before purchasing is so critical. You need to know your actual bottleneck. Does your model spend most of its time waiting for data, or does it max out the GPU? Does your model fit in memory with all the headroom you need, or are you constantly running out of space? Different answers point to different hardware.

Many organizations waste enormous resources by defaulting to the biggest GPU. They buy H100s for everyone because H100s are the fastest. But a large percentage of workloads do not need the speed. An H100 is overkill for fine-tuning a pre-trained model on a dataset that fits in memory. An A100 does the same work for half the cost. And for some workloads like single-GPU inference, an RTX 4090 or L40S is genuinely more cost-effective because you are not paying for multi-GPU interconnect and you do not need the memory bandwidth of HBM3.

The Evolution of Hardware Capability Matching

As your organization grows, your GPU selection decisions become more sophisticated. Early on, you might have a single cluster with identical hardware. Everyone trains on the same GPUs. This is simple but inefficient because different workloads have different requirements.

As you scale, you create specialized clusters. You have a training cluster with high-bandwidth GPUs for large-model training. You have an inference cluster with lower-bandwidth GPUs optimized for latency. You have a development cluster with smaller GPUs where researchers experiment. This segmentation means you are paying only for the capability you actually use.

The most mature organizations go further. They build infrastructure that automatically matches workloads to hardware. A researcher submits a training job that requires sixty terabytes of memory. The scheduler routes it to a multi-GPU NVLink pod because no single GPU has that much memory. A different researcher fine-tunes a model that fits on a single GPU. The scheduler routes them to an A100, not an H100, because the job is not bandwidth-limited. This intelligent matching requires monitoring and profiling infrastructure, but the cost savings are dramatic.

Understanding Your Team's GPU Knowledge

Another aspect that often gets underestimated is the knowledge asymmetry within teams. Some engineers understand GPU internals deeply. They can look at a training loop and predict exactly where bottlenecks are. Other engineers know GPU programming but not architecture. And some ML engineers use GPUs without really understanding how they work under the hood.

This knowledge gap creates risk. Engineers who do not understand GPU constraints often make decisions that seem fine locally but are expensive globally. They write code that is inefficient on GPUs but they do not realize it because training is fast enough. But when you scale to larger models or longer training runs, those inefficiencies become catastrophic. Bandwidth gets wasted. Compute sits idle. You are paying for hardware that is not being used effectively.

Building shared knowledge about GPU fundamentals helps here. Not everyone needs to be able to write CUDA kernels, but everyone should understand the hierarchy from registers to HBM. Everyone should know that memory bandwidth is often the bottleneck, not compute. Everyone should know that mixing GDDR6 and HBM in the same job creates performance cliffs. With this shared mental model, teams make better decisions about hardware allocation and training design.

The Practical Reality of GPU Scarcity

In practice, many organizations operate under GPU scarcity. You have fewer GPUs than you want and more people who want to use them. This scarcity is actually good because it forces discipline. You cannot afford to leave expensive hardware idle. You start measuring GPU utilization. You start asking whether that H100 is really being used efficiently or if it is mostly waiting for data. You start scheduling jobs to keep GPUs busy.

This scarcity-driven optimization is how many organizations discover their first big efficiency gains. By forcing better utilization, they effectively get fifty percent more capacity without buying new hardware. The practical techniques are simple: batch multiple jobs per GPU, pipeline data loading so the GPU is never waiting, use mixed-precision training to increase throughput. These are not rocket science, but they compound.

The key insight is that GPU scarcity is a feature, not a bug, if you use it correctly. It creates the pressure that drives efficiency improvements. Organizations with unlimited budget tend to over-provision and never optimize. Organizations that have to make hard choices about GPU allocation end up with leaner, more efficient systems.

Want to dive deeper? Check out these resources:

• NVIDIA A100 vs. H100: Architecture and Specs
• NVIDIA Hopper Architecture In-Depth
• Tensor Core Evolution: From Volta To Blackwell
• NVLink vs PCIe: Key Differences for AI Workloads
• High Bandwidth Memory (HBM): Everything You Need to Know