Why GPUs became AI’s default hardware, the GPU/TPU/NPU/ASIC landscape and its real tradeoffs, the CUDA software stack beneath every framework, memory bandwidth as the bottleneck raw compute specs hide, and how training, inference, and edge hardware each demand something different.
A neural network is, underneath everything, matrix multiplication happening layer after layer — the exact computation Module 3 covers, and the exact computation a GPU’s thousands of parallel cores are architecturally built to accelerate. This module explains why that architectural match made GPUs the default over CPUs, walks the real tradeoffs between GPUs, TPUs, NPUs, and fully custom ASICs, and opens up the CUDA software stack that sits between raw silicon and a line of PyTorch code most engineers never think to look underneath.
By the end of this module you should be able to explain why a GPU can be “compute-cheap but memory-bound” during LLM inference, why training and inference hardware optimize for different economics entirely, and why quantization matters more at the edge than in a data center. Module 12, immediately following, is where these hardware realities turn into an actual deployment decision.