Complete guide to AI compute hardware and software: GPU architecture (H100, H200, B200), TPU systolic arrays, NPUs (Apple ANE 38 TOPS, Qualcomm Hexagon), ASICs (Cerebras WSE-3 4 trillion transistors, Groq LPU), HBM memory bandwidth (3.35 TB/s HBM3), NVLink interconnects (900 GB/s), CUDA programming model, cuBLAS/cuDNN/NCCL libraries, inference engines (TensorRT-LLM, vLLM PagedAttention, llama.cpp), cloud AI pricing (AWS p5 $98/hr), and AI compute trends from Moore's Law to system-scale engineering.