Parallelism

Walk through how tensor parallelism shards one matmul across GPUs and what it adds to decode step time

Compare tensor, pipeline and expert parallelism for inference serving.

Match each multi-GPU parallelism strategy to its defining property