ML Engineering: GPU, LLM Serving, Compression, Distributed Training

About the course

The course covers the systems engineering required to train and deploy industrial-scale models. Students study GPU architecture, CUDA kernel optimization, distributed training techniques (data/model/pipeline parallelism), model compression, and high-performance LLM inference systems. The course directly underpins team success in Practicum 3 and serves as a prerequisite for research tracks on efficient architectures.

What you'll learn

✓Profile CUDA kernels and eliminate bottlenecks in GPU computation

✓Implement distributed training with FSDP or tensor parallelism for models with >1B parameters

✓Apply PTQ quantization and measure quality degradation on a task

✓Configure an LLM serving system with continuous batching and KV-cache to hit a target latency

Key topics

•GPU architecture: SIMT, warps, shared memory, memory coalescing

•CUDA programming: kernels, streams, profiling (Nsight)

•Mixed precision: FP16, BF16, FP8, loss scaling

•Distributed training: DDP, FSDP, tensor/pipeline parallelism

•Transformer optimization: FlashAttention, KV-cache, continuous batching

•Quantization: post-training (PTQ), quantization-aware training (QAT)

•Pruning: structured/unstructured, magnitude- and gradient-based

•Knowledge distillation: response-based, feature-based

•LLM serving: vLLM, TensorRT-LLM, speculative decoding

This description was generated automatically and has not yet been reviewed by an instructor — it's a draft for discussion.

About the course

What you'll learn

Key topics

Feedback