Episode 78

Gradient Accumulation & Distributed Training Basics

Training with bigger effective batches and more GPUs.

• Gradient accumulation
• Data parallelism
• DDP overview
• When you need it

Notes for this episode are coming soon.