Episode 84

The Big-Model Toolkit: Checkpointing, ZeRO & FlashAttention

Awareness-level tour of the techniques that make billion-parameter training possible.

• Gradient checkpointing
• ZeRO / FSDP
• FlashAttention
• Putting it together

Notes for this episode are coming soon.