Episode 64

Self-Attention from Scratch

Every token attending to every other token, coded in PyTorch.

• Scaled dot-product attention
• Why scale by √d
• Causal masking
• Code from scratch

Notes for this episode are coming soon.