Episode 65

Multi-Head Attention & Positional Encodings

Several attention patterns at once, and how order is injected.

• Multiple heads
• Concatenate & project
• Sinusoidal positions
• Learned positions

Notes for this episode are coming soon.