Episode 54

Vanishing & Exploding Gradients

Why plain RNNs forget, shown numerically.

• Repeated multiplication
• Vanishing gradients
• Exploding gradients
• Gradient clipping

Notes for this episode are coming soon.