Zenaique
Topics
Practice
Study
Browse
Reference
Pricing
Search…
⌘K
Topics
/
Fine-Tuning
/
Warmup
Warmup
Subtopic
4 questions
Questions tagged with Warmup — part of Fine-Tuning.
Premium questions for this topic
Format
Difficulty
Role
Experience
Companies
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
Why do large runs ramp learning rate before entering cosine decay?
Multiple Choice
Easy
What does the first forward pass set up that the second one doesn't pay for?
Short Answer
Medium
Why ramp the learning rate during warmup instead of starting at the peak value?
Flashcard
Easy
Loss spikes the moment warmup ends: which hyperparameter is the prime suspect?
Short Answer
Medium
NVIDIA
OpenAI
Bcg
Pinecone
Cresta
Ltimindtree
Stability Ai
Typeface