Zenaique
Topics
Practice
Study
Browse
Reference
Pricing
Search…
⌘K
Topics
/
Tokenization
/
Unigram Lm
Unigram Lm
Subtopic
4 questions
Questions tagged with Unigram Lm — part of Tokenization.
Premium questions for this topic
Format
Difficulty
Role
Experience
Companies
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
BPE grows the vocabulary by adding merges. Unigram does the opposite. Explain the difference.
Flashcard
Easy
Explain the Unigram LM tokenization algorithm and how its training differs fundamentally from BPE. What does stochastic tokenization enable?
Short Answer
Hard
Which statement correctly describes the Unigram Language Model tokenizer's training procedure?
Multiple Choice
Medium
Spot the error: 'BPE finds the globally optimal vocabulary by iteratively merging the most frequent pair.'
Spot the Error
Medium
Snap
Synthesia
Google
Kore Ai
Google
Siemens
Goldman Sachs
OpenAI