Zenaique
Topics
Practice
Browse
Reference
Challenges
Pricing
Search…
⌘K
Topics
/
Tokenization
/
Bpe Algorithm
Bpe Algorithm
Subtopic
8 questions
Questions tagged with Bpe Algorithm — part of Tokenization.
Format
Difficulty
Role
Experience
Companies
Sort
Newest
Quality
Difficulty ↑
Difficulty ↓
Questions
How does WordPiece decide which pair to merge, compared to BPE?
Multiple Choice
Medium
·
Qual 4.0
How would you train a custom tokenizer for a genomics LLM on DNA sequences?
Short Answer
Hard
·
Qual 4.0
Which tokenizer can learn merges that span whitespace boundaries by design?
Multiple Choice
Medium
·
Qual 4.0
Google
What is the primary purpose of regex-based pre-tokenization in cl100k_base, and what constraint does it impose on BPE?
Multiple Choice
Medium
·
Qual 4.0
How does weighting BPE merges by document frequency instead of raw token frequency change the resulting vocabulary?
Short Answer
Hard
·
Qual 4.0
Put the BPE training steps in the correct order, from raw corpus to final vocabulary.
Order Steps
Medium
·
Qual 4.0
Spot the error: 'BPE finds the globally optimal vocabulary by iteratively merging the most frequent pair.'
Spot the Error
Medium
·
Qual 4.0
What constraint does BoundlessBPE relax, and what improvement does it yield?
Flashcard
Hard
·
Qual 4.0