AI Researcher

Name a 2026 task where an encoder-decoder transformer still beats a decoder-only LLM

What fraction of a 7B transformer lives in its embedding table at vocab 128k?

Identify the locations where dropout typically sits inside a transformer block

Spend a fixed parameter budget: more layers or wider d_model, and what does the evidence say?

Why did decoder-only architectures take over tasks T5-style encoder-decoders used to win?

Complete the pre-norm transformer block recipe

Order the operations inside a single pre-norm transformer block forward pass

Predict the typical layer count for a Llama 7B and a Llama 70B

Pair each 2017 transformer ingredient with its 2026 production replacement

Defend pinning the tokenizer revision alongside the model weights when shipping a fine-tune

Why does completion-only loss usually outperform full-sequence loss on the same SFT data?

Walk through what happens during backprop for a LoRA-wrapped linear layer

Sequence the forward passes and loss steps inside a single DPO training step

Pick the most likely cause when training loss drops for 1k steps, plateaus for 2k, then drops again at step 3k

LoRA freezes the base weights: does that mean activation checkpointing on the base is unnecessary?

Merging a LoRA back into an NF4-quantized base hits one subtle problem: name it and explain why

Adam stores two fp32 buffers per trainable parameter: what does that imply for LoRA vs full FT on a 7B?

Spot the bug: packed-sequence SFT where the model learns to copy across unrelated examples

Why does using the same model family as synthetic-data teacher AND eval judge inflate FT scores?

When eval prompts appear verbatim inside an SFT file after a synthetic-data run, what's the root cause?

A distilled model hits 92% on a public benchmark but 31% on an internal hold-out: diagnose.

RLOO's name hides its baseline trick: what does 'Leave-One-Out' literally compute?

With max_grad_norm=1.0 and a raw gradient L2 of 12.5, what is the post-clip norm and scale factor?

Training loss draws a sharp saw-tooth at every epoch boundary: diagnose.

Why has GRPO eaten PPO's lunch in 2025-2026 reasoning post-training?