Fresher / New Grad

BERT prepends a [CLS] token to every input, what is its actual job in the model?

For a causal mask over a 4-token sequence, what value sits at row 2, column 3?

Name the additive bias term in attention and the two things it most commonly carries.

After W_O projects the multi-head output back to d_model, what happens before the next sub-layer runs?

Define a single attention head, what does one head actually own and produce?

ALiBi, decode the acronym and pinpoint where in the pipeline it injects position

Pick the statement that best describes a scoring rubric in LLM evaluation.

Before running LLM-as-judge, someone insists on writing a rubric first. What is a rubric in this context?

ROUGE keeps showing up in summarization evals. How does it work and where does it break?

A model card reports perplexity of 3.2. Explain what that number means in plain English.

Pick the best definition: in LLM evaluation, a metric is _____.

Why might F1 score be a better choice than raw accuracy for evaluating entity extraction?

When someone says 'eval prompt template,' what are they referring to and why does the wording matter so much?

A teammate asks what 'running evals' means in the LLM context. How would an engineer explain it without jargon?

A model aces a benchmark with suspiciously high scores. The first hypothesis is contamination. Explain what that means to a non-technical PM.

Your LLM judge gives everything a 4 out of 5. Your colleague says the judge is not calibrated. What does calibration mean here?

BLEU score comes up in a meeting. What does it measure and when does it still make sense?

Every model release cites benchmark numbers. Define what a benchmark actually is and name two things it cannot tell you.

You have no reference answers for your eval set. Does that mean you cannot evaluate the model?

Explain the difference between pointwise and pairwise evaluation in one breath.

Offline eval now or wait for the A/B test results? Pick the key difference between online and offline evaluation.

Automated eval vs human eval: which is better for catching subtle factual errors in medical text?

TruthfulQA is designed to catch a specific model failure. Describe what it tests and why larger models sometimes do worse.

MT-Bench is used alongside Chatbot Arena. Describe what MT-Bench tests that single-turn benchmarks miss.

MMLU appears on every model leaderboard. Describe what it tests and its biggest blind spot.