Red Teaming
Also known as: Adversarial testing
Adversarially probe the model/app to find safety failures, jailbreaks, and prompt-injection holes.
Systematic adversarial probing of a model or LLM app to surface harmful outputs, bypass safety filters, or exploit prompt injection. Combines manual creativity with automated attack generation.
In practice
Standard pre-launch step for any user-facing LLM app. Safety interviews probe attack categories and coverage measurement.
How it compares
Prompt injection is one attack class; red teaming is the broader practice of finding any safety failures.
Related topics
Related terms
Hallucination
When a model confidently makes up something that isn't true.
LLM Evaluation
Measuring whether an LLM does what you want, beyond "looks fine to me".
Perplexity
Exp(average cross-entropy) on held-out text; lower means the model is less surprised by real data.
Reasoning Model
An LLM trained to reason at length internally before answering. Slower and more expensive, but much better on hard problems.
Guardrails
Pre- and post-processing layers that block bad inputs/outputs and enforce policy on top of an LLM.
MMLU (Massive Multitask Language Understanding)
Multiple-choice benchmark across 57 academic subjects; the standard "raw knowledge" headline number.