Garak is an open-source LLM vulnerability scanner. You point it at a model endpoint and it runs a battery of probes, prompt injection, jailbreak templates, encoding attacks, data leakage, malware generation, then reports which probes succeeded with a structured findings file. Think 'nmap for LLMs', it is the first-pass scanner before more targeted red-team campaigns.
**Garak is an open-source LLM vulnerability scanner, think 'nmap for LLMs.'** You point it at a model endpoint and it runs a battery of probes, producing a structured findings file that lists which probes succeeded.
The probe categories cover the common attack surface:
- **Prompt injection**, known templates (DAN, AIM, role-override) and variants.
- **Jailbreaks**, encoding attacks (base64, ROT13), prefix injection, refusal suppression.
- **Data leakage**, probes designed to elicit training-data extraction or system-prompt disclosure.
- **Malware and harmful content**, probes for generating malicious code, instructions for harm.
- **Hallucination**, probes that test made-up entity recognition and false-fact generation.
- **Toxicity and bias**, adversarial templates that elicit slurs, stereotypes, or biased reasoning.
Garak's role in the red-team toolchain is first-pass discovery. Before a security team commits to a targeted red-team campaign, they run garak to catch the obvious weaknesses. The findings file becomes the input to deeper investigation: probes that succeeded indicate categories where the model is vulnerable; those categories get focused attention.
Garak is open-source from NVIDIA and the broader community. It plugs into common model endpoints (Hugging Face, OpenAI-compatible APIs, REST). It runs in CI for regression detection, in periodic schedules for ongoing scanning, and on-demand during incident investigation.
**What garak is not.** It is not a substitute for bespoke red-teaming. It is not continuous evaluation against your custom policies. Its probe library is public, so a model trained to pass garak's tests may still be vulnerable to novel attacks. Treat it as the floor of red-team coverage, not the ceiling.
**Garak is the open-source LLM vulnerability scanner, the nmap for LLMs that produces a first-pass findings report for any model endpoint you point it at.**
**What it does.** Garak runs a battery of probes against a model endpoint and reports which probes succeeded. The probe library covers prompt injection (direct templates, indirect via tool-output simulation), jailbreaks (encoding attacks, prefix injection, refusal suppression, persona overrides), data leakage (training-data extraction, system-prompt extraction), harmful generation (malware, weapons, illicit instructions), hallucination (fabricated-entity recognition), and toxicity and bias adversarial probing. Each probe has a generator (the prompt to send) and a detector (how to decide whether the response indicates success).
The output is a structured JSON findings file: per-probe success rates, example successful responses, severity classification. The findings file is consumable by humans (red-team triage) and by tooling (regression detection, CI gating).
**Where it fits in the red-team toolchain.** Three roles.
First, *first-pass discovery.* Before a security team commits to a targeted red-team campaign, they run garak. The findings highlight which attack categories the model is vulnerable to; the campaign then focuses on those categories rather than starting from scratch.
Second, *CI regression.* After model upgrades, fine-tune refreshes, or prompt changes, run garak in CI to catch regressions. A previously-passing probe that now succeeds indicates a regression.
Third, *periodic ongoing scanning.* On a schedule (weekly, monthly), run garak against production endpoints to catch drift or new probe-template additions in the garak library.
**Comparison with PyRIT.** Garak is a scanner with a fixed probe library; PyRIT (Microsoft's Python Risk Identification Tool) is a framework for building custom red-team campaigns.
Quick reference
TL;DR
Garak is an open-source LLM vulnerability scanner, the nmap for LLMs first-pass tool.
When a network security team wants to know what services are running on a server, they reach for a scanner like nmap. They point the scanner at the server, it runs through a long list of known checks, and it produces a report listing what is exposed and what looks risky. Garak does the same job for a language model. You give it a model endpoint, it tries every known trick, pretending to be a different system, asking the model to leak its training data, requesting harmful instructions in encoded form, and it reports which tricks worked. It is not a sophisticated targeted attack, just a broad first-pass check that catches the obvious weaknesses before a deeper review.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.
Production red-teaming of a high-stakes deployment where coverage depth matters more than breadth Garak is shallow and broad; targeted campaigns with PyRIT or expert red-teamers give better depth on specific attack categories.
Domain-specific testing where the relevant attack surface is not in garak's probe library Medical-specific misinformation probes, financial-specific manipulation probes, and similar domain-specific risks need custom probes.
Continuous production-traffic evaluation against custom policies Garak is point in time scanning; continuous evaluation against your specific policy surface is the job of Patronus, Promptfoo continuous mode, or equivalent.
Quick scan · 2–4 min
Everything important, quickly.
2 min: garak is the open-source LLM vulnerability scanner, nmap for LLMs. Battery of probes (injection, jailbreak, encoding, leakage, malware, hallucination, toxicity) produces a structured findings report. Roles: first-pass discovery before bespoke red-teaming, CI regression gate, periodic scanning. Limits: public probe library, detector noise, breadth not depth. Composes with PyRIT (custom campaigns), Promptfoo (CI), and continuous evaluation platforms.
Real-world examples
Real products, models, and research that use this idea.
1Garak is maintained by NVIDIA and is open-source on GitHub; 2026 versions ship probes derived from the OWASP LLM Top 10 and published jailbreak research.
2Promptfoo's safety-mode probes pull from the garak library along with PyRIT and OWASP for CI integration.
3Anthropic and OpenAI both cite garak in their model cards as part of the external evaluation suite their models are tested against.
4Enterprise red-team programs at financial institutions and healthcare companies use garak as the first-pass scanner before engaging bespoke red-team consultants.
Interviewer follow-ups
What an interviewer would ask next. Try answering before peeking at the approach.
QHow do you handle garak's detector noise, probes marked as succeeded when the model actually refused?
Sign in or go Pro to see the approach for this follow-up.
QWhy is garak insufficient for continuous evaluation, even when run on a schedule?
Sign in or go Pro to see the approach for this follow-up.
QWhat is a meaningful way to compose garak with the production-time output rail?
Sign in or go Pro to see the approach for this follow-up.
Don't say this
Red flags and common mistakes that signal junior thinking. Click to expand.
Most common mistake
Treating garak's pass as a guarantee of safety. It is a first-pass scanner; bespoke red-teaming and continuous evaluation are still needed.