DSPy is a compiler for LLM programs. You declare a Signature plus Module plus Metric, and an Optimizer produces the prompt and few-shot demos. You never write the final prompt by hand.
Imagine you want to teach a recipe to a friend. The old way is to write the recipe out perfectly yourself and hope it works. DSPy is the other way around. You describe what the dish should taste like (the metric) and give your friend 100 examples of good and bad dishes (the trainset). Then you let them iterate until their recipe consistently scores well. You never wrote the recipe; you wrote the goal and provided the feedback. That is how DSPy treats prompts: the framework writes them for you against a metric, instead of asking you to hand-craft each one.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
Concept explanation~2 min read
Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.
DSPy is often mistaken for a LangChain-style library because both are Python libraries that call LLMs. DSPy does something different: it is a compiler for LLM programs. You write the function spec and the loss, and an Optimizer writes the prompt. Every other detail follows from that.
In PyTorch terms, Signature is the model spec, Module is the nn.Module, Optimizer is the optimizer, Metric is the loss, trainset is the dataset, compile() is the training loop, and the compiled program is the checkpoint.
The four primitives and the PyTorch analogy
Signature
A Signature declares the input and output fields of an LLM function, with short semantic descriptions:
class QA(dspy.Signature):
"""Answer questions from the given context."""
context: str = dspy.InputField()
question: str = dspy.InputField()
answer: str = dspy.OutputField()
The Signature is what the prompt will eventually be compiled against. You commit to the function shape; the optimizer commits to the prompt text.
Module
A Module wraps the Signature into a callable LLM step. The common ones:
dspy.Predict(Signature). Direct call, no intermediate reasoning.dspy.ChainOfThought(Signature). Adds areasoningfield the LLM fills in before the output.dspy.ReAct(Signature, tools=[...]). Tool-using agent loop.
Modules are composable, so a multi-step program is Modules calling Modules.
Optimizer and Metric
The Optimizer is the analogue of torch.optim. Built-in options include BootstrapFewShot, MIPROv2, and COPRO. The metric is any Python callable returning a scalar (or boolean) that takes a prediction and a ground-truth example. The trainset is a list of dspy.Example objects.
compile()
compile() is the training loop. It runs the program on the trainset, scores each output with the metric, and searches the prompt and demonstration space. The search continues until it finds a configuration whose metric score on the trainset (or a held-out split) is good. The result is a compiled program with frozen instructions and demonstrations.
Situations where this technique stops working.
2–4 min · Everything important, quickly.
Real products, models, and research that use this idea.
- Stanford STORM builds its multi-step research and article-writing pipeline from DSPy modules.
- DSPy's HotPotQA examples compile retrieval-augmented multi-hop QA programs to exact-match scores higher than hand-tuned baselines.
What an interviewer would ask next. Try answering before peeking at the approach.
QWalk me through how BootstrapFewShot actually generates demonstrations.
It runs the program over the trainset using a teacher Module (same LM, higher temperature, or a stronger LM). It captures (input, intermediate, output) traces and scores each output with your metric. Only traces that pass the metric become demonstrations, up to max_bootstrapped_demos, alongside up to max_labeled_demos labeled examples. Random search over demo sets on a validation split is what BootstrapFewShotWithRandomSearch adds.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Red flags & common mistakes
The phrases that signal junior thinking. Click to expand.
Treating DSPy as 'LangChain with pipes.' The compile loop is the main point. Without a metric and trainset, DSPy is only an awkward way to call an LLM.
60 second bullets to scan on the way to the call.
The four DSPy primitives (Signature, Module, Optimizer, Metric) and the PyTorch analogy
What compile() does end to end
Primary sources. Browse if you want the original framing.
Same topic, related formats. Practice these next.