Match each DSPy optimizer to its core search strategy

Match pairs·Hard·4.0 · 0·~2 min·Asked atDatabricks Cohere Anthropic

Attempt it

Bootstraps few-shot examples from a stronger teacher model and selects the ones that pass the metric

Jointly optimises instructions and demonstrations via Bayesian search over a candidate space

Refines instructions through coordinate ascent, perturbing one prompt component at a time and keeping winners

Bootstraps demonstrations with random search over a finite candidate set, the cheap baseline

Fine-tunes the underlying LM weights against the metric instead of editing the prompt

DSPy is a framework that treats prompts as something a compiler should construct, not something humans hand-tune. The piece that does the constructing is called an **optimizer** (sometimes 'teleprompter' in older docs). Each optimizer is a search strategy over some part of a prompt program. They differ along two axes: **what they edit** (the few-shot demonstrations, the instruction string, or the model weights themselves) and **how they search** (greedy bootstrap, random sampling, Bayesian optimization, coordinate ascent, fine-tuning). The five in this question span the design space. `BootstrapFewShot` greedily collects demos from a teacher model that score well on your metric. `BootstrapFewShotWithRandomSearch` does the same with random search over candidate sets. It is the cheap and cheerful baseline. `COPRO` perturbs the instruction string one slot at a time, keeping changes that improve the metric. `MIPROv2` is the most sophisticated of the prompt-editing optimizers. It runs Bayesian search jointly over instructions and demonstrations. `BootstrapFinetune` skips prompt search entirely and fine-tunes the model weights. Learning the names is half the battle; the names actually telegraph what each does once you know the conventions.

DSPy compiles prompt programs the way a compiler compiles code: optimizers search over the program's tunable parts to maximize a user-supplied metric. The five in the question cover most of the active design space. `BootstrapFewShot` runs your program with a teacher LM, collects traces, keeps the demonstrations that scored on the metric, and uses them as few-shot examples for the student. Cheap, effective, the most-used optimizer in early DSPy work. `BootstrapFewShotWithRandomSearch` extends that: instead of one set of bootstrapped demos, it samples many candidate sets and picks the best by validation score. Random search is the honest baseline. `COPRO` (Coordinate-ascent Prompt Optimization) edits the instruction string. It generates several candidate instructions, evaluates them, keeps the winner, and iterates. One component changes at a time. Classic coordinate ascent. `MIPROv2` is the most ambitious prompt optimizer: it builds a candidate space over both instructions and demonstrations, uses Bayesian optimization with a surrogate model to pick promising combinations, and converges faster than random search at higher cost per round. The current default for serious DSPy work. `BootstrapFinetune` leaves prompts alone and fine-tunes the LM weights against your metric using the bootstrapped traces as training data. Useful when prompt search has hit a ceiling and you have a model you can fine-tune.

Quick reference

TL;DR

DSPy optimizers vary by what they edit (demos vs instructions vs weights) and how they search (greedy bootstrap, random, Bayesian, coordinate ascent, gradient).

Memory aid

Easy to grasp

Picture a chef trying to nail a dish for a critic. BootstrapFewShot watches a master chef make it five times and copies the bites the critic liked. RandomSearch tries random combinations of plates and keeps the best. COPRO tweaks one ingredient at a time, keeping the change if the critic smiles. MIPROv2 is the data-driven chef who runs a smart trial and error plan over both the recipe and the plating. BootstrapFinetune sends the apprentice back to culinary school to actually learn new techniques.

Key concepts

LLM frameworks

Detailed answer & concept explanation

~5 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

5 minutes: walk the two axes (what edited, how searched), map each optimizer onto the axes, justify the escalation order from BootstrapFewShot to MIPROv2, and call out BootstrapFinetune as the weight-touching outlier.

Code example

python

import dspy
from dspy.teleprompt import BootstrapFewShot, BootstrapFewShotWithRandomSearch, COPRO, MIPROv2

class QA(dspy.Signature):
    """Answer the question concisely."""
    question: str = dspy.InputField()
    answer: str = dspy.OutputField()

program = dspy.ChainOfThought(QA)

def metric(example, pred, trace=None):
    return example.answer.lower() in pred.answer.lower()

# Cheap baseline: bootstrap a few demos from a teacher
compiled = BootstrapFewShot(metric=metric, max_bootstrapped_demos=4).compile(
    program, trainset=trainset
)

# Broader sweep: random search over candidate demo sets
compiled = BootstrapFewShotWithRandomSearch(metric=metric, num_candidate_programs=10).compile(
    program, trainset=trainset, valset=valset
)

# Instruction tuning: coordinate ascent
compiled = COPRO(metric=metric).compile(program, trainset=trainset, eval_kwargs={})

# Joint search: Bayesian over instructions and demos
compiled = MIPROv2(metric=metric, auto='medium').compile(
    program, trainset=trainset, valset=valset
)

Side-by-side

Optimizer	What it edits	Search strategy	Cost
BootstrapFewShot	Few-shot demos	Greedy bootstrap from teacher	Low
BootstrapFewShotWithRandomSearch	Few-shot demos	Random search over candidate sets	Low to medium
COPRO	Instruction string	Coordinate ascent	Medium
MIPROv2	Instructions plus demos	Bayesian optimization	Medium to high
BootstrapFinetune	Model weights	Gradient updates on bootstrapped traces	High plus GPU time

Real-world examples

Real products, models, and research that use this idea.

Stanford's STORM Wikipedia-generation system uses DSPy with MIPROv2 to tune its multi-stage pipeline
The DSPy documentation's benchmark notebooks consistently rank MIPROv2 above plain BootstrapFewShot on HotpotQA and MATH
Teams using Claude Opus 4.7 as the teacher and Sonnet 4.6 as the student see meaningful BootstrapFewShot gains
BootstrapFinetune is most commonly applied with open-weights models like Llama 4 Maverick where fine-tuning infra is in-house

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QWhen does using a stronger teacher than student matter most in BootstrapFewShot?

QHow does MIPROv2's Bayesian search differ from random search in practice?

QWhy might COPRO underperform MIPROv2 on a task with strong instruction sensitivity?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Conflating 'optimizer' in DSPy with 'optimizer' in deep learning. DSPy optimizers search over prompts and demos, not gradient updates (except BootstrapFinetune).

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.