You hear about 'eval harnesses' like Promptfoo and DeepEval. Explain what an eval harness does that a Jupyter notebook cannot.

An **eval harness** is a framework that automates the lifecycle of LLM evaluation. It loads test cases from a defined source, calls the model with each input, applies scoring functions to the outputs, aggregates results, and stores them for comparison with future runs. A Jupyter notebook can do the first three steps manually. You can write a loop that calls the API, score the outputs, and print the results. But the notebook falls short in three critical areas. First, repeatability. A notebook depends on the person running it to set up the environment, parameters, and test cases correctly every time. A harness reads a config file and runs identically regardless of who triggers it. Second, comparison. After you run an eval in a notebook, you have numbers in a cell output. Next week, you run it again and get new numbers. Comparing them means scrolling through cells and doing manual arithmetic. A harness stores every run, timestamps it, and produces a diff showing exactly what improved and what regressed. Third, enforcement. A notebook cannot block a pull request. A harness integrated into CI/CD can. When a prompt change causes scores to drop below a threshold, the harness fails the CI check and prevents the merge. This is what turns evaluation from a suggestion into a gate.

An **eval harness** is a framework that automates the full eval lifecycle: loading test cases, calling the model, applying metrics, aggregating scores, comparing runs, and integrating with CI/CD pipelines. A Jupyter notebook handles one-off exploration well. You can prototype a scoring function, run it on a handful of examples, and inspect outputs interactively. But notebooks break down at the team level for three reasons. Repeatability: a harness reads a declarative config (YAML, JSON, or code) and runs the same eval identically every time. A notebook relies on cell execution order and runtime state, which drift between sessions. Comparison: harnesses store every run with metadata and produce diffs showing score deltas across versions. Notebooks leave comparison to manual effort. Enforcement: harnesses integrate into CI/CD as merge-blocking gates. If a prompt change drops the summarization ROUGE score below 0.72, the pull request fails. Notebooks have no way to block a merge. In 2026, the major harnesses are Promptfoo (YAML-first, strong CI integration), DeepEval (Python-native, built-in metrics library), Braintrust (SaaS with experiment tracking), and LangSmith (tracing plus eval in one platform). Each wraps the same core loop: load cases, run model, score, aggregate, compare, gate. The harness does not replace good eval design. It automates execution and comparison. If your test cases are weak or your metrics are misaligned with user needs, the harness just runs bad evals faster. The value is in making good eval design repeatable and enforceable.