Your input is 9,000 tokens but max_length is 8,192. What are your truncation options?

**Truncation** is what the tokenizer does when an input has more tokens than the model can fit. Every model has a maximum context length (`model_max_length`), and any input over that limit either has to be cut down or rejected. HuggingFace exposes four named truncation strategies, picked via the `truncation` argument: - `longest_first` (the default when you write `truncation=True`) trims from the longer of a paired input one token at a time until both fit. - `only_first` always trims the first input in a pair. - `only_second` always trims the second input. - `do_not_truncate` raises an error if the input is over `max_length` instead of cutting. For a single input (not a pair), the strategies all collapse to the same behavior: drop tokens from the end. In chat applications, the named strategies are rarely enough by themselves. A typical pattern is **manual middle truncation**: keep the system prompt at the start, keep the most recent few turns at the end, and drop turns from the middle. The middle turns are usually the least important to the current reply. The recurring production bug is silent truncation: the tokenizer cuts content without warning the application, the model never sees the dropped tokens, and answers degrade in a way that is hard to trace. Always log when truncation fires, by checking the difference between the pre truncation and post truncation token counts.

**Truncation** drops tokens from an input that exceeds `max_length` so the result fits the model's context window. It is the tokenizer's response to the hard limit imposed by `model_max_length` or by the application's chosen budget. HuggingFace exposes four named strategies on the `truncation` argument. `longest_first`, the default for `truncation=True`, trims from the longer of paired inputs one token at a time; useful for question and context pairs in retrieval where you want both sides to be present. `only_first` always cuts from the first input; `only_second` always cuts from the second; both are useful when one side carries the irreducible content. `do_not_truncate` raises an error rather than cutting, which is the right choice when silent loss is unacceptable. For a single input these all reduce to trimming from the end. The truncation side defaults to `right` (drop the tail) but `tokenizer.truncation_side = 'left'` flips it for use cases where the tail matters most, like chat history where the most recent turns are at the end. None of the built in strategies handle chat history well. The standard production pattern is manual middle truncation: keep the system prompt at the top, keep the recent few user and assistant turns, drop turns from the middle by oldest first. The system prompt and the recent context are usually what the model needs. The recurring production bug is silent truncation. The tokenizer cuts and returns ids; the application has no idea content was dropped, and quality degrades without an obvious symptom. Always log when truncation fires by comparing pre truncation and post truncation token counts, or by checking the `overflow_to_sample_mapping` and `num_truncated_tokens` fields the tokenizer can return.