DataCollator pads to max_length and sets attention_mask = ones: find what breaks

**Two separate bugs** share the same root cause: the collator treats pad tokens as real tokens in two different tensors. The first bug is the attention mask. Setting it to all ones tells the model that every position in the sequence carries real content, including the pad slots. So when the attention layer computes its softmax, it spreads probability across the pad keys too. The contextualised hidden states absorb noise from the pad embeddings, and gradients flow back into those embeddings on every step. The second bug is the label tensor. Setting labels equal to input_ids means cross-entropy is computed at every position, pad slots included. The model is being graded on predicting a pad token given a pad token, which is trivially easy. As more of each batch fills with pad, the loss looks artificially low and the gradient signal on the real response tokens shrinks. The fix is two changes. Build the attention mask from (input_ids != pad_token_id), so pad positions get a zero. And set labels to -100 (the cross-entropy ignore index) at pad positions, so they are skipped in the loss.

**Two coupled errors** around pad handling, both fixable in the collator. The attention mask should be 1 at real tokens and 0 at pad. Setting it to all ones lets every query position attend to pad keys. The softmax then distributes probability across slots that hold no content, the value-side weighted sum mixes pad embeddings into the hidden state, and the gradient pulls the pad embedding in whatever direction the loss demands. The model's representations of real content get diluted, and the pad embedding becomes a learned junk vector. Labels should be -100 (the standard ignore_index) at pad positions. Setting labels equal to input_ids puts cross-entropy on the pad positions too. Predicting pad given pad is trivial, so the loss looks low. Worse, the gradient signal on the response tokens is diluted because the per-token average is dominated by easy pad positions. Training loss and eval loss decouple because the eval metric usually ignores pad correctly. The fixes are independent and both required. Build attention_mask = (input_ids != pad_token_id).long(). Replace labels at pad positions with -100. A common refinement on top is completion-only loss, where prompt tokens are also masked, but that is a separate design choice from the pad bug.