What an internal model card looks like for a deployed fine-tune

An internal model card is the **spec sheet** for a fine-tuned model. It has eight sections that together answer 'what is this, what is it for, and who owns it': - **Identity**: model name, version, base model (Llama 4, Qwen 3.5, Mistral), training-data version. - **Purpose**: one paragraph naming the task, the input shape (raw email body, ticket text), and the output shape (one of seven intent labels with confidence). - **Training summary**: dataset size, label distribution, train/eval split, hyperparameters (learning rate, epochs, LoRA rank if applicable). - **Evaluation**: scores on the golden set with confidence intervals; the metrics that matter for the task (accuracy, F1, per-class precision and recall). - **Known limitations**: intent classes the model handles poorly, language coverage, demographic-bias check results. - **Data provenance**: where the training data came from, consent posture, PII handling. - **Deployment scope**: which services call this model, which tenants it serves, the production region. - **Owner and on-call**: the team that owns it and the rotation to page during incidents. The card lives in source control next to the model artifact and updates with every retrain. Three audiences read it: compliance auditors, new engineers, and the on-call during incidents.

**An internal model card is a one-page (or one-document) artifact that answers what the model is, what it was trained on, what it has been tested on, what it is known to fail at, who can use it, and who to page when it breaks.** Eight standard sections: - **Identity**: model name, semver version, base model (Llama 4 Maverick, Qwen 3.5, Mistral, etc.), training-data version hash, weights artifact hash, date. - **Purpose**: one paragraph stating the task, the input shape, the output shape. For a support classifier: 'Classifies a customer email or ticket body into one of seven intent labels with a confidence score; expects raw text up to 4000 tokens; returns label and confidence.' - **Training summary**: dataset size, label distribution (per-class counts, class imbalance ratios), train and eval split methodology, hyperparameters (learning rate schedule, epochs or steps, LoRA rank and alpha if applicable), compute used (GPU type and hours), base-model version it was fine-tuned from. - **Evaluation results**: scores on the golden set with confidence intervals; per-class precision, recall, F1; aggregate accuracy; calibration plots if confidence is exposed downstream; comparison against the prior model version. - **Known limitations**: intent classes the model handles poorly (with examples), language coverage (English-only? multilingual? per-language F1), demographic-bias check results (slice F1 by region or by tenant size), failure modes the team has observed in production. - **Data provenance**: where the training data came from (production tickets, synthetic data, vendor-purchased), consent posture (opt-in, contractual right, anonymized only), PII handling (was the data redacted? what regex was used? what was missed?), retention policy on the source data. - **Deployment scope**: which services call this model, which tenants it serves, the production regions, the SLO it sits under, the fallback chain it participates in. - **Owner and on-call**: the team, the