Llama 3

Estimate the trainable param count for a Llama-3 8B LoRA at r=16 targeting all-linear modules.

You want to add 5,000 Hindi medical terms to Llama 3 8B. Walk through the procedure and the risks.

Self-hosted Llama 3 chat returns fluent-looking nonsense. You check and the prompt was tokenized with cl100k_base. What went wrong?

Spot the bug: a Llama-3 fine-tune that skips the end-of-turn token