Replicate

What is the KV cache in transformer inference?

Identify valid evaluation methods for a post-trained aligned model

What is the goal and risk of injecting noise into pre-softmax attention scores?

Calculate the per token KV cache footprint of a Llama-3-8B style config in fp16

Compare ALiBi's linear distance bias with RoPE's vector rotation. Which one modifies scores additively, and which modifies Q and K directly?

What does ALiBi stand for, and where does it add position in the attention pipeline?

Reward model in RLHF: what does it score, and on what data does it train?

If attention temperature divides QK^T and sampling temperature divides the output logits, where is each one applied?

In a 7B transformer, do attention or MLP layers use more FLOPs, and does context length change that?

Order a post-training pipeline from base checkpoint to deployed assistant