Complete the RMSNorm formula

**RMSNorm has a single learnable scale and a small numerical-stability constant.** The full formula is: $$\text{RMSNorm}(x) = \gamma \cdot \frac{x}{\sqrt{\text{mean}(x^2) + \epsilon}}$$ The mean of squared elements is taken along the feature axis. Taking the square root gives the **root mean square** of x, hence the name. Dividing x by that quantity rescales it so the typical magnitude is 1. Gamma is a learnable vector with one entry per model-width feature that gives the model a per-feature scale knob after normalization. Without it, every normalized output would have the same magnitude, which would limit expressivity. Epsilon is a tiny constant, typically around 1e-5 or 1e-6, added inside the square root to prevent division by zero when x is exactly all zeros. It does not affect normal operation; it is purely a numerical safeguard. Notice what is missing relative to LayerNorm. There is no mean subtraction (so RMSNorm does not recenter), and there is no learnable bias term. Both omissions are deliberate: they make RMSNorm cheaper and have been shown empirically to lose no quality.

**The two missing terms are `gamma` (the learnable per-feature scale) and `eps` (a small numerical-stability constant inside the square root).** The complete formula is: $$\text{RMSNorm}(x) = \gamma \cdot \frac{x}{\sqrt{\text{mean}(x^2) + \epsilon}}$$ The gamma is a learnable vector of shape `(d_model,)` initialized to all ones; it gives the model one scale parameter per feature to recover the post-norm magnitude it wants. The eps is a fixed scalar, typically `1e-5` or `1e-6`, that prevents the denominator from being zero when `x` is exactly the zero vector. Llama uses `1e-5`; some implementations use `1e-6` or `1e-8`. **Compared to LayerNorm**, RMSNorm drops two things. First, it drops the mean subtraction: LayerNorm subtracts `mean(x)` before normalizing, RMSNorm does not. Second, it drops the learnable bias `beta`: LayerNorm has both a scale `gamma` and a bias `beta`, RMSNorm has only the scale. **Why those omissions are safe.** Zhang and Sennrich (2019) showed empirically that for transformer blocks, the mean-subtraction step contributes essentially nothing to final quality; what mattered was the rescaling. Dropping the mean subtraction saves one reduction pass over the feature axis. Dropping the bias saves a small number of parameters per layer. Together, RMSNorm is roughly 30 to 50 percent faster than LayerNorm on modern GPUs and has been adopted in Llama 1, 2, 3, Mistral, Qwen, and DeepSeek decoder LLMs.