Gradient Clipping

Pick the most likely cause when training loss drops for 1k steps, plateaus for 2k, then drops again at step 3k

With max_grad_norm=1.0 and a raw gradient L2 of 12.5, what is the post-clip norm and scale factor?

Gradient norm clipping, name the failure mode it exists to prevent