Serving Frameworks

Heavy multi-turn chat shares system prompts across requests, pick the serving framework that exploits that best.

Recognize vLLM and the optimization that made it famous

Spell out TensorRT-LLM and pinpoint its production differentiator

Name TGI's maintainer and its niche among serving stacks