Explain what LiteLLM solves and why it shows up in almost every multi-provider stack

An **LLM provider SDK** is the client library a vendor ships (OpenAI's `openai` package, Anthropic's `anthropic` package, AWS Bedrock's `boto3` client, Google Vertex's `google-cloud-aiplatform`). Each has its own request shape, its own auth, its own error types, and its own way of streaming. If your application calls all of them directly, every place in the codebase that calls an LLM has to handle the differences. Switching from OpenAI to Anthropic is a code change in every file that calls an LLM. That is fine for a prototype and painful at scale. **LiteLLM** is a library that gives you one interface shaped like the OpenAI Chat Completions API and translates calls behind the scenes to whichever provider you point it at. Your application code calls `litellm.completion(model='claude-3-5-sonnet', messages=...)` or `litellm.completion(model='gpt-4o', messages=...)` and the shape is identical. The immediate win is that provider swap is a config change. The bigger win is that the routing, fallback, rate limit, and budget logic all live in LiteLLM's config and not in your application code. You change behavior centrally instead of touching N call sites.

**LiteLLM** is a Python (and Node, via a proxy server) library that exposes one OpenAI-compatible interface in front of 100-plus LLM providers: OpenAI, Anthropic, Bedrock, Vertex, Azure OpenAI, Cohere, Together, Replicate, Groq, vLLM and TGI for self-hosted, and many others. Application code calls `litellm.completion(model=..., messages=...)` with an OpenAI-shaped payload, and LiteLLM translates to the provider's native API. **Four things it unlocks at once.** First, provider swap moves from a code change to a config change; the application no longer cares which provider answered. Second, fallback chains (try Sonnet then Haiku then a self-hosted Llama) live in the LiteLLM config instead of as conditional logic in application code. Third, cost-routing rules (cheap intent class to the cheap model, hard intent class to the frontier model) sit in one place. Fourth, per-provider rate limits and per-tenant budgets are enforced centrally with a single source of truth. **Two deployment modes.** Library mode: import `litellm` into your service and call it like an SDK. Proxy mode: run the LiteLLM proxy server as a separate service; everything in the stack (Python services, Node services, Langflow workflows, third-party agents) hits the same OpenAI-compatible URL and the proxy applies the policy. Most production stacks adopt the proxy mode once more than two services need LLM access. **The trade-off.** Provider-specific features lag the upstream SDK. Anthropic's tool_choice nuances, OpenAI's structured outputs schemas, the latest prompt-caching headers, and provider-specific streaming events sometimes take days or weeks to land in LiteLLM. Teams that need bleeding-edge features for one specific workload sometimes mix LiteLLM (for the bulk) with the direct provider SDK (for the edge case).