API LLM
Also known as: Hosted LLM, Provider API
An LLM accessed through a provider API: pay per token, get the frontier model, hand over ops.
A large language model accessed through a provider API (OpenAI, Anthropic, Google, Bedrock). The provider owns weights, inference infrastructure, and scaling; you pay per input and output token. Ops burden is minimal and the model ceiling is the frontier, at the cost of data leaving your premises unless you use a private-cloud tier.
In practice
The API-vs-self-hosted decision is a canonical applied-AI interview question. Break-even volume, data residency, and fine-tuning control all trace back to this choice.
How it compares
APIs give you the frontier model with zero ops; self-hosting gives you data control, unrestricted fine-tuning, and predictable cost past break-even.
Comparisons that include API LLM
Related topics
Questions that mention this term
- Flashcard: what is a stop sequence in an LLM API call and what is it used for?
- The model provider just updated the model behind the API endpoint. Describe the regression testing process before allowing traffic.
- Identify the three highest leverage sources of vendor lock in in a hosted LLM stack
- Why does the 'frameworks abstract away the provider' pitch leak in practice?
- Sequence a robust retry policy for a transient LLM API failure
- Define a stop sequence in an LLM API call
Related terms
Beam Search
Keep the K best partial sequences at each step; deterministic, breadth-first decoding.
FlashAttention
A memory-aware attention kernel that's 2-4x faster than vanilla, with identical math.
GGUF
Self-contained binary format for quantized LLMs; the standard for llama.cpp / Ollama / LM Studio.
Greedy Decoding
At each step, pick the single highest-probability token. Fast and deterministic, but often loops.
Grouped-Query Attention (GQA)
Compromise between MHA and MQA: query heads share KV heads in groups, cutting KV cache by 4-8x.
Knowledge Distillation
Train a small student model to match a big teacher's outputs: cheap, fast inference with most of the quality.