All topics
24 GenAI sub-disciplines organized by category. Click any topic for its question library and atomic flashcard deck.
Foundations
4Attention Mechanism
The mechanism that lets each token attend to every other token. Self-attention, multi-head, scaled dot-product, MQA, GQA, FlashAttention, KV cache.
Tokenization
How text becomes tokens. BPE, SentencePiece, WordPiece, vocabulary tradeoffs, special tokens, and why tokenizer choice matters at inference time.
Embeddings
Vector representations of meaning. How embedding models are trained, similarity metrics, dimensionality, and choosing the right model for retrieval.
Transformer Architecture
End-to-end design of the transformer block: embeddings, attention, FFN, residual, normalization, positional encoding. Encoder vs decoder vs encoder-decoder.
Training & Alignment
3Pretraining
Building base models from scratch. Data curation, compute budgets, scaling laws (Kaplan, Chinchilla), distributed training, synthetic data, contamination.
Fine-Tuning
Adapting base models for downstream use. SFT, LoRA, QLoRA, PEFT, distillation. When to fine-tune vs RAG vs prompt engineering.
RLHF & Alignment
Teaching models to follow human preferences. Reward modeling, PPO, DPO, KTO, constitutional AI, preference data, the alignment tax.
Building with LLMs
3Prompt Engineering
Production prompting: few-shot, chain-of-thought, structured output, JSON mode, function calling, system prompts, temperature and sampling.
Context Engineering
Managing what the model sees. Context window strategy, memory, compression, lost-in-the-middle, summarization-as-context, RAG-as-context patterns.
Safety & Guardrails
Deploying LLMs safely. Content filters, Llama Guard, NeMo Guardrails, prompt injection defense, jailbreak detection, red-teaming, OWASP LLM Top 10.
Retrieval Augmented Generation (RAG)
2Retrieval-Augmented Generation
Grounding LLM responses in external knowledge. Chunking strategies, retrieval, reranking, query rewriting, hybrid search, generation, end-to-end evaluation.
Vector Databases
Storing and searching embeddings at scale. HNSW, IVF, PQ, DiskANN, hybrid filtering, sharding, vendor selection (Pinecone, Weaviate, Qdrant, Milvus, pgvector).
Agentic AI
3AI Agents
Multi-step LLM systems. ReAct, Plan-and-Execute, memory, tool use, planning, reflection, multi-agent orchestration, trajectory evaluation.
Multi-Agent Systems
Coordinating multiple agents. Hierarchies, supervisor-worker patterns, message passing, debate, consensus, shared scratchpads, role specialization.
Model Context Protocol
Anthropic's open protocol for connecting LLMs to tools and data. Spec, server development, transports, auth, tool / resource / prompt schemas.
Production & Inference
3LLM Evaluation
Measuring model quality. BLEU, ROUGE, LLM-as-judge, rubrics, faithfulness, hallucination detection, A/B testing, benchmark design, RAG eval.
Inference Optimization
Serving LLMs at scale. KV cache, FlashAttention, speculative decoding, quantization, batching, vLLM, TensorRT-LLM, TTFT vs TPOT tradeoffs.
LLMOps
MLOps for LLMs. Model and prompt versioning, registries, canary deploys, online evaluation, A/B testing in prod, rollback policies, CI/CD for AI.
Advanced Model Architectures
3Reasoning Models
Frontier inference-time compute. o1, o3, DeepSeek R1, thinking tokens, process reward models, extended chain-of-thought, MCTS-style search.
Multimodal AI
Beyond text. Vision-language models, image tokens, diffusion (Stable Diffusion, Flux), audio (Whisper, voice agents), video generation (Sora, Veo).
Mixture of Experts
Sparse activation at scale. Routing, top-k gating, expert parallelism, load balancing, MoE serving cost, Mixtral / Llama 4 / DeepSeek architectures.
Tooling & Frameworks
2LLM Frameworks
Picking and using the right abstractions. LangChain, LlamaIndex, DSPy, AutoGen, CrewAI, Mastra, Vercel AI SDK, when to use each and their tradeoffs.
Observability & Tracing
Debugging and monitoring LLM apps in production. LangSmith, Langfuse, Phoenix, Helicone, trace structure, span hierarchies, cost and latency tracking.