Part ofAI Product Manager·Week 2: RAG & Use Cases·question 2 of 6View roadmap →

What does LoRA do, and why is it popular for fine-tuning?

MCQ·Medium·4.7 · 156·~1 min·Relevant atCohere Databricks Meta Microsoft

Next in this stepContinue

Similar questions

Same topic, related formats. Practice these next.

4 curated

PPO vs DPO, what's the practical difference?

Flashcard·Medium·4.6 ★

Fill in the memory delta when a 7B weight tensor moves from fp32 to bf16

Fill in blank·Easy·4.0 ★

Prompt-tuning vs LoRA: pick the architectural reason LoRA tends to win

MCQ·Medium·4.0 ★

In LoRA, where in the forward pass does `lora_dropout` apply its mask?

MCQ·Easy·4.0 ★

Next in Top 10 · NVIDIA

In LLM serving, what is the primary driver of end to end latency for a generation request?

MCQ·Medium

from peft import LoraConfig, get_peft_model from transformers import AutoModelForCausalLM base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-4-8B") config = LoraConfig( r=16, # rank lora_alpha=32, # scaling factor target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"], lora_dropout=0.05, bias="none", task_type="CAUSAL_LM", ) model = get_peft_model(base, config) model.print_trainable_parameters() # trainable params: 41M || all params: 8.04B || trainable: 0.51%