Modern language models excel at general tasks but often need customization for specific applications. The natural starting point is prompt engineering — using clear instructions and few-shot examples to guide behavior. For missing information like internal documents or updated policies, Retrieval-Augmented Generation (RAG) supplies relevant context during inference without changing model parameters.

When prompting and RAG leave gaps — such as persistent classification errors or inconsistent writing styles — fine-tuning becomes valuable. Fine-tuning trains the model on many examples so desired behaviors become part of its default parameters. Techniques like LoRA (Low-Rank Adaptation) and QLoRA (Quantized LoRA) make this practical by reducing compute and memory requirements. LoRA learns a smaller set of weight changes rather than updating all parameters, while QLoRA further compresses the model using 4-bit quantization.

The article outlines a progression: start with prompts and RAG, then apply parameter-efficient fine-tuning when needed. This layered approach balances effort, cost, and performance for teams adapting models to production requirements.