Fine-tuning

What is fine-tuning?

Large language models are trained on massive, general datasets. This gives them broad capabilities but no specialization. Fine-tuning is a second training step: you take a pre-trained model and continue training it on a smaller, task-specific dataset. The model's weights are adjusted to perform better on your specific use case.

A legal firm might fine-tune a model on thousands of contracts to make it better at legal drafting. A bank might fine-tune on financial reports to improve analysis quality. A customer service team might fine-tune on past support tickets to make responses match their tone and policies.

Fine-tuning vs. RAG: which should you use?

This is one of the most common questions in enterprise AI, and the answer is usually: RAG first, fine-tuning if needed. RAG (Retrieval-Augmented Generation) gives the model access to your documents at query time without modifying the model itself. It is faster to implement, easier to update, and provides citations. For most knowledge retrieval use cases, RAG is sufficient.

Fine-tuning is the right choice when: you need the model to internalize a specific writing style or format (not just access information), the task requires consistent behavioral changes that are hard to achieve with prompting, or you need to distill a large model's capabilities into a smaller, faster model for a specific task.

The cost and complexity of fine-tuning

Fine-tuning requires a quality training dataset (typically 500–10,000 examples for instruction fine-tuning), GPU compute for the training run, and evaluation infrastructure to measure improvement. For open models, a fine-tuning run for a 7B parameter model takes 2–8 hours on a single A100 GPU. For larger models, it scales accordingly.

The more significant cost is data preparation: curating, cleaning, and formatting training examples is time-intensive and requires domain expertise. Plan for 2–4 weeks of data work before the first fine-tuning run.