Skip to main content
XenonStack
Fine-Tuning: Methods, LoRA and When to Use It
J
K
N
O
R
X
Y
Z

XenonStack Glossary Fine-Tuning: Methods, LoRA and When to Use It

Key takeaway

  • Fine-tuning adapts a pre-trained model to a specific task using your own examples. See SFT, DPO and LoRA methods, and why to try prompting first.

Fine-tuning is a machine learning technique that makes small adjustments to a pre-trained model so it performs better on a specific task. It is usually more efficient than training a model from scratch, because the model already carries useful knowledge from its original training.

This page is for teams deciding how to adapt a language, vision or other model to their own work. It covers what fine-tuning is, why teams use it, the main methods, how parameter-efficient approaches such as LoRA differ, and what to settle first.

What is fine-tuning?

OpenAI's model optimisation guide describes fine-tuning as taking a base model, providing the kinds of inputs and outputs you expect in your application, and getting a model that excels at those tasks. In practice you pick a pre-trained model, prepare a task-specific dataset in the format the model expects, and continue training so the model's behaviour shifts towards your task. Depending on the task, you may also change the architecture, for example the number of output units in the final layer of a classifier. A closely related technique is universal language model fine-tuning.

Two-column diagram. Full fine-tuning: trains all weights, highest memory, separate copies, same quality goal. LoRA: base weights frozen, small update matrices, light adapters, no added latency.

Why do teams fine-tune a model?

The same guide lists several reasons: fitting more examples than a context window can hold, shortening prompts to cut token cost and latency, avoiding sensitive data in every request, and training a smaller, cheaper model for one task. Other benefits follow from starting with a pre-trained model.

  • Knowledge transfer: patterns learned in initial training carry over to your task.
  • Data efficiency: you can adapt a model with limited task-specific data rather than building a new dataset from nothing.
  • Specialisation: the model is tuned to behave well in your specific situation.
  • Time and compute: training starts from learned features, so it generally needs less than a full training run.

What are the main fine-tuning methods?

OpenAI's guide lists three methods for its models, each suited to a different kind of problem.

MethodWhat you provideBest suited to
Supervised fine-tuningExamples of correct responses to promptsClassification, nuanced translation, specific output formats, fixing instruction-following failures
Vision fine-tuningImage inputs for supervised fine-tuningImage classification and complex image prompts
Direct preference optimisationA correct and an incorrect response for each promptSummaries that focus on the right points, and matching tone and style

Check the current documentation of your provider, because supported models and availability change. The OpenAI page itself notes that the platform is being wound down for new users.

How does LoRA differ from full fine-tuning?

Full fine-tuning updates all of a model's weights. LoRA, or low-rank adaptation, keeps the pre-trained weights frozen and trains small additional matrices instead. The LoRA paper reports that, compared with fully fine-tuning GPT-3 175B with Adam, LoRA reduced the trainable parameters by 10,000 times and the GPU memory requirement by 3 times, while matching or beating full fine-tuning on the models tested. The Hugging Face PEFT guide adds that you can keep several lightweight adapters for different tasks on top of one base model, and that adapter weights can be merged into the base model so there is no added inference latency.

QuestionFull fine-tuningLoRA
What is trained?All model weightsSmall update matrices, with base weights frozen
Memory and parametersHighestFar fewer trainable parameters
Several tasksSeparate full copiesLightweight adapters on one base model

How does fine-tuning relate to AI agents?

A fine-tuned model is one component of an agent, not the whole agent. The agent still needs tools, context and rules about what it may do. Platforms such as ElixirClaw, XenonStack's AI agents computing platform, sit in the layer where models are put to work.

What should you do first?

  1. Write evaluations that measure output quality and set a baseline, as OpenAI's guide recommends.
  2. Try prompt engineering, which the guide says may be all you need.
  3. If it falls short, collect a clean, task-specific training dataset.
  4. Choose a method, supervised, preference-based or parameter-efficient, to fit the problem.
  5. Evaluate on data that looks like real inputs, then adjust the prompt or dataset and repeat.

To map one decision your agents make, and the data and policy it depends on, book a Context Workshop. See what happens in the workshop.

Frequently asked questions

What is the difference between fine-tuning and training from scratch?

Fine-tuning starts from a pre-trained model and adjusts it for a task, while training from scratch begins with no learned knowledge. Fine-tuning is usually more efficient.

Should I try prompting before fine-tuning?

Yes. OpenAI's guide says prompt engineering may be all you need, and it recommends writing evaluations first so you can measure whether fine-tuning helps.

What is LoRA?

LoRA is a method that keeps the pre-trained weights frozen and trains small low-rank update matrices, which sharply reduces the number of trainable parameters.

What data does fine-tuning need?

A dataset relevant to your task, ideally with labelled examples for supervised tasks, organised in the input format the pre-trained model expects.

Does fine-tuning replace the need for context or rules in an AI agent?

No. Fine-tuning changes how a model behaves, but an agent still needs tools, current context and policy that define what it may do.

Interested in solving your challenges with the XenonStack team?
Get Started

Tell us about you and your priorities so we can prepare the right conversation. It takes about 2 minutes.

Please Select your Industry
Banking
Fintech
Payment Providers
Wealth Management
Discrete Manufacturing
Semiconductor
Machinery Manufacturing / Automation
Appliances / Electrical / Electronics
Elevator Manufacturing
Defense & Space Manufacturing
Computers & Electronics / Industrial Machinery
Motor Vehicle Manufacturing
Food and Beverages
Distillery & Wines
Beverages
Shipping
Logistics
Mobility (EV / Public Transport)
Energy & Utilities
Hospitality
Digital Gaming Platforms
SportsTech with AI
Public Safety - Explosives
Public Safety - Firefighting
Public Safety - Surveillance
Public Safety - Others
Media Platforms
City Operations
Airlines & Aviation
Defense Warfare & Drones
Robotics Engineering
Drones Manufacturing
AI Labs for Colleges
AI MSP / Quantum / AGI Institutes
Retail Apparel and Fashion

Continue