Build a Domain-Specific Embedding Model in Under a Day

Introduction: Overcoming the Walls of RAG Systems

When building a Retrieval-Augmented Generation (RAG) system, you may encounter problems that make it seem like everything is working perfectly. This is often tied to the performance of the embedding model. General embedding models are trained to understand the internet as a whole, so they struggle to grasp the subtle nuances of specific domains. Documents like contracts, manufacturing logs, proprietary chemical formulations, and internal classification schemes are difficult for general embedding models to capture.

But don’t worry! Fine-tuning the embedding model can significantly improve the performance of your retrieval pipeline, especially when the base model doesn’t understand domain-specific nuances. While embedding models play a vital role in RAG performance, the process can often be complex and time-consuming. This article will guide you through how to transform a general embedding model into a domain-optimized model using a single GPU and less than a day of training time.

📚 Step 1: Generating Training Data from Domain Documents

Fine-tuning an embedding model requires thousands of (query, relevant document) pairs. In most cases, these data are not readily available. Manually generating data is costly and time-consuming and can be biased by the annotator’s personal interpretations. You can overcome this challenge by leveraging NeMo Data Designer to use an LLM (nvidia/nemotron-3-nano-30b-a3b) to read documents and automatically generate high-quality question-answer pairs.

nemotron embed sdg -c default corpus_dir=./data/my_domain_docs

Generated QA Pair Example

{ 
  "question": "What is the TDP of H100 SXM?", 
  "answer": "The TDP of H100 SXM is 700W.", 
  "query_type": "contextual", 
  "reasoning_type": "factual", 
  "question_complexity": 3, 
  "segment_ids": [1], 
  "quality_score": 8.5 
}

⛏️ Step 2: Hard Negative Mining (and its Importance)

Training an embedding model using only relevant documents will cause the embedding model to be proficient at distinguishing clearly different documents but fail on ‘nearly relevant’ difficult cases. These Near-misses are the cause of incorrect answers in real-world retrieval systems. Hard Negative Mining finds these confusing clues and trains the embedding model to distinguish them.

nemotron embed prep -c default

🔍 Step 3: Understanding Multi-Hop Questions and Improving Retrieval Performance

Standard embedding model fine-tuning generates a single question for each document and trains the model to match it. This is suitable for simple factual retrieval, but users often ask complex questions spanning multiple documents or sections. If the model only sees single-hop training data, it will struggle to retrieve all relevant documents for these complex queries. The SDG pipeline inherently generates questions with 1-3 hops, allowing the embedding model to learn to retrieve contextually relevant documents.

🧠 Step 4: Fine-tuning a Bi-Encoder Embedding Model

nemotron embed finetune -c default

📈 Step 5: Measuring Performance Improvement

To ensure that fine-tuning actually contributes to performance improvement, you must compare the base model and the fine-tuned checkpoint through standardized evaluation.

🏆 Real-World Results: Atlassian’s Success Story

Atlassian used this recipe to fine-tune Llama-Nemotron-Embed-1B-v2 with a Jira dataset, achieving a remarkable 26% improvement in Recall@60, from 0.751 to 0.951. This demonstrates the powerful effect fine-tuning an embedding model can have in real-world enterprise settings.

🚀 Step 6: Exporting and Deploying the Model

PyTorch checkpoints are useful for evaluation, but in production environments, they are slow. Therefore, you should convert the model to ONNX or TensorRT and deploy it via an API.

nemotron embed deploy -c default

By following these steps, you can maximize the performance of your embedding model and improve the efficiency of your RAG system.

In conclusion, fine-tuning an embedding model is an essential process for improving the performance of RAG systems, and you can easily get started with the methods outlined in this article. With NVIDIA’s support, build a powerful, domain-specific embedding model and maximize the potential of your RAG system.

If you have any additional questions, feel free to contact us! We are committed to helping you explore the world of embedding models together.

In-Depth Analysis and Implications

Array

Original Source: Build a Domain-Specific Embedding Model in Under a Day