Optimizing a RAG-Based Helpdesk Chatbot: Improving Accuracy with pgvector
Introduction In my previous post, I covered how I implemented pgvector in a RAG (Retrieval-Augmented Generation) system for a helpdesk chatbot. While the system performed well, accuracy issues arose: Some retrieved helpdesk articles weren’t fully relevant The chatbot sometimes misinterpreted the query context Long answers from the retrieved content confused GPT To fix these, I optimized the retrieval process, improved embedding quality, and refined GPT’s response generation. In this post, I’ll walk through the optimizations that improved the model’s accuracy. Key Optimizations for Higher Accuracy To improve accuracy, I focused on three key areas: 1️⃣ Enhanced Query Preprocessing: Cleaning and reformulating user queries 2️⃣ Better Retrieval Strategies: Improving pgvector search results 3️⃣ Refining Response Generation: Giving GPT a structured context 1. Enhancing Query Preprocessing User queries are often ambiguous, unstructured, or too short. For example: User input: "VPN issue" → Too vague Better reformulated query: "How to fix VPN connection issues on Windows?" Improvements in Query Preprocessing ✅ Synonym Expansion: Expanding user queries with relevant synonyms ✅ Query Normalization: Lowercasing, removing special characters ✅ Prompt Expansion: Reformulating short queries into full questions Example: Query Expansion Using NLP To improve query quality, I used Natural Language Processing (NLP) techniques for synonym-based expansion using WordNet. This helps broaden the search scope and retrieve more relevant documents in a RAG-based system. ⚠ Experimental Feature: This API is not stable and may undergo significant changes. Use it with caution in production environments. Optimized TypeScript Implementation I used Node.js + TypeScript with the natural library, which provides a WordNet interface. import natural from "natural"; const wordnet = new natural.WordNet(); async function expandQuery(query: string): Promise { const words = query.split(" "); const expandedWords: string[] = []; for (const word of words) { const synonyms = await getSynonyms(word); expandedWords.push(synonyms.length > 0 ? synonyms[0] : word); // Use first synonym if available } return expandedWords.join(" "); } function getSynonyms(word: string): Promise { return new Promise((resolve) => { wordnet.lookup(word, (results) => { if (results.length > 0) { resolve(results[0].synonyms); } else { resolve([]); } }); }); } How It Works Splits the user’s query into individual words. Fetches synonyms from WordNet for each word. Replaces words with the first available synonym (if found). Reconstructs the modified query and returns it. Example Output
Introduction
In my previous post, I covered how I implemented pgvector in a RAG (Retrieval-Augmented Generation) system for a helpdesk chatbot. While the system performed well, accuracy issues arose:
- Some retrieved helpdesk articles weren’t fully relevant
- The chatbot sometimes misinterpreted the query context
- Long answers from the retrieved content confused GPT
To fix these, I optimized the retrieval process, improved embedding quality, and refined GPT’s response generation. In this post, I’ll walk through the optimizations that improved the model’s accuracy.
Key Optimizations for Higher Accuracy
To improve accuracy, I focused on three key areas:
1️⃣ Enhanced Query Preprocessing: Cleaning and reformulating user queries
2️⃣ Better Retrieval Strategies: Improving pgvector search results
3️⃣ Refining Response Generation: Giving GPT a structured context
1. Enhancing Query Preprocessing
User queries are often ambiguous, unstructured, or too short. For example:
- User input: "VPN issue" → Too vague
- Better reformulated query: "How to fix VPN connection issues on Windows?"
Improvements in Query Preprocessing
✅ Synonym Expansion: Expanding user queries with relevant synonyms
✅ Query Normalization: Lowercasing, removing special characters
✅ Prompt Expansion: Reformulating short queries into full questions
Example: Query Expansion Using NLP
To improve query quality, I used Natural Language Processing (NLP) techniques for synonym-based expansion using WordNet. This helps broaden the search scope and retrieve more relevant documents in a RAG-based system.
⚠ Experimental Feature: This API is not stable and may undergo significant changes. Use it with caution in production environments.
Optimized TypeScript Implementation
I used Node.js + TypeScript with the natural library, which provides a WordNet interface.
import natural from "natural";
const wordnet = new natural.WordNet();
async function expandQuery(query: string): Promise<string> {
const words = query.split(" ");
const expandedWords: string[] = [];
for (const word of words) {
const synonyms = await getSynonyms(word);
expandedWords.push(synonyms.length > 0 ? synonyms[0] : word); // Use first synonym if available
}
return expandedWords.join(" ");
}
function getSynonyms(word: string): Promise<string[]> {
return new Promise((resolve) => {
wordnet.lookup(word, (results) => {
if (results.length > 0) {
resolve(results[0].synonyms);
} else {
resolve([]);
}
});
});
}
How It Works
- Splits the user’s query into individual words.
- Fetches synonyms from WordNet for each word.
- Replaces words with the first available synonym (if found).
- Reconstructs the modified query and returns it.
Example Output
