{"id":223053,"date":"2025-05-04T15:08:28","date_gmt":"2025-05-04T15:08:28","guid":{"rendered":"https:\/\/peraltafinancing.com\/analytics\/how-to-build-an-intelligent-faq-chatbot-using-agentic-rag\/"},"modified":"2025-05-04T15:08:28","modified_gmt":"2025-05-04T15:08:28","slug":"how-to-build-an-intelligent-faq-chatbot-using-agentic-rag","status":"publish","type":"post","link":"https:\/\/fivemor.com\/?p=223053","title":{"rendered":"How to Build an Intelligent FAQ Chatbot Using Agentic RAG"},"content":{"rendered":"


\n<\/p>\n

\n

AI agents are now a part of enterprises big and small. From filling forms at hospitals and checking legal documents to analyzing video footage and handling customer support<\/a> \u2013 we have AI agents for all kinds of tasks. Companies often spend hundreds of thousands of dollars on hiring customer support staff who can understand the needs of a customer and resolve them based on the company\u2019s guidelines. Today, having an intelligent chatbot to answer FAQs can efficiently improve customer service. In this article, we will learn how to build an FAQ chatbot that can resolve customer queries in seconds, using agentic RAG<\/a> (Retrieval Augmented Generation), LangGraph<\/a> and ChromaDB<\/a>.<\/p>\n

Brief on Agentic RAG<\/h2>\n

RAG is a hot topic nowadays. Everyone is talking about RAG and building applications on top of it. RAG helps LLMs to get access to the real-time data, which makes LLMs more accurate than ever before.\u00a0 However, traditional RAG systems<\/a> tend to fail when it comes to choosing the best retrieval method, changing the retrieval workflow, or providing multi-step reasoning. This is where agentic RAG comes in.<\/p>\n

Agentic RAG enhances traditional RAG by incorporating the capabilities of AI agents into it. With this superpower, RAGs can dynamically change the workflow based on the nature of the query, do multi-step reasoning, and multi-step retrieval as well. We can even integrate tools into the agentic RAG system, and it can dynamically decide which tool to use when. Overall, it results in improved accuracy and makes the system more efficient and scalable.<\/p>\n

Here\u2019s an example of an agentic RAG workflow.<\/p>\n

\"Agentic<\/figure>\n

The image above denotes the architecture of an agentic RAG framework. It shows how AI agents, when combined with RAG, can make decisions under certain conditions. The image clearly shows that if a conditional node is there, the agent will decide which edge to choose based on the context provided.<\/p>\n

Also Read: 10 Business Applications of LLM Agents<\/a><\/em><\/p>\n

Architecture of the Intelligent FAQ Chatbot<\/h2>\n

Now we are going to dive into the architecture of the chatbot we are going to build. We\u2019ll be exploring how it works and what its important components are.<\/p>\n

The following figure shows the overall structure of our system. We will be implementing this using LangGraph, which is an open-source AI agents framework from LangChain.<\/p>\n

\"FAQ<\/figure>\n

The key components of our system include:<\/p>\n

    \n
  1. LangGraph:<\/strong> A powerful open-source AI agent framework that efficiently creates complex, multi-agent, cyclic graph-based agents. These agents can maintain the states throughout the workflow and can efficiently handle the complex queries.<\/li>\n
  2. LLM: <\/strong>An efficient and powerful Large Language Model<\/a> that can follow the instructions of the user and reply accordingly with the best of its knowledge. Here we will be using OpenAI\u2019s o4-mini, which is a small reasoning model that is specifically designed for speed, affordability, and tool use.<\/li>\n
  3. Vector Database: <\/strong>A vector database is used to store, manage and retrieve vector embeddings which are usually the numeric representation of data. Here we are using ChromaDB which is an open source AI native vector database. It is designed to empower the systems that depend on similarity searches, semantic searches, and other tasks involving vector data.<\/li>\n<\/ol>\n

    Also Read: How to Build a Customer Support Voice Agent<\/a><\/em><\/p>\n

    Hands-on Implementation on Building the Intelligent FAQ Chatbot<\/h2>\n

    Now, we will be implementing the end-to-end workflow of our chatbot based on the architecture that we have discussed above. We will be doing it step-by-step with detailed explanations, code, as well as sample outputs. So let\u2019s begin.<\/p>\n

    Step 1: Install Dependencies<\/h3>\n

    We will start by installing all the required libraries into our Jupyter notebook. This includes libraries such as langchain, langgraph, langchain-openai, langchain-community, chromadb, openai, python-dotenv, pydantic, and pysqlite3.<\/p>\n

    !pip install -q langchain langgraph langchain-openai langchain-community chromadb openai python-dotenv pydantic pysqlite3<\/code><\/pre>\n
    Step 2: Import Required Libraries<\/h3>\n
    Now we are ready to import all the remaining libraries that we will need for this project.<\/p>\n
    import os\nimport json\nfrom typing import List, TypedDict, Annotated, Dict\nfrom dotenv import load_dotenv\n\n# Langchain & LangGraph specific imports\nfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings\nfrom langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\nfrom pydantic import BaseModel, Field\nfrom langchain_core.messages import SystemMessage, HumanMessage, AIMessage\nfrom langchain_core.documents import Document\nfrom langchain_community.vectorstores import Chroma\nfrom langgraph.graph import StateGraph, END<\/code><\/pre>\n
    Step 3: Set Up the OpenAI API Key<\/h3>\n
    Enter your OpenAI key to set it as an environment variable.<\/p>\n
    from getpass import getpass\nOPENAI_API_KEY = getpass(\"OpenAI API Key:\")\nload_dotenv()\nos.getenv(\"OPENAI_API_KEY\")<\/code><\/pre>\n
    Step 4: Download the Dataset<\/h3>\n
    We have made a sample FAQ dataset in json format for different departments. We\u2019ll need to download it from the drive and unzip it.<\/p>\n
    !gdown 1j6pdIansfQzKOZSEUinnHd8w6GlkKE6w\n!unzip -o \/content\/blog_faq_files.zip<\/code><\/pre>\n
    Output:<\/strong><\/p>\n
    \"Chatbot<\/figure>\n
    Step 5: Defining the Department Names for Mapping<\/h3>\n
    Now, let\u2019s define the mapping of the departments so that our agentic system can understand which file belongs to which department.<\/p>\n
    # Define Department Names (ensure these match metadata used during ingestion)\nDEPARTMENTS = [\n   \"Customer Support\",\n   \"Product Information\",\n   \"Loyalty Program \/ Rewards\"\n]\nUNKNOWN_DEPARTMENT = \"Unknown\/Other\"\n\nFAQ_FILES = {\n   \"Customer Support\": \"customer_support_faq.json\",\n   \"Product Information\": \"product_information_faq.json\",\n   \"Loyalty Program \/ Rewards\": \"loyalty_program_faq.json\",\n}<\/code><\/pre>\n
    Step 6: Define the Helper Functions<\/h3>\n
    We will define some helper functions which will be responsible for loading FAQs from the json files and also storing them in ChromaDB.<\/p>\n
    1. load_faqs(\u2026):<\/strong> It is a helper function which loads the FAQ from the json files and store them in a list called all_faqs.<\/p>\n
    def load_faqs(file_paths: Dict[str, str]) -> Dict[str, List[Dict[str, str]]]:\n   \"\"\"Loads QA pairs from JSON files for each department.\"\"\"\n   all_faqs = {}\n   print(\"Loading FAQs...\")\n   for dept, file_path in file_paths.items():\n       try:\n           with open(file_path, 'r', encoding='utf-8') as f:\n               all_faqs[dept] = json.load(f)\n               print(f\"  - Loaded {len(all_faqs[dept])} FAQs for {dept}\")\n       except FileNotFoundError:\n           print(f\"  - WARNING: FAQ file not found for {dept}: {file_path}. Skipping.\")\n       except json.JSONDecodeError:\n           print(f\"  - ERROR: Could not decode JSON for {dept} from {file_path}. Skipping.\")\n   return all_faqs<\/code><\/pre>\n
    2. setup_chroma_vector_store(\u2026): <\/strong>This function sets up the ChromaDB to store the vector embeddings. For this, we will first define the Chroma configuration i.e., the directory which will contain the chroma database files. Then we will convert the FAQs to LangChain\u2019s Documents. It will contain metadata and page content which is the predefined format for an accurate RAG. We can combine question and answers for better contextual retrieval or just embed the answer. We are keeping the question as well department name in the metadata.<\/p>\n
    # ChromaDB Configuration\nCHROMA_PERSIST_DIRECTORY = \".\/chroma_db_store\"\nCHROMA_COLLECTION_NAME = \"Chatbot_faqs\"\n\ndef setup_chroma_vector_store(\n   all_faqs: Dict[str, List[Dict[str, str]]],\n   persist_directory: str,\n   collection_name: str,\n   embedding_model: OpenAIEmbeddings,\n) -> Chroma:\n   \"\"\"Creates or loads a Chroma vector store with FAQ data and metadata.\"\"\"\n   documents = []\n   print(\"\\nPreparing documents for vector store...\")\n   for department, faqs in all_faqs.items():\n       for faq in faqs:\n           # Combine Q&A for better contextual embedding, or just embed answers\n           # content = f\"Question: {faq['question']}\\nAnswer: {faq['answer']}\"\n           content = faq['answer'] # Often embedding just the answer is effective for FAQ retrieval\n           doc = Document(\n               page_content=content,\n               metadata={\n                   \"department\": department,\n                   \"question\": faq['question'] # Keep question in metadata for potential display\n                   }\n           )\n           documents.append(doc)\n\n   print(f\"Total documents prepared: {len(documents)}\")\n\n   if not documents:\n       raise ValueError(\"No documents found to add to the vector store. Check FAQ loading.\")\n\n   print(f\"Initializing ChromaDB vector store (Persistence: {persist_directory})...\")\n   vector_store = Chroma(\n       collection_name=collection_name,\n       embedding_function=embedding_model,\n       persist_directory=persist_directory,\n   )\n   try:\n     vector_store = Chroma.from_documents(\n             documents=documents,\n             embedding=embedding_model,\n             persist_directory=persist_directory,\n             collection_name=collection_name\n             )\n     print(f\"Created and populated ChromaDB with {len(documents)} documents.\")\n     vector_store.persist() # Ensure persistence after creation\n     print(\"Vector store persisted.\")\n   except Exception as create_e:\n           print(f\"FATAL ERROR: Could not create Chroma vector store: {create_e}\")\n           raise create_e\n\n   print(\"ChromaDB setup complete.\")\n   return vector_store<\/code><\/pre>\n
    Step 7: Define the LangGraph Agent Components<\/h3>\n
    Let\u2019s now define our AI agent component which is the main component of our work flow.<\/p>\n
    1. State definition:<\/strong> It is a python class containing the current state of the agent while running. It contains variables such as query, sentiment, department.<\/p>\n
    class AgentState(TypedDict):\n   query: str\n   sentiment: str\n   department: str\n   context: str # Retrieved context for RAG\n   response: str # Final response to the user\n   error: str | None # To capture potential errors<\/code><\/pre>\n
    2. Pydantic model:<\/strong> We have defined a pydantic model<\/a> here which will ensure a structured LLM output. It contains a sentiment which will have three values, \u201cpositive\u201d, \u201cnegative\u201d and \u201cneutral\u201d and a department name which will be predicted by the LLM.<\/p>\n
    class ClassificationResult(BaseModel):\n   \"\"\"Structured output for query classification.\"\"\"\n   sentiment: str = Field(description=\"Sentiment of the query (positive, neutral, negative)\")\n   department: str = Field(description=f\"Most relevant department from the list: {DEPARTMENTS + [UNKNOWN_DEPARTMENT]}. Use '{UNKNOWN_DEPARTMENT}' if unsure or not applicable.\")<\/code><\/pre>\n
    3. Nodes:<\/strong> The following are the node functions which will handle each task one by one.<\/p>\n
      \n