Retrieval-Augmented Generation (RAG)<\/a> systems enhance generative AI capabilities by integrating external document retrieval to produce contextually rich responses. With the release of GPT 4.1, characterized by exceptional instruction-following, coding excellence, long-context support (up to 1 million tokens), and notable affordability, building agentic RAG systems becomes more powerful, efficient, and accessible. In this article, we\u2019ll discover what makes GPT-4.1 so powerful and learn how to build an agentic RAG system using GPT-4.1 mini.<\/p>\n GPT 4.1 significantly improves upon its predecessors, providing substantial gains in:<\/p>\n OpenAI has rolled out the GPT-4.1 lineup, including three models: GPT-4.1, GPT-4.1 Mini, and GPT-4.1 Nano. Here is what it offers:<\/p>\n One of the headline features is the 1-million-token context window<\/strong> \u2013 a first for OpenAI. You can now feed in massive<\/em> blocks of code, research papers, or entire document sets in one go. That said, while it handles scale impressively, pinpoint accuracy fades as the input grows, so it\u2019s best used for broad context understanding rather than surgical precision.<\/p>\n When it comes to programming, GPT-4.1 steps up significantly:<\/p>\n GPT-4.1 is now more responsive to multi-step instructions<\/strong> and nuanced formatting rules. Whether you\u2019re designing workflows or building AI agents, this model is much better at doing what you actually<\/em> ask for.<\/p>\n This version is optimized for performance and affordability:<\/p>\n Read this article to know more: All About OpenAI\u2019s Latest GPT 4.1 Family<\/a><\/p>\n I am building a multi-document, agentic RAG system with GPT 4.1 mini. Here\u2019s the workflow.<\/p>\n I am pinning specific versions of LangChain<\/a> packages and related dependencies for compatibility\u2014smart move.<\/p>\n Loads the PDFs into LangChain Document objects. Each page becomes one Document.<\/p>\n This is the heart of your long-context handling. This tool:<\/p>\n The recursive splitter tries splitting on paragraphs \u2192 sentences \u2192 characters, preserving as much semantic structure as possible.<\/p>\n I am using the 2024 text-embedding-3-small model, which is:<\/p>\n Here, I am creating two vector stores:<\/p>\n Using cosine similarity for retrieval:<\/p>\n Only return the top 5 chunks with enough similarity. Keeps answers tight.<\/p>\n Here, I am creating context-specific prompts:<\/p>\n These prompts also guard against hallucination with:<\/p>\n \u201cIf the answer cannot be found in the context\u2026 respond with: \u2018I don\u2019t have enough information\u2026\u2019\u201d<\/em><\/p>\n Perfect for reliability.<\/p>\n This is where LangChain Expression Language (LCEL) shines.<\/p>\n It\u2019s elegant, reusable, and modular.<\/p>\n Each chain becomes a Tool in LangChain. Tools are like plug-and-play capabilities for the agent.<\/p>\n I am using the Zero-Shot ReAct agent, which interprets the query, decides which tool (ML or GenAI) to use, and routes the input accordingly.<\/p>\n Agent:<\/strong><\/p>\n Agent:<\/strong><\/p>\n GPT-4.1 proves to be exceptionally effective for working with large documents, thanks to its extended context window of up to 1 million tokens. This enhancement eliminates the long-standing limitations faced with previous models, where documents had to be heavily chunked into small segments, often losing semantic coherence.\u00a0<\/p>\n With the ability to handle large chunks, such as the 5000-token segments used here, GPT-4.1 can ingest and reason over dense, information-rich sections without missing contextual links across paragraphs or pages. This is especially valuable in scenarios involving complex documents like academic papers or industry whitepapers, where understanding often depends on multi-page continuity. The model handles these extended chunks accurately and delivers context-grounded responses without hallucinations, a capability further amplified by well-designed retrieval prompts.\u00a0<\/p>\n Moreover, in a RAG pipeline, the quality of responses is heavily tied to how much useful context the model can consume at once. GPT-4.1 removes the previous ceiling, making it possible to retrieve and reason over complete conceptual units rather than fragmented excerpts. As a result, you can ask deep, nuanced questions about long documents and receive precise, well-informed answers, making GPT-4.1 a game-changer for production-grade document analysis and retrieval-based applications.<\/p>\nOverview of GPT 4.1<\/h2>\n
\n
What\u2019s New in GPT 4.1?<\/h2>\n
1. 1M Token Context: Think Bigger Prompts<\/h3>\n
2. Coding Upgrades: Smarter, Multilingual, More Accurate<\/h3>\n
![\"SWE](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/1-2.webp\")
<\/figure>\n<\/div>\n\n
3. Better Instruction Following<\/h3>\n
![\"Hard](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/2-2.webp\")
<\/figure>\n<\/div>\n4. Speed & Cost: Half the Latency, Fraction of the Price<\/h3>\n
![\"intelligence](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/3-2.webp\")
<\/figure>\n<\/div>\n\n
\n
![\"cost](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/Screenshot-from-2025-04-17-13-31-38.webp\")
<\/figure>\n![\"Aider](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/4-2.webp\")
<\/figure>\n<\/div>\nBuilding Agentic RAG Using GPT 4.1 mini<\/h2>\n
\n
1. Setup and Installation<\/h3>\n
!pip install langchain==0.3.23\n!pip install -U langchain-openai\n!pip install langchain-community==0.3.11\n!pip install langchain-chroma==0.1.4\n!pip install pypdf<\/code><\/pre>\nInstall the necessary imports<\/h4>\n
from langchain_community.document_loaders import PyPDFLoader\nfrom langchain.text_splitter import RecursiveCharacterTextSplitter\nfrom langchain_chroma import Chroma\nfrom langchain_openai import ChatOpenAI, OpenAIEmbeddings\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langchain_core.runnables import RunnablePassthrough\nfrom langchain_core.output_parsers import StrOutputParser\nfrom langchain.agents import AgentType, Tool, initialize_agent<\/code><\/pre>\n2. OpenAI API Key<\/h3>\n
from getpass import getpass\nOPENAI_KEY = getpass('Enter Open AI API Key: ')\nimport os\nos.environ['OPENAI_API_KEY'] = OPENAI_KEY<\/code><\/pre>\n3. Load PDFs using PyPDFLoader<\/h3>\n
pdf_dir = \"\/content\/document_pdf\"\nmachinelearning_paper = os.path.join(pdf_dir, \"Machinelearningalgorithm.pdf\")\ngenai_paper = os.path.join(pdf_dir, \"the-economic-potential-of-generative-ai-\nthe-next-productivity-frontier.pdf\")\n\n# Load individual PDF documents\nprint(\"Loading ml pdf...\")\nml_loader = PyPDFLoader(machinelearning_paper)\nml_documents = ml_loader.load()\n\nprint(\"Loading genai pdf...\")\ngenai_loader = PyPDFLoader(genai_paper)\ngenai_documents = genai_loader.load()<\/code><\/pre>\n3. Chunk with RecursiveCharacterTextSplitter<\/h3>\n
# Split the documents\ntext_splitter = RecursiveCharacterTextSplitter(chunk_size=5000, chunk_overlap=300)\n\nml_splits = text_splitter.split_documents(ml_documents)\ngenai_splits = text_splitter.split_documents(genai_documents)\n\nprint(f\"Created {len(ml_splits)} splits for ml PDF\")\nprint(f\"Created {len(genai_splits)} splits for genai PDF\")<\/code><\/pre>\n\n
Ml_splits[:3]<\/h3>\n
![\"Output\"](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/10-1.webp\")
<\/figure>\n<\/div>\ngenai_splits[:5]<\/h3>\n
![\"Output\"](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/11-1.webp\")
<\/figure>\n<\/div>\n4. Embedding with OpenAIEmbeddings<\/h3>\n
# details here: https:\/\/openai.com\/blog\/new-embedding-models-and-api-updates\nopenai_embed_model = OpenAIEmbeddings(model=\"text-embedding-3-small\")<\/code><\/pre>\n\n
5. Store in Chroma Vector Stores<\/h3>\n
# Create separate vectorstores\nml_vectorstore = Chroma.from_documents(\n\u00a0\u00a0\u00a0documents=ml_splits,\n\u00a0\u00a0\u00a0embedding=openai_embed_model,\n\u00a0\u00a0\u00a0collection_metadata={\"hnsw:space\": \"cosine\"},\n\u00a0\u00a0\u00a0collection_name=\"ml-knowledge\"\n)\ngenai_vectorstore = Chroma.from_documents(\n\u00a0\u00a0\u00a0documents=genai_splits,\n\u00a0\u00a0\u00a0embedding=openai_embed_model,\n\u00a0\u00a0\u00a0collection_metadata={\"hnsw:space\": \"cosine\"},\n\u00a0\u00a0\u00a0collection_name=\"genai-knowledge\"\n)<\/code><\/pre>\n\n
ml_retriever = ml_vectorstore.as_retriever(search_type=\"similarity_score_threshold\",search_kwargs={\"k\": 5,\"score_threshold\": 0.3})\n\ngenai_retriever = genai_vectorstore.as_retriever(search_type=\"similarity_score_threshold\",search_kwargs={\"k\": 5,\"score_threshold\": 0.3})<\/code><\/pre>\nquery = \"what are ML algorithms?\"\ntop3_docs = ml_retriever.invoke(query)\ntop3_docs<\/code><\/pre>\n![\"Output\"](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/12-1.webp\")
<\/figure>\n<\/div>\n6. Retrieval and Prompt Creation<\/h3>\n
# Create the prompt templates\nml_prompt = ChatPromptTemplate.from_template(\n\n\u00a0\u00a0\u00a0\"\"\"\n\u00a0\u00a0\u00a0You are an expert in machine learning algorithms with deep technical knowledge of the field.\n\u00a0\u00a0\u00a0Answer the following question based solely on the provided context extracted from relevant machine learning research documents.\n\u00a0\u00a0\u00a0Context:\n\u00a0\u00a0\u00a0{context}\n\u00a0\u00a0\u00a0Question:\n\u00a0\u00a0\u00a0{question}\n\n\u00a0\u00a0\u00a0If the answer cannot be found in the context, please respond with: \"I don't have enough information to answer this question based on the provided context.\"\n\u00a0\u00a0\u00a0\"\"\"\n)\n\ngenai_prompt = ChatPromptTemplate.from_template(\n\n\u00a0\u00a0\u00a0\"\"\"\n\u00a0\u00a0\u00a0You are an expert in the economic impact and potential of generative AI technologies across industries and markets.\n\u00a0\u00a0\u00a0Answer the following question based only on the provided context related to the economic aspects of generative AI.\n\u00a0\u00a0\u00a0Context:\n\u00a0\u00a0\u00a0{context}\n\u00a0\u00a0\u00a0Question:\n\u00a0\u00a0\u00a0{question}\n\n\u00a0\u00a0\u00a0\u00a0If the answer cannot be found in the context, please state \"I don't have enough information to answer this question based on the provided context.\"\n\u00a0\u00a0\u00a0\"\"\"\n\n)<\/code><\/pre>\n\n
7. LCEL Chains<\/h3>\n
from langchain_openai import ChatOpenAI\nllm = ChatOpenAI(model_name=\"gpt-4.1-mini-2025-04-14\", temperature=0)<\/code><\/pre>\ndef format_docs(docs):\n\u00a0\u00a0\u00a0return \"\\n\\n\".join(doc.page_content for doc in docs)<\/code><\/pre>\n# Create the RAG chains using LCEL\nml_chain = (\n\u00a0\u00a0\u00a0{\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"context\": lambda question: format_docs(ml_retriever.get_relevant_documents(question)),\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"question\": RunnablePassthrough()\n\n\u00a0\u00a0\u00a0}\n\n\u00a0\u00a0\u00a0| ml_prompt\n\u00a0\u00a0\u00a0| llm\n\u00a0\u00a0\u00a0| StrOutputParser()\n\n)\n\ngenai_chain = (\n\n\u00a0\u00a0\u00a0{\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"context\": lambda question: format_docs(genai_retriever.get_relevant_documents(question)),\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\"question\": RunnablePassthrough()\n\n\u00a0\u00a0\u00a0}\n\n\u00a0\u00a0\u00a0| genai_prompt\n\u00a0\u00a0\u00a0| llm\n\u00a0\u00a0\u00a0| StrOutputParser()\n\n)<\/code><\/pre>\n\n
8. Define Tools for Agent<\/h3>\n
# Define the tools\n\ntools = [\n\u00a0\u00a0\u00a0Tool(\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0name=\"ML Knowledge QA System\",\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0func=ml_chain.invoke,\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0description=\"Useful for when you need to answer questions related to machine learning concepts, models, training techniques, evaluation metrics, algorithms and practical implementations. Covers supervised and unsupervised learning, model optimization, bias-variance tradeoff, feature engineering, and algorithm selection. Input should be a fully formed question.\"\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0),\n\n\u00a0\u00a0\u00a0Tool(\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0name=\"GenAI QA System\",\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0func=genai_chain.invoke,\n\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0description=\"Useful for when you need to answer questions about the economic impact, market potential, and cross-industry implications of generative AI technologies. Input should be a fully formed question. Responses are based strictly on the provided context related to the economics of generative AI.\"\n\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0)\n\n]<\/code><\/pre>\n9. Initialize Agent<\/h3>\n
# Initialize the agent\nagent = initialize_agent(\n\u00a0\u00a0\u00a0tools,\n\u00a0\u00a0\u00a0llm,\n\u00a0\u00a0\u00a0agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n\u00a0\u00a0\u00a0verbose=True\n)<\/code><\/pre>\n10. Query Time!<\/h3>\n
result = agent.invoke(\"How marketing and sale could be transformed using Generative AI?\")<\/code><\/pre>\n![\"Output\"](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/13.webp\")
<\/figure>\n<\/div>\n\n
result1 = agent.invoke(\"why Self-Attention is used?\")<\/code><\/pre>\n![\"Output\"](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/14.webp\")
<\/figure>\n<\/div>\n\n
result2 = agent.invoke(\"what are Tree-based algorithms?\")<\/code><\/pre>\n![\"Output\"](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/15.webp\")
<\/figure>\n<\/div>\n
![\"needle](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/04\/5-2.webp\")
<\/figure>\n<\/div>\n![\"OpenAI](\"https:\/\/images.ctfassets.net\/kftzwdyauwt9\/2oTJ2p3iGsEPnBrYeNhxbb\/9d14d937dc6004da8a49561af01b6781\/OpenAI-MRCR_accuracy_2needle_Lightmode.svg?w=3840&q=90\"\/)
![\"OpenAI](\"https:\/\/images.ctfassets.net\/kftzwdyauwt9\/5HRJ1DFBDAvOGhAcQ1x61m\/1b4aaca14d8c8caaaf7f3c70f285d089\/OpenAI-MRCR_accuracy_4needle_Lightmode.svg?w=3840&q=90\"\/)
![\"OpenAI](\"https:\/\/images.ctfassets.net\/kftzwdyauwt9\/nOUIReIO4isSJZA5c98FI\/12e3f5f67eba0e996808dcb5181b3ea8\/OpenAI-MRCR_accuracy_8needle_Lightmode.svg?w=3840&q=90\"\/)
![\"Pankaj](\"https:\/\/av-eks-lekhak.s3.amazonaws.com\/media\/lekhak-profile-images\/converted_image_Lb7Lh0T.webp\")
<\/p>\n