{"id":70582,"date":"2025-02-06T01:47:36","date_gmt":"2025-02-06T01:47:36","guid":{"rendered":"https:\/\/peraltafinancing.com\/analytics\/running-olmo-2-locally-with-gradio-and-langchain\/"},"modified":"2025-02-06T01:47:36","modified_gmt":"2025-02-06T01:47:36","slug":"running-olmo-2-locally-with-gradio-and-langchain","status":"publish","type":"post","link":"https:\/\/fivemor.com\/?p=70582","title":{"rendered":"Running OLMo-2 Locally with Gradio and LangChain"},"content":{"rendered":"<p> <br \/>\n<\/p>\n<div id=\"article-start\">\n<p><a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2017\/01\/ultimate-guide-to-understand-implement-natural-language-processing-codes-in-python\/\" target=\"_blank\" rel=\"noreferrer noopener\">Natural Language Processing<\/a> has grown quickly in recent years. While private models have been leading the way, open-source models have been catching up. OLMo 2 is a big step forward in the open-source world, offering power and accessibility similar to private models. This article provides a detailed discussion of OLMo 2, covering its training, performance, and how to use it locally.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-learning-objectives\">Learning Objectives<\/h3>\n<ul class=\"wp-block-list\">\n<li>Understand the significance of open-source<a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2023\/03\/an-introduction-to-large-language-models-llms\/\" target=\"_blank\" rel=\"noreferrer noopener\"> LLMs<\/a> and OLMo 2\u2019s role in AI research.<\/li>\n<li>Explore OLMo 2\u2019s architecture, training methodology, and performance benchmarks.<\/li>\n<li>Differentiate between open-weight, partially open, and fully open models.<\/li>\n<li>Learn how to run OLMo 2 locally using Gradio and <a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2024\/06\/langchain-guide\/\" target=\"_blank\" rel=\"noreferrer noopener\">LangChain<\/a>.<\/li>\n<li>Implement OLMo 2 in a chatbot application with <a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2021\/05\/introduction-to-python-programming-beginners-guide\/\" target=\"_blank\" rel=\"noreferrer noopener\">Python<\/a> code examples.<\/li>\n<\/ul>\n<p><em><strong>This article was published as a part of the\u00a0<\/strong><\/em><a href=\"https:\/\/www.analyticsvidhya.com\/datahack\/blogathon\" target=\"_blank\" rel=\"noreferrer noopener\"><em><strong>Data Science Blogathon.<\/strong><\/em><\/a><\/p>\n<h2 class=\"wp-block-heading\" id=\"h-understanding-the-need-for-open-source-llms\">Understanding the Need for Open-Source LLMs<\/h2>\n<p>The initial dominance of proprietary LLMs created concerns about accessibility, transparency, and control. Researchers and developers were limited in their ability to understand the inner workings of these models, thus hindering further innovation and possibly perpetuating biases. Open-source LLMs have addressed these concerns by providing a collaborative environment where researchers can scrutinize, modify, and improve upon existing models. An open approach is crucial for advancing the field and ensuring that the benefits of LLMs are widely available.<\/p>\n<p><a href=\"https:\/\/www.analyticsvidhya.com\/blog\/2025\/01\/olmo-2\/\" target=\"_blank\" rel=\"noreferrer noopener\">OLMo<\/a>, initiated by the Allen Institute for AI (AI2), has been at the forefront of this movement. With the release of OLMo 2, they have solidified their commitment to open science by providing not just the model weights, but also the training data, code, recipes, intermediate checkpoints, and instruction-tuned models. This comprehensive release enables researchers and developers to fully understand and reproduce the model\u2019s development process, paving the way for further innovation. Running OLMo 2 Locally with Gradio and LangChain<\/p>\n<figure class=\"wp-block-image size-full figure mt-2 mb-2 d-table mx-auto\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1348\" height=\"606\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/Allen-AI.webp\" alt=\"Allen AI\" class=\"wp-image-219702\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/Allen-AI.webp 1348w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/Allen-AI-300x135.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/Allen-AI-768x345.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/Allen-AI-150x67.webp 150w\" sizes=\"(max-width: 1348px) 100vw, 1348px\"\/><\/figure>\n<h2 class=\"wp-block-heading\" id=\"h-what-is-olmo-2\">What is OLMo 2?<\/h2>\n<p>OLMo 2 marks a significant upgrade from its forefather, the OLMo-0424. The novel family of parameter models 7B and 13B showcase comparable performance or sometimes better-than-similar fully open models while competing with an open-weight version such as Llama 3.1 over English academic benchmarks. This makes the achievement very remarkable given a reduced total amount of training FLOPs relative to some similar models.<\/p>\n<ul class=\"wp-block-list\">\n<li><b>OLMo-2 Shows Significant Improvement:<\/b> The OLMo-2 models (both 7B and 13B parameter versions) demonstrate a clear performance jump compared to the earlier OLMo models (OLMo-7B, OLMo-7B-0424, OLMOE-1B-7B-0924). This suggests substantial progress in the model\u2019s architecture, training data, or training methodology.<\/li>\n<li><b>Competitive with MAP-Neo-7B:<\/b> The OLMo-2 models, especially the 13B version, achieve scores comparable to MAP-Neo-7B, which was likely a stronger baseline among the fully open models listed.<\/li>\n<\/ul>\n<h2 class=\"wp-block-heading\" id=\"h-breaking-down-olmo-2-s-training-process\">Breaking Down OLMo 2\u2019s Training Process<\/h2>\n<p>OLMo 2\u2019s architecture builds upon the foundation of the original OLMo, incorporating several key changes to enhance training stability and performance.\u00a0<\/p>\n<p><b>The pretraining process for OLMo 2 is divided into two stages:<\/b><\/p>\n<ul class=\"wp-block-list\">\n<li><b>Stage 1:<\/b> Foundation Training: This stage utilizes the OLMo-Mix-1124 dataset, a massive collection of approximately 3.9 trillion tokens sourced from various open datasets. This stage focuses on building a strong foundation for the model\u2019s language understanding capabilities.<\/li>\n<li><b>Stage 2:<\/b> Refinement and Specialization: This stage employs the Dolmino-Mix-1124 dataset, a curated mixture of high-quality web data and domain-specific data, including academic content, Q&amp;A forums, instruction data, and math workbooks. This stage refines the model\u2019s knowledge and skills in specific areas. The use of \u201cmodel souping\u201d to combine multiple trained models further enhances the final checkpoint.<\/li>\n<\/ul>\n<p>As OLMO-2 is Fully Open Model, Let\u2019s see what is the difference between Open Weight Models, Partially Open Models and Fully Open Models:<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-open-weight-models\">Open Weight Models<\/h3>\n<p><b>Llama-2-13B, Mistral-7B-v0.3, Llama-3.1-8B, Mistral-Nemo-12B, Qwen-2.5-7B, Gemma-2-9B, Qwen-2.5-14B:<\/b> These models share a key trait: their weights are publicly available. This allows developers to use them for various NLP tasks. However, critical details about their training process, such as the exact dataset composition, training code, and hyperparameters, are not fully disclosed. This makes them \u201copen weight,\u201d but not fully transparent.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-partially-open-models\">Partially Open Models<\/h3>\n<p><b>StableLM-2-128, Zamba-2-7B:<\/b> These models fall into a gray area. They offer some additional information beyond just the weights, but not the full picture. StableLM-2-128, for example, lists training FLOPS, suggesting more transparency than purely open-weight models. However, the absence of complete training data and code places it in the \u201cpartially open\u201d category.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-fully-open-models\">Fully Open Models<\/h3>\n<p><b>Amber-7B, OLMo-7B, MAP-Neo-7B, OLMo-0424-7B, DCLM-7B, OLMo-2-1124-7B, OLMo-2-1124-13B:<\/b> These models stand out due to their comprehensive openness. AI2 (Allen Institute for AI), the organization behind the OLMo series, has released everything necessary for full transparency and reproducibility: weights, training data (or detailed descriptions of it), training code, the full training \u201crecipe\u201d (including hyperparameters), intermediate checkpoints, and instruction-tuned versions. This allows researchers to deeply analyze these models, understand their strengths and weaknesses, and build upon them.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-key-differences\">Key Differences<\/h3>\n<div class=\"table-responsive mb-3\">\n<table class=\"table table-bordered border-black table-striped\">\n<thead\/>\n<tbody>\n<tr>\n<td><b>Feature<\/b><\/td>\n<td><b>Open Weight Models<\/b><\/td>\n<td><b>Partially Open Models<\/b><\/td>\n<td><b>Fully Open Models<\/b><\/td>\n<\/tr>\n<tr>\n<td><b>Weights\u00a0<\/b><\/td>\n<td>Released\u00a0<\/td>\n<td>Released\u00a0<\/td>\n<td>Released\u00a0<\/td>\n<\/tr>\n<tr>\n<td><b>Training Data\u00a0<\/b><\/td>\n<td>Typically Not\u00a0<\/td>\n<td>Partially Available<\/td>\n<td>\u00a0Fully Available<\/td>\n<\/tr>\n<tr>\n<td><b>Training Code<\/b><\/td>\n<td>Typically Not\u00a0<\/td>\n<td>Partially Available<\/td>\n<td>\u00a0Fully Available<\/td>\n<\/tr>\n<tr>\n<td><b>Training Recipe<\/b><\/td>\n<td>Typically Not\u00a0<\/td>\n<td>Partially Available<\/td>\n<td>\u00a0Fully Available<\/td>\n<\/tr>\n<tr>\n<td><b>Reproducibility<\/b><\/td>\n<td>Limited<\/td>\n<td>More than Open Weight, Less than Fully Open<\/td>\n<td>Full<\/td>\n<\/tr>\n<tr>\n<td><b>Transparency\u00a0<\/b><\/td>\n<td>Low\u00a0<\/td>\n<td>Medium\u00a0<\/td>\n<td>High<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<h2 class=\"wp-block-heading\" id=\"h-explore-nbsp-olmo-2\">Explore\u00a0OLMo 2<\/h2>\n<p>OLMo 2 is an advanced open-source language model designed for efficient and powerful AI-driven conversations. It integrates seamlessly with frameworks like LangChain, enabling developers to build intelligent chatbots and AI applications. Explore its capabilities, architecture, and how it enhances natural language understanding in various use cases.<\/p>\n<ul class=\"wp-block-list\">\n<li><b>Get the Model and Data: <\/b>Download <a href=\"https:\/\/huggingface.co\/collections\/allenai\/olmo-2-674117b93ab84e98afc72edc\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">Here<\/a><\/li>\n<li><b>Training Code:\u00a0<\/b><a href=\"https:\/\/github.com\/allenai\/OLMo\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">View<\/a><\/li>\n<li><b>Evaluation:<\/b> <a href=\"https:\/\/github.com\/allenai\/OLMo-Eval\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">View <\/a><\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\" id=\"h-let-s-run-it-locally\">Let\u2019s Run It Locally<\/h3>\n<p><b>Download Ollama <a href=\"https:\/\/ollama.com\/\" target=\"_blank\" rel=\"nofollow noopener\">here<\/a><\/b>.<\/p>\n<p>To Download Olmo-2 open Cmd and Type<\/p>\n<pre class=\"wp-block-code\"><code>ollama run olmo2:7b<\/code><\/pre>\n<p>This will download Olmo2 in your system<\/p>\n<p><b>Install Libraries\u00a0<\/b><\/p>\n<pre class=\"wp-block-code\"><code>pip install langchain-ollama\npip install gradio<\/code><\/pre>\n<h2 class=\"wp-block-heading\" id=\"h-building-a-chatbot-with-olmo-2\">Building a Chatbot with OLMo 2<\/h2>\n<p>Leverage the power of OLMo 2 to build an intelligent chatbot with open-weight LLM capabilities. Learn how to integrate it with Python, Gradio, and LangChain for seamless interactions.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-step1-importing-required-libraries\">Step1: Importing Required Libraries<\/h3>\n<p>Load essential libraries, including Gradio for UI, LangChain for prompt handling, and OllamaLLM for leveraging the OLMo 2 model in chatbot responses.<\/p>\n<pre class=\"wp-block-code\"><code>import gradio as gr\nfrom langchain_core.prompts import ChatPromptTemplate\nfrom langchain_ollama.llms import OllamaLLM<\/code><\/pre>\n<h3 class=\"wp-block-heading\" id=\"h-step2-defining-the-response-generation-function\">Step2: Defining the Response Generation Function<\/h3>\n<p>Create a function that takes chat history and user input, formats the prompt, invokes the OLMo 2 model, and updates the conversation history with AI-generated responses.<\/p>\n<pre class=\"wp-block-code\"><code>def generate_response(history, question):\n    template = \"\"\"Question: {question}\n\n    Answer: Let's think step by step.\"\"\"\n    prompt = ChatPromptTemplate.from_template(template)\n    model = OllamaLLM(model=\"olmo2\")\n    chain = prompt | model\n    answer = chain.invoke({\"question\": question})\n    history.append({\"role\": \"user\", \"content\": question})\n    history.append({\"role\": \"assistant\", \"content\": answer})\n    return history<\/code><\/pre>\n<p>The generate_response function takes a chat history and a user question as input. It defines a prompt template where the question is inserted dynamically, instructing the AI to think step by step. The function then creates a ChatPromptTemplate and initializes the OllamaLLM model (olmo2). Using LangChain\u2019s pipeline (prompt | model), it generates a response by invoking the model with the provided question. The conversation history is updated, appending the user\u2019s question and AI\u2019s answer. It returns the updated history for further interactions.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-step3-creating-the-gradio-interface\">Step3: Creating the Gradio Interface<\/h3>\n<p>Use Gradio\u2019s <code>Blocks<\/code>, <code>Chatbot<\/code>, and <code>Textbox<\/code> components to design an interactive chat interface, allowing users to input questions and receive responses dynamically.<\/p>\n<pre class=\"wp-block-code\"><code>with gr.Blocks() as iface:\n    chatbot = gr.Chatbot(type=\"messages\")\n    with gr.Row():\n        with gr.Column():\n            txt = gr.Textbox(show_label=False, placeholder=\"Type your question here...\")\n    txt.submit(generate_response, [chatbot, txt], chatbot)<\/code><\/pre>\n<ul class=\"wp-block-list\">\n<li>Uses gr.Chatbot() for displaying conversations.<\/li>\n<li>Uses gr.Textbox() for user input.<\/li>\n<\/ul>\n<h3 class=\"wp-block-heading\" id=\"h-step4-launching-the-application\">Step4: Launching the Application<\/h3>\n<p>Run the Gradio app using <code>iface.launch()<\/code>, deploying the chatbot as a web-based interface for real-time interactions.<\/p>\n<pre class=\"wp-block-code\"><code>iface.launch()<\/code><\/pre>\n<p>This starts the Gradio interface and runs the chatbot as a web app.<\/p>\n<p><b>Get Code from GitHub <a href=\"https:\/\/github.com\/Gouravlohar\/OLMo-2\" target=\"_blank\" rel=\"nofollow noopener\">Here<\/a><\/b>.<\/p>\n<h4 class=\"wp-block-heading\" id=\"h-output\">Output<\/h4>\n<figure class=\"wp-block-image size-full is-resized figure mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1224\" height=\"485\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Ncc90WS.webp\" alt=\"output\" class=\"wp-image-219685\" style=\"width:809px;height:auto\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Ncc90WS.webp 1224w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Ncc90WS-300x119.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Ncc90WS-768x304.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Ncc90WS-150x59.webp 150w\" sizes=\"auto, (max-width: 1224px) 100vw, 1224px\"\/><\/figure>\n<h4 class=\"wp-block-heading\" id=\"h-prompt\">Prompt<\/h4>\n<p>Write a Python function that returns True if a given number is a power of 2 without using loops or recursion.<\/p>\n<p><b>Response<\/b><\/p>\n<figure class=\"wp-block-image size-full figure mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1137\" height=\"200\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Wr8FlgR.webp\" alt=\"response: Running OLMo-2 Locally with Gradio and LangChain\" class=\"wp-image-219686\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Wr8FlgR.webp 1137w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Wr8FlgR-300x53.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Wr8FlgR-768x135.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_Wr8FlgR-150x26.webp 150w\" sizes=\"auto, (max-width: 1137px) 100vw, 1137px\"\/><\/figure>\n<figure class=\"wp-block-image size-full figure mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1125\" height=\"213\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_8EcDVDP.webp\" alt=\"output: Running OLMo-2 Locally with Gradio and LangChain\" class=\"wp-image-219689\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_8EcDVDP.webp 1125w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_8EcDVDP-300x57.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_8EcDVDP-768x145.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_8EcDVDP-150x28.webp 150w\" sizes=\"auto, (max-width: 1125px) 100vw, 1125px\"\/><\/figure>\n<figure class=\"wp-block-image size-full figure mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1130\" height=\"232\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_rUC6sxv.webp\" alt=\"output: Running OLMo-2 Locally with Gradio and LangChain\" class=\"wp-image-219690\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_rUC6sxv.webp 1130w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_rUC6sxv-300x62.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_rUC6sxv-768x158.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/image_rUC6sxv-150x31.webp 150w\" sizes=\"auto, (max-width: 1130px) 100vw, 1130px\"\/><\/figure>\n<h2 class=\"wp-block-heading\" id=\"h-conclusion\">Conclusion<\/h2>\n<p>Therefore, OLMo-2 stands out as one of the largest contributions to the open-source LLM ecosystem. It is one of the most powerful performer in the arena of full transparency, with focus on training efficiency. It reflects the growing significance of open collaboration in the world of AI and will pave the way for future progress in accessible and transparent language models.<\/p>\n<p>While OLMo-2-138 is a very strong model, it\u2019s not distinctly dominating on all tasks. Some partially open models and Qwen-2.5-14B, for instance, obtain higher scores on some benchmarks (for example, Qwen-2.5-14B significantly outperforms on ARC\/C and WinoG). Besides, OLMo-2 lags significantly behind the very best models at particular challenging tasks like GSM8k (grade school math) and probably AGIEval.<\/p>\n<p>Unlike many other LLMs, OLMo-2 is fully open, providing not only the model weights but also the training data, code, recipes, and intermediate checkpoints. This level of transparency is crucial for research, reproducibility, and community-driven development. It allows researchers to thoroughly understand the model\u2019s strengths, weaknesses, and potential biases.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-key-takeaway\">Key Takeaway<\/h3>\n<ul class=\"wp-block-list\">\n<li>The OLMo-2 models, especially the 13B parameter version, are showing great performance results on a host of benchmarks, beating other open-weight and even partially open architectures. It appears that full openness is indeed one of the ways to make powerful LLMs.<\/li>\n<li>The Fully Open models (particularly OLMo) tend to perform well. This supports the argument that having access to the full training process (data, code, etc.) facilitates the development of more effective models.<\/li>\n<li>The chatbot maintains conversation history, ensuring responses consider previous interactions.<\/li>\n<li>Gradio\u2019s event-based UI (txt.submit) updates in real-time, making the chatbot responsive and user-friendly.<\/li>\n<li>OllamaLLM integrates AI models into the pipeline, enabling seamless question-answering functionality.<\/li>\n<\/ul>\n<h2 class=\"wp-block-heading\" id=\"h-frequently-asked-questions\">Frequently Asked Questions<\/h2>\n<div class=\"schema-faq wp-block-yoast-faq-block\">\n<div class=\"schema-faq-section\" id=\"faq-question-1738738464504\"><strong class=\"schema-faq-question\">Q1. <b>What are FLOPS, and why are they important?<\/b><\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. FLOPS stand for Floating Point Operations. They represent the amount of computation a model performs during training. Higher FLOPS generally mean more computational resources were used. They\u2019re an important, though not sole, indicator of potential model capability. However, architectural efficiency and training data quality also play huge roles.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1738738477876\"><strong class=\"schema-faq-question\">Q2. <b>What\u2019s the difference between \u201cOpen weights,\u201d \u201cPartially open,\u201d and \u201cFully open\u201d models?<\/b><\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. This refers to the level of access to the model\u2019s components. \u201cOpen weights\u201d only provides the trained parameters. \u201cPartially open\u201d provides some additional information (e.g., some training data or high-level training details). \u201cFully open\u201d provides everything: weights, training data, code, recipes, etc., enabling full transparency and reproducibility.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1738738492741\"><strong class=\"schema-faq-question\">Q3. <b>Why is Chat Prompt Template used?<\/b><\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. Chat Prompt Template allows dynamic insertion of user queries into a predefined prompt format, ensuring the AI responds in a structured and logical manner.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1738738508788\"><strong class=\"schema-faq-question\">Q4. <b>How does Gradio manage the chatbot UI?<\/b><\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. Gradio\u2019s gr.Chatbot component visually displays the conversation. The gr.Textbox allows users to input questions, and upon submission, the chatbot updates with new responses dynamically.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1738738521922\"><strong class=\"schema-faq-question\">Q5. <b>Can this chatbot support different AI models?<\/b><\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. Yes, by changing the <b>model=\u201dolmo2\u2033<\/b> line to another available model in <b>Ollama<\/b>, the chatbot can use different AI models for response generation.<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<p><strong>The media shown in this article is not owned by Analytics Vidhya and is used at the Author\u2019s discretion.<\/strong><\/p>\n<div class=\"border-top py-3 author-info my-4\">\n<div class=\"author-card d-flex align-items-center\">\n<div class=\"flex-shrink-0 overflow-hidden\">\n                                    <a href=\"https:\/\/www.analyticsvidhya.com\/blog\/author\/gourav3493022\/\" class=\"text-decoration-none active-avatar\"><br \/>\n                                                                       <img decoding=\"async\" src=\"https:\/\/av-eks-lekhak.s3.amazonaws.com\/media\/lekhak-profile-images\/converted_image_b56toW7.webp\" width=\"48\" height=\"48\" alt=\"Gourav Lohar\" loading=\"lazy\" class=\"rounded-circle\"\/><\/p>\n<p>                                <\/a>\n                                <\/div>\n<\/p><\/div>\n<p>Hi I&#8217;m Gourav, a Data Science Enthusiast with a medium foundation in statistical analysis, machine learning, and data visualization. My journey into the world of data began with a curiosity to unravel insights from datasets.<\/p>\n<\/p><\/div>\n<\/p><\/div>\n\n","protected":false},"excerpt":{"rendered":"<p>Natural Language Processing has grown quickly in recent years. While private models have been leading the way, open-source models have been catching up. OLMo 2 is a big step forward in the open-source world, offering power and accessibility similar to private models. This article provides a detailed discussion of OLMo 2, covering its training, performance, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":70583,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12033],"tags":[5815,37074,37075,31984,37073,325],"dealstore":[],"offerexpiration":[],"class_list":["post-70582","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analytics","tag-blogathon","tag-gradio","tag-langchain","tag-locally","tag-olmo2","tag-running"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Running OLMo-2 Locally with Gradio and LangChain - Som2ny Network<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/fivemor.com\/?p=70582\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Running OLMo-2 Locally with Gradio and LangChain - Som2ny Network\" \/>\n<meta property=\"og:description\" content=\"Natural Language Processing has grown quickly in recent years. While private models have been leading the way, open-source models have been catching up. OLMo 2 is a big step forward in the open-source world, offering power and accessibility similar to private models. This article provides a detailed discussion of OLMo 2, covering its training, performance, [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/fivemor.com\/?p=70582\" \/>\n<meta property=\"og:site_name\" content=\"Som2ny Network\" \/>\n<meta property=\"article:published_time\" content=\"2025-02-06T01:47:36+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp\" \/>\n\t<meta property=\"og:image:width\" content=\"872\" \/>\n\t<meta property=\"og:image:height\" content=\"473\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/webp\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/fivemor.com\/?p=70582#article\",\"isPartOf\":{\"@id\":\"https:\/\/fivemor.com\/?p=70582\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\/\/fivemor.com\/#\/schema\/person\/b85e3c3dc0e1daea076524dc8810c371\"},\"headline\":\"Running OLMo-2 Locally with Gradio and LangChain\",\"datePublished\":\"2025-02-06T01:47:36+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/fivemor.com\/?p=70582\"},\"wordCount\":1788,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/fivemor.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/fivemor.com\/?p=70582#primaryimage\"},\"thumbnailUrl\":\"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp\",\"keywords\":[\"Blogathon\",\"Gradio\",\"LangChain\",\"Locally\",\"OLMo2\",\"Running\"],\"articleSection\":[\"Analytics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/fivemor.com\/?p=70582#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/fivemor.com\/?p=70582\",\"url\":\"https:\/\/fivemor.com\/?p=70582\",\"name\":\"Running OLMo-2 Locally with Gradio and LangChain - Som2ny Network\",\"isPartOf\":{\"@id\":\"https:\/\/fivemor.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/fivemor.com\/?p=70582#primaryimage\"},\"image\":{\"@id\":\"https:\/\/fivemor.com\/?p=70582#primaryimage\"},\"thumbnailUrl\":\"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp\",\"datePublished\":\"2025-02-06T01:47:36+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/fivemor.com\/?p=70582#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/fivemor.com\/?p=70582\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/fivemor.com\/?p=70582#primaryimage\",\"url\":\"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp\",\"contentUrl\":\"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp\",\"width\":872,\"height\":473},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/fivemor.com\/?p=70582#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/fivemor.com\/?bp_activities=1\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Running OLMo-2 Locally with Gradio and LangChain\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/fivemor.com\/#website\",\"url\":\"https:\/\/fivemor.com\/\",\"name\":\"Som2ny Network\",\"description\":\"Daily Deals\",\"publisher\":{\"@id\":\"https:\/\/fivemor.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/fivemor.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/fivemor.com\/#organization\",\"name\":\"Som2ny Network\",\"url\":\"https:\/\/fivemor.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/fivemor.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/fivemor.com\/wp-content\/uploads\/2026\/07\/4a0953c4-logo-300x86-1.png\",\"contentUrl\":\"https:\/\/fivemor.com\/wp-content\/uploads\/2026\/07\/4a0953c4-logo-300x86-1.png\",\"width\":300,\"height\":86,\"caption\":\"Som2ny Network\"},\"image\":{\"@id\":\"https:\/\/fivemor.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/fivemor.com\/#\/schema\/person\/b85e3c3dc0e1daea076524dc8810c371\",\"name\":\"admin\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/fivemor.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/729ae85bf62b9917e93538db2f2688ca?s=96&r=g&default=https%3A%2F%2Ffivemor.com%2Fwp-content%2Fplugins%2Fbuddypress-first-letter-avatar%2Fimages%2Fdefault%2F96%2Flatin_a.png\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/729ae85bf62b9917e93538db2f2688ca?s=96&r=g&default=https%3A%2F%2Ffivemor.com%2Fwp-content%2Fplugins%2Fbuddypress-first-letter-avatar%2Fimages%2Fdefault%2F96%2Flatin_a.png\",\"caption\":\"admin\"},\"sameAs\":[\"https:\/\/fivemor.com\"],\"url\":\"https:\/\/fivemor.com\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Running OLMo-2 Locally with Gradio and LangChain - Som2ny Network","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/fivemor.com\/?p=70582","og_locale":"en_US","og_type":"article","og_title":"Running OLMo-2 Locally with Gradio and LangChain - Som2ny Network","og_description":"Natural Language Processing has grown quickly in recent years. While private models have been leading the way, open-source models have been catching up. OLMo 2 is a big step forward in the open-source world, offering power and accessibility similar to private models. This article provides a detailed discussion of OLMo 2, covering its training, performance, [&hellip;]","og_url":"https:\/\/fivemor.com\/?p=70582","og_site_name":"Som2ny Network","article_published_time":"2025-02-06T01:47:36+00:00","og_image":[{"width":872,"height":473,"url":"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp","type":"image\/webp"}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"Written by":"admin","Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/fivemor.com\/?p=70582#article","isPartOf":{"@id":"https:\/\/fivemor.com\/?p=70582"},"author":{"name":"admin","@id":"https:\/\/fivemor.com\/#\/schema\/person\/b85e3c3dc0e1daea076524dc8810c371"},"headline":"Running OLMo-2 Locally with Gradio and LangChain","datePublished":"2025-02-06T01:47:36+00:00","mainEntityOfPage":{"@id":"https:\/\/fivemor.com\/?p=70582"},"wordCount":1788,"commentCount":0,"publisher":{"@id":"https:\/\/fivemor.com\/#organization"},"image":{"@id":"https:\/\/fivemor.com\/?p=70582#primaryimage"},"thumbnailUrl":"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp","keywords":["Blogathon","Gradio","LangChain","Locally","OLMo2","Running"],"articleSection":["Analytics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/fivemor.com\/?p=70582#respond"]}]},{"@type":"WebPage","@id":"https:\/\/fivemor.com\/?p=70582","url":"https:\/\/fivemor.com\/?p=70582","name":"Running OLMo-2 Locally with Gradio and LangChain - Som2ny Network","isPartOf":{"@id":"https:\/\/fivemor.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/fivemor.com\/?p=70582#primaryimage"},"image":{"@id":"https:\/\/fivemor.com\/?p=70582#primaryimage"},"thumbnailUrl":"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp","datePublished":"2025-02-06T01:47:36+00:00","breadcrumb":{"@id":"https:\/\/fivemor.com\/?p=70582#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/fivemor.com\/?p=70582"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/fivemor.com\/?p=70582#primaryimage","url":"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp","contentUrl":"https:\/\/fivemor.com\/wp-content\/uploads\/2025\/02\/Running-OLMo-2-Locally-with-Gradio-and-LangChain-.webp.webp","width":872,"height":473},{"@type":"BreadcrumbList","@id":"https:\/\/fivemor.com\/?p=70582#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/fivemor.com\/?bp_activities=1"},{"@type":"ListItem","position":2,"name":"Running OLMo-2 Locally with Gradio and LangChain"}]},{"@type":"WebSite","@id":"https:\/\/fivemor.com\/#website","url":"https:\/\/fivemor.com\/","name":"Som2ny Network","description":"Daily Deals","publisher":{"@id":"https:\/\/fivemor.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/fivemor.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/fivemor.com\/#organization","name":"Som2ny Network","url":"https:\/\/fivemor.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/fivemor.com\/#\/schema\/logo\/image\/","url":"https:\/\/fivemor.com\/wp-content\/uploads\/2026\/07\/4a0953c4-logo-300x86-1.png","contentUrl":"https:\/\/fivemor.com\/wp-content\/uploads\/2026\/07\/4a0953c4-logo-300x86-1.png","width":300,"height":86,"caption":"Som2ny Network"},"image":{"@id":"https:\/\/fivemor.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/fivemor.com\/#\/schema\/person\/b85e3c3dc0e1daea076524dc8810c371","name":"admin","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/fivemor.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/729ae85bf62b9917e93538db2f2688ca?s=96&r=g&default=https%3A%2F%2Ffivemor.com%2Fwp-content%2Fplugins%2Fbuddypress-first-letter-avatar%2Fimages%2Fdefault%2F96%2Flatin_a.png","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/729ae85bf62b9917e93538db2f2688ca?s=96&r=g&default=https%3A%2F%2Ffivemor.com%2Fwp-content%2Fplugins%2Fbuddypress-first-letter-avatar%2Fimages%2Fdefault%2F96%2Flatin_a.png","caption":"admin"},"sameAs":["https:\/\/fivemor.com"],"url":"https:\/\/fivemor.com\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/posts\/70582","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=70582"}],"version-history":[{"count":0,"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/posts\/70582\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=\/wp\/v2\/media\/70583"}],"wp:attachment":[{"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=70582"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=70582"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=70582"},{"taxonomy":"dealstore","embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fdealstore&post=70582"},{"taxonomy":"offerexpiration","embeddable":true,"href":"https:\/\/fivemor.com\/index.php?rest_route=%2Fwp%2Fv2%2Fofferexpiration&post=70582"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}