The creation and deployment of applications that\u00a0utilize\u00a0Large Language Models (LLMs) comes with their own set of problems. LLMs have non-deterministic nature, can generate plausible but false information and tracing their actions in convoluted sequences can be very troublesome. In this guide,\u00a0we\u2019ll\u00a0see how\u00a0Langfuse\u00a0comes up as an essential instrument for solving these problems, by offering\u00a0a strong foundation\u00a0for whole observability, assessment, and prompt handling of LLM applications.<\/p>\n
What is\u00a0Langfuse?<\/h2>\n
Langfuse\u00a0is a groundbreaking observability and assessment platform that is open source and specifically created for LLM applications. It is the foundation for tracing, viewing, and debugging all the stages of an LLM interaction, starting from the\u00a0initial\u00a0prompt\u00a0and ending with the final response, whether it is a simple call or a complicated multi-turn conversation between agents.<\/p>\n
![\"Langfuse](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/11\/media_image-scaled.webp\")
<\/figure>\n<\/div>\nLangfuse\u00a0is not only a logging tool but also a means of systematically evaluating LLM performance, A\/B testing of prompts, and collecting user feedback which in turn helps to close the feedback loop essential for iterative improvement. The main point of its value is the transparency that it brings to the LLMs world, thus letting the developers to:\u00a0<\/p>\n
- \n
- Understand LLM\u00a0behaviour:<\/strong>\u00a0Find out the exact prompts that were sent, the responses that were received, and the intermediate steps in a multi-stage application.\u00a0<\/li>\n
- Find issues:<\/strong>\u00a0Locate\u00a0the source of errors, low performance, or unexpected outputs rapidly.\u00a0<\/li>\n
- Quality evaluation:\u00a0<\/strong>Effectiveness of LLM responses can be measured against the pre-defined metrics with both manual and automated measures.\u00a0<\/li>\n
- Refine and improve:<\/strong>\u00a0Data-driven insights can be used to perfect prompts, models, and application logic.<\/li>\n
- Handle prompts:\u00a0<\/strong>control the version of prompts and test them to get the best LLM.<\/li>\n<\/ul>\n
Key Features and Concepts<\/h2>\n
There are various key features that\u00a0Langfuse\u00a0offers like:\u00a0<\/p>\n
- \n
- Tracing and Monitoring\u00a0<\/li>\n<\/ol>\n
Langfuse\u00a0helps us capturing the detailed traces of every interaction that LLM has. The \u2018trace\u2019 is\u00a0basically the\u00a0representation of an end-to-end user request or application flow. Within a trace, logical units of work\u00a0is\u00a0denoted by \u201cspans\u201d and calls to an LLM refers to \u201cgenerations\u201d.<\/p>\n
- \n
- Evaluation\u00a0<\/li>\n<\/ol>\n
Langfuse\u00a0allows evaluation both manually and programmatically as well. Custom metrics can be defined by the developers which can then be used to run evaluations for different datasets and then be integrated as LLM-based evaluators.<\/p>\n
- \n
- Prompt Management\u00a0<\/li>\n<\/ol>\n
Langfuse\u00a0provides direct control over prompt management along with storage and versioning capabilities. It is possible to test various prompts through A\/B testing and at the same time\u00a0maintain\u00a0accuracy across diverse places, which paves the way for data-driven prompt optimization as well.\u00a0\u00a0<\/p>\n
- \n
- Feedback Collection\u00a0<\/li>\n<\/ol>\n
Langfuse\u00a0absorbs the user suggestions and incorporates them right into your traces. You will be able to link\u00a0particular remarks\u00a0or user ratings to the precise LLM interaction that resulted in an output, thus giving us the real-time feedback for troubleshooting and enhancing.\u00a0\u00a0<\/p>\n
\n![\"Feedback](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/11\/media_image3-scaled.webp\")
<\/figure>\n<\/div>\nWhy\u00a0Langfuse? The Problem It Solves<\/h2>\n
Traditional software observability tools have\u00a0very different\u00a0characteristics and do not satisfy the LLM-powered applications criteria in the following aspects:\u00a0<\/p>\n
- \n
- Non-determinism:<\/strong>\u00a0LLMs will not always produce the same outcome even for an identical input which makes debugging quite challenging.\u00a0Langfuse, in turn, records each interaction\u2019s input and output giving a clear picture of the operation at that moment.\u00a0<\/li>\n
- Prompt Sensitivity:<\/strong>\u00a0Any minor change in a prompt might alter LLM\u2019s answer completely.\u00a0Langfuse\u00a0is there to help keeping track of prompt versions along with their performance metrics.\u00a0<\/li>\n
- Complex Chains:<\/strong>\u00a0The majority of LLM applications are characterized by a combination of multiple LLM calls, different tools, and retrieving data (e.g., RAG<\/a> architectures). The only way to know the flow and to pinpoint the place where the bottleneck or the error is the tracing.\u00a0Langfuse\u00a0presents a visual timeline for these interactions.\u00a0<\/li>\n
- Subjective Quality:<\/strong>\u00a0The term \u201cgoodness\u201d for an LLM\u2019s answer is often synonymous with\u00a0personal opinion.\u00a0Langfuse\u00a0enables both\u00a0objective\u00a0(e.g., latency, token count) and subjective (human feedback, LLM-based evaluation) quality assessments.\u00a0<\/li>\n
- Cost Management:<\/strong>\u00a0Calling LLM APIs comes with a price. Understanding and\u00a0optimizing\u00a0your costs will be easier if you have\u00a0Langfuse\u00a0monitoring your token usage and call volume.\u00a0<\/li>\n
- Lack of Visibility:\u00a0<\/strong>The developer\u00a0is not able to\u00a0see how their LLM applications are performing on the market and therefore it is hard for them to make these applications gradually better because of the lack of observability.\u00a0<\/li>\n<\/ul>\n
Langfuse\u00a0does not only offer a systematic method for LLM<\/a>\u00a0interaction,\u00a0but it also transforms the development process into a data-driven, iterative, engineering discipline instead of trial and error.\u00a0<\/p>\n
Getting Started with\u00a0Langfuse<\/h2>\n
Before you can start using\u00a0Langfuse, you must first install the client library and set it up to\u00a0transmit\u00a0data to a\u00a0Langfuse\u00a0instance, which could either be a cloud-hosted or a self-hosted one.\u00a0<\/p>\n
Installation<\/h3>\n
Langfuse\u00a0has client libraries available for both Python and JavaScript\/TypeScript.\u00a0<\/p>\n
Python Client<\/em>\u00a0<\/p>\n
pip\u00a0install\u00a0langfuse\u00a0<\/code><\/pre>\nJavaScript\/TypeScript Client<\/em>\u00a0<\/p>\n
npm\u00a0install\u00a0langfuse\u00a0<\/code><\/pre>\nOr\u00a0<\/p>\n
yarn\u00a0add\u00a0langfuse\u00a0<\/code><\/pre>\nConfiguration\u00a0<\/h3>\n
After installation, remember to set up the client with your project keys and host. You can find these in your\u00a0Langfuse\u00a0project settings.\u00a0\u00a0\u00a0<\/p>\n
- \n
- public_key<\/strong>: This is for the frontend applications or for cases where only limited and non-sensitive data are getting sent.<\/li>\n
- secret_key<\/strong>:\u00a0This is for backend applications and scenarios where the full observability, including sensitive inputs\/outputs, is a requirement.\u00a0\u00a0\u00a0<\/li>\n
- host<\/strong>: This refers to the URL of your\u00a0Langfuse\u00a0instance (e.g.,\u00a0https:\/\/cloud.langfuse.com<\/a>).\u00a0\u00a0\u00a0<\/li>\n
- environment<\/strong>: This is an optional string that can be used to distinguish between different environments (e.g., production, staging, development).\u00a0\u00a0\u00a0<\/li>\n<\/ul>\n
For security and flexibility reasons, it is considered good practice to define these as environment variables.<\/p>\n
export\u00a0LANGFUSE_PUBLIC_KEY=\"pk-lf-...\"\u00a0\nexport\u00a0LANGFUSE_SECRET_KEY=\"sk-lf-...\"\u00a0\nexport\u00a0LANGFUSE_HOST=\"https:\/\/cloud.langfuse.com\"\u00a0\nexport\u00a0LANGFUSE_ENVIRONMENT=\"development\"<\/code><\/pre>\nThen, initialize the\u00a0Langfuse\u00a0client in your application:\u00a0<\/p>\n
Python Example<\/em>\u00a0<\/p>\n
from\u00a0langfuse\u00a0import\u00a0Langfuse\nimport\u00a0os\n\nlangfuse\u00a0=\u00a0Langfuse(public_key=os.environ.get(\"LANGFUSE_PUBLIC_KEY\"),\u00a0\u00a0\u00a0\u00a0secret_key=os.environ.get(\"LANGFUSE_SECRET_KEY\"),\u00a0\u00a0\u00a0\u00a0host=os.environ.get(\"LANGFUSE_HOST\"))<\/code><\/pre>\nJavaScript\/TypeScript Example<\/em>\u00a0<\/p>\n
import\u00a0{\u00a0Langfuse\u00a0}\u00a0from\u00a0\"langfuse\";\n\nconst\u00a0langfuse\u00a0= new\u00a0Langfuse({\u00a0\u00a0publicKey:\u00a0process.env.LANGFUSE_PUBLIC_KEY,\u00a0\u00a0secretKey:\u00a0process.env.LANGFUSE_SECRET_KEY,\u00a0 host:\u00a0process.env.LANGFUSE_HOST});<\/code><\/pre>\nSetting up Your First Trace<\/h2>\n
The fundamental unit of observability in\u00a0Langfuse\u00a0is the\u00a0trace. A trace typically\u00a0represents\u00a0a single user interaction or a complete request lifecycle. Within a trace, you log individual LLM calls (generation) and arbitrary computational steps (span).\u00a0<\/p>\n
Let\u2019s\u00a0illustrate with a simple LLM call using OpenAI\u2019s API<\/a>.\u00a0<\/p>\n
Python Example\u00a0<\/strong><\/p>\n
import os\nfrom openai import OpenAI\nfrom langfuse import Langfuse\nfrom langfuse.model import InitialGeneration\n\n# Initialize Langfuse\nlangfuse = Langfuse(\n public_key=os.environ.get(\"LANGFUSE_PUBLIC_KEY\"),\n secret_key=os.environ.get(\"LANGFUSE_SECRET_KEY\"),\n host=os.environ.get(\"LANGFUSE_HOST\"),\n)\n\n# Initialize OpenAI client\nclient = OpenAI(api_key=os.environ.get(\"OPENAI_API_KEY\"))\n\ndef simple_llm_call_with_trace(user_input: str):\n # Start a new trace\n trace = langfuse.trace(\n name=\"simple-query\",\n input=user_input,\n metadata={\"user_id\": \"user-123\", \"session_id\": \"sess-abc\"},\n )\n\n try:\n # Create a generation within the trace\n generation = trace.generation(\n name=\"openai-generation\",\n input=user_input,\n model=\"gpt-4o-mini\",\n model_parameters={\"temperature\": 0.7, \"max_tokens\": 100},\n metadata={\"prompt_type\": \"standard\"},\n )\n\n # Make the actual LLM call\n chat_completion = client.chat.completions.create(\n model=\"gpt-4o-mini\",\n messages=[{\"role\": \"user\", \"content\": user_input}],\n temperature=0.7,\n max_tokens=100,\n )\n\n response_content = chat_completion.choices[0].message.content\n\n # Update generation with the output and usage\n generation.update(\n output=response_content,\n completion_start_time=chat_completion.created,\n usage={\n \"prompt_tokens\": chat_completion.usage.prompt_tokens,\n \"completion_tokens\": chat_completion.usage.completion_tokens,\n \"total_tokens\": chat_completion.usage.total_tokens,\n },\n )\n\n print(f\"LLM Response: {response_content}\")\n return response_content\n\n except Exception as e:\n # Record errors in the trace\n trace.update(\n level=\"ERROR\",\n status_message=str(e)\n )\n print(f\"An error occurred: {e}\")\n raise\n\n finally:\n # Ensure all data is sent to Langfuse before exit\n langfuse.flush()\n\n\n# Example call\nsimple_llm_call_with_trace(\"What is the capital of France?\")<\/code><\/pre>\nEventually, your next step after executing this code would be to go to the\u00a0Langfuse\u00a0interface. There will be a new trace \u201csimple-query\u201d that consists of one generation \u201copenai-generation\u201d.\u00a0It\u2019s\u00a0possible for you to click it\u00a0in order to\u00a0view the input, output, model used, and other metadata.\u00a0<\/p>\n
Core Functionality in Detail<\/h2>\n
Learning to work with trace, span, and generation objects is the main requirement to take advantage of\u00a0Langfuse.\u00a0<\/p>\n
Tracing LLM Calls<\/h3>\n
- \n
langfuse.trace()<\/code>: This command starts a new trace. The top-level container for a whole operation.\u00a0\n- \n
- name<\/strong>: The trace\u2019s very descriptive name.\u00a0\u00a0<\/li>\n
- input<\/strong>: The first input of the whole procedure.\u00a0\u00a0<\/li>\n
- metadata<\/strong>: A dictionary of any key-value pairs for filtering and analysis (e.g.,\u00a0
user_id<\/code>,\u00a0session_id<\/code>,\u00a0AB_test_variant<\/code>).\u00a0\u00a0<\/li>\n- session_id<\/strong>: (Optional) An identifier shared by all traces that come from the same user session.\u00a0\u00a0<\/li>\n
- user_id<\/strong>: (Optional) An identifier shared by all interactions of a particular user.\u00a0\u00a0<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
- \n
trace.span()<\/code>: This is a logical step or minor operation within a trace that is not a direct input-output interaction with the LLM. Tool calls, database lookups, or complex calculations can be traced in this way.\u00a0\n- \n
- name<\/strong>: Name of the span (e.g. \u201cretrieve-docs\u201d, \u201cparse-json\u201d).\u00a0\u00a0<\/li>\n
- input<\/strong>: The input relevant to this span.\u00a0\u00a0<\/li>\n
- output<\/strong>: The output created by this span.\u00a0\u00a0<\/li>\n
- metadata<\/strong>: The span metadata is formatted as\u00a0additional.\u00a0\u00a0<\/li>\n
- level<\/strong>: The severity level (INFO, WARNING, ERROR, DEBUG).\u00a0\u00a0<\/li>\n
- status_message<\/strong>: A message that is linked to the status (e.g. error details).\u00a0\u00a0<\/li>\n
- parent_observation_id<\/strong>: Connects this span to a parent span or trace for nested structures.\u00a0<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
- \n
trace.generation()<\/code>: Signifies a particular LLM invocation.\u00a0\n- \n
- name<\/strong>: The name of the generation (for instance, \u201cinitial-response\u201d, \u201crefinement-step\u201d).\u00a0\u00a0<\/li>\n
- input<\/strong>: The prompt or messages that were communicated to the LLM.\u00a0\u00a0<\/li>\n
- output<\/strong>: The reply received from the LLM.\u00a0\u00a0<\/li>\n
- model<\/strong>: The precise LLM model that was employed (for example, \u201cgpt-4o-mini<\/a><\/em>\u201c, \u201cclaude-3-opus<\/a><\/em>\u201c).\u00a0\u00a0<\/li>\n
- model_parameters<\/strong>: A dictionary of\u00a0particular model\u00a0parameters (like
temperature<\/code>,\u00a0max_tokens<\/code>,\u00a0top_p<\/code>).\u00a0\u00a0<\/li>\n- usage<\/strong>: A dictionary displaying the number of tokens\u00a0utilized\u00a0(
prompt_tokens<\/code>,\u00a0completion_tokens<\/code>,\u00a0total_tokens<\/code>).\u00a0\u00a0<\/li>\n- metadata<\/strong>: Additional metadata for the LLM invocation.\u00a0\u00a0<\/li>\n
- parent_observation_id<\/strong>: Links this generation to a parent span or trace.\u00a0\u00a0<\/li>\n
- prompt<\/strong>: (Optional) Can\u00a0identify\u00a0a particular prompt template that is under management in\u00a0Langfuse.\u00a0<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
Conclusion<\/h2>\n
Langfuse\u00a0makes the development and upkeep of LLM-powered applications a less strenuous undertaking by turning it into a structured and data-driven process. It does this by giving developers access to the interactions with the LLM like never before through extensive tracing, systematic evaluation, and powerful prompt management.\u00a0\u00a0<\/p>\n
Moreover, it encourages the developers to debug their work with certainty, speed up the iteration process, and keep on improving their AI products in terms of quality and performance. Hence,\u00a0Langfuse\u00a0provides the necessary instruments to make sure that LLM applications are trustworthy, cost-effective, and\u00a0really powerful, no matter if you are developing a basic chatbot or a sophisticated autonomous agent.\u00a0<\/p>\n
Frequently Asked Questions<\/h2>\n
\nQ1. What problem does Langfuse solve for LLM applications?<\/strong> <\/p>\nA. It gives you full visibility into every LLM interaction, so you can track prompts, outputs, errors, and token usage without guessing what went wrong.<\/p>\n<\/p><\/div>\n
Q2. How does Langfuse help with prompt management?<\/strong> <\/p>\nA. It stores versions, tracks performance, and lets you run A\/B tests so you can see which prompts actually improve your model\u2019s responses.<\/p>\n<\/p><\/div>\n
Q3. Can Langfuse evaluate the quality of LLM outputs?<\/strong> <\/p>\nA. Yes. You can run manual or automated evaluations, define custom metrics, and even use LLM-based scoring to measure relevance, accuracy, or tone.<\/p>\n<\/p><\/div>\n<\/p><\/div>\n
\n\n\n
\n![\"Riya](\"https:\/\/av-eks-lekhak.s3.amazonaws.com\/media\/lekhak-profile-images\/converted_image_6IQzGzn.webp\")
<\/p>\n<\/a>\n <\/div>\n<\/p><\/div>\n
Data Science Trainee at Analytics Vidhya
I am currently working as a Data Science Trainee at Analytics Vidhya, where I focus on building data-driven solutions and applying AI\/ML techniques to solve real-world business problems. My work allows me to explore advanced analytics, machine learning, and AI applications that empower organizations to make smarter, evidence-based decisions.
With a strong foundation in computer science, software development, and data analytics, I am passionate about leveraging AI to create impactful, scalable solutions that bridge the gap between technology and business.
\ud83d\udce9 You can also reach out to me at [email\u00a0protected]<\/span><\/a><\/p>\n<\/p><\/div>\n<\/p><\/div>\nLogin to continue reading and enjoy expert-curated content.<\/h4>\n
- input<\/strong>: The prompt or messages that were communicated to the LLM.\u00a0\u00a0<\/li>\n
- name<\/strong>: The name of the generation (for instance, \u201cinitial-response\u201d, \u201crefinement-step\u201d).\u00a0\u00a0<\/li>\n
- input<\/strong>: The input relevant to this span.\u00a0\u00a0<\/li>\n
- name<\/strong>: Name of the span (e.g. \u201cretrieve-docs\u201d, \u201cparse-json\u201d).\u00a0\u00a0<\/li>\n
- input<\/strong>: The first input of the whole procedure.\u00a0\u00a0<\/li>\n
- name<\/strong>: The trace\u2019s very descriptive name.\u00a0\u00a0<\/li>\n
- secret_key<\/strong>:\u00a0This is for backend applications and scenarios where the full observability, including sensitive inputs\/outputs, is a requirement.\u00a0\u00a0\u00a0<\/li>\n
- Prompt Sensitivity:<\/strong>\u00a0Any minor change in a prompt might alter LLM\u2019s answer completely.\u00a0Langfuse\u00a0is there to help keeping track of prompt versions along with their performance metrics.\u00a0<\/li>\n
- Non-determinism:<\/strong>\u00a0LLMs will not always produce the same outcome even for an identical input which makes debugging quite challenging.\u00a0Langfuse, in turn, records each interaction\u2019s input and output giving a clear picture of the operation at that moment.\u00a0<\/li>\n
- Feedback Collection\u00a0<\/li>\n<\/ol>\n
- Prompt Management\u00a0<\/li>\n<\/ol>\n
- Evaluation\u00a0<\/li>\n<\/ol>\n
- Find issues:<\/strong>\u00a0Locate\u00a0the source of errors, low performance, or unexpected outputs rapidly.\u00a0<\/li>\n