Langfuse launches AI Gateway - Langfuse

Get Started with Tracing
Step-by-step guide to ingesting your first trace using OpenAI, LangChain, or the SDKs.
Get Started with Tracing
This guide walks you through ingesting your first trace.
Langfuse joins ClickHouse
Our goal continues to be building the best AI engineering platform
One email per month with our latest ships and product announcements.
One open source entry point for all AI usage in your company. See every model call, who made it and what it cost, natively traced in Langfuse.
Today, we are launching Langfuse AI Gateway in public beta. We're excited to introduce our next product. Thank you to everyone who shared feedback that this is what you want us to build next to establish a unified way use models, track costs, and govern access to AI.
Over the last three years, something changed in how companies use AI. It is no longer one chatbot built by one team. It is coding agents on every engineer's laptop, AI assistants in every team, workflow tools adopted bottom-up, and dozens of custom AI apps in production.
It is becoming increasingly hard for companies to answer the questions: who in our organization uses which models, what for, how much does it cost, is it secure, and how can we optimize and govern it?
Managing AI usage within a company is becoming its own discipline:
Langfuse AI Gateway is a central entry point to govern all AI traffic that is easy to host, adopt, and enforce. It can log all requests natively in Langfuse, so you can analyze AI usage and make decisions based on it.
Langfuse is the leading open source project for AI observability and evals (35k+ GitHub stars, 110B+ observations/month, 90M+ package downloads/month). 21 of the Fortune 50 and leading AI-native businesses like Ramp, Hugging Face, and Canva rely on Langfuse to understand and improve their AI agents. Open source is important here as an AI gateway needs to be scalable, reliable, and self-hosting it air-gapped can solve many security concerns in regulated industries.
Adding a gateway was widely requested by our users. To get a complete picture of your company's AI usage, you need to see every model call. Our 100+ integrations already cover many of the AI agents you build, but much of a company's AI usage happens in tools nobody instruments, like coding agents, internal assistants, and scripts. A gateway sits centrally, where every call passes, so nothing is missed. And unlike a standalone gateway, which can only count tokens and cost, Langfuse turns each call into a full trace you can search, evaluate, and improve. This is why adding a native gateway is a natural next step for Langfuse.
Most gateways tell you what AI costs. Langfuse AI Gateway goes further, because every call through it becomes a full Langfuse trace (opt-in).
See your spend. Cost and usage for every call land in one AI spend dashboard, by model and provider. Get alerted before anyone runs into a surprise bill.
Know who spends it. Every key belongs to a person or service and can carry a team or cost center, so AI spend breaks down the way your company is organized.
Understand what it's spent on. Each call is a complete trace, not just a cost line. You can see what an application or a team actually did with a model, and use the same search, evaluations, and annotation you already use in Langfuse. Or simply ask Halo: "What does this team use Claude Code for?"
Control which models are used and keep them secure. Approve models centrally, so expensive or unvetted ones don't creep in unnoticed. You decide what gets recorded: usage and cost only, full prompts and responses, or nothing at all.
Incrementally adopt deeper tracing. Enforce logging of every model call by mandating the gateway, and let teams incrementally add more detail to their AI traces in Langfuse, such as the tool call executions of complex agents, via OpenTelemetry or one of our 50+ integrations. OpenTelemetry distributed tracing merges these traces within Langfuse.
Use Langfuse as memory for agents. All data in Langfuse is available via MCP and CLI, so you can give agents access to search all AI requests they made in the past. This is the foundation for self-improvement loops.
Export metrics and raw logs to your data lake. This enables any custom downstream process in finance and data teams.
This is only the beginning. We have a strong roadmap for AI Gateway to help govern AI usage:
If you are interested in being a design partner on one of these, reach out to us.
The gateway is in public beta, and you can try it out today. If you're an admin, set up the gateway for your organization, then create a gateway key under Organization settings → AI Gateway.
Point your app or coding agent at your region's gateway host (for example, https://gateway.cloud.langfuse.com) and use the gateway key instead of your provider key. Open the ingestion project in Tracing to see incoming traffic.
For a full walkthrough of setting up Claude Code, Codex, and more, see Send your first request.
Langfuse is part of ClickHouse since January and the Langfuse AI Gateway is one of many big launches at today. We're announcing the largest expansion of the ClickHouse platform since the launch of ClickHouse Cloud, introducing new capabilities across postgres, observability, security, and AI engineering. Read more.
Yes. The gateway is MIT-licensed and runs on Langfuse Cloud and self-hosted deployments.
No. Change the base URL and use a gateway key. Requests and responses stay in the provider's native format, whether you build custom agents or use a coding agent harness on your own machine.
Only if you opt in. By default, the gateway records metadata, usage, and cost, not content. On self-hosted Langfuse, the NONE ingestion mode records nothing. Self-hosting keeps all data on your infrastructure. Turn on full logging, and every request becomes a complete Langfuse trace you can debug, evaluate, and turn into datasets, all without touching your code.
OpenAI and Anthropic in the beta. OpenRouter, AWS, GCP, Az