KrakenD 3.0 Is Here: AI Router, Semantic Cache, and On-the-Fly Stream Manipulation | KrakenD API Gateway

News KrakenD 3.0 Is Here: AI Router, Semantic Cache, and On-the-Fly Stream Manipulation News KrakenD CE 3.0.4, EE 3.0.4 and EE 2.13.14 Update Released News What Is an AI Gateway? Architecture, Controls and Use Cases News KrakenD CE 3.0.3, EE 3.0.3 and EE 2.13.13 Update Released
Open Source API Gateway Free to use The high-performance open source API gateway, free to use and trusted by thousands of teams.
API Gateway Enterprise-Grade with SLAs Production-grade gateway with security, governance, and 24/7 support backed by SLAs.
AI Gateway Same Gateway, Now for AI LLM routing, prompt guardrails, token budgets, and a built-in MCP server, all from one gateway.
Enterprise Architecture, lastminute.com
Senior Software Engineer, Paylocity
Senior Engineer / Coop Atlantique
Today we release KrakenD 3.0, for both the Community Edition and the Enterprise Edition. It’s the first version of the next generation of KrakenD, which we are calling Series 3, and the biggest jump in the gateway’s capabilities since we launched. Most of the features below are part of the Enterprise Edition. If you run the open source version, jump to KrakenD Community Edition 3.0 or read the KrakenD CE v3.0 release notes. This post walks through what ships in 3.0 and why it matters, plus a preview of what’s coming next in the Series 3 line.
Streaming becomes a first-class part of the gateway core in 3.0. A streamed response integrates with KrakenD’s native features the same way a regular response does: your quota rules can read from it, and you can manipulate it as it flows through the gateway.
This is a key difference from the major AI gateways on the market. They treat a stream as a pipe: once it starts, the response either skips the response policies or has to be buffered in full before anything can change it. KrakenD can modify the stream on the fly, message by message, while it’s still reaching the client.
With LLM streaming, token-by-token responses flow through without buffering the whole completion first, which is critical for anything chat-shaped. You get that responsiveness without giving up the controls you apply to every other response.
The headline AI feature is the AI Router. Instead of statically declaring which LLM handles which type of request, 3.0 lets you route dynamically in two ways:
Send simple queries to a cheap, fast model and complex ones to a frontier model, automatically, based on the content of the request.
The router also has a failover: when no route matches, or the classifier cannot decide, the request goes to the provider you choose, so every request gets an answer. Combined with the fallback proxy strategy and quota fallbacks, you also decide what happens when a provider goes down or a user spends all their quota, like falling back to a free local model.
This is the piece that turns KrakenD from “a gateway that can reach LLMs” into “a gateway that manages your LLM spend, latency, and reliability for you.” See the AI Router documentation to set it up.
The Semantic Cache stores LLM responses and serves them again when a new prompt means the same thing as a previous one, even when the wording is different. “How do I reset my password?” and “I forgot my password, what do I do?” get the same cached answer, and the second one never reaches your model provider.
Every cache hit is a call you don’t pay for and a response that doesn’t wait for the model to generate it. It pays off most where people ask the same questions in different words: support assistants, internal knowledge bots, and FAQ-style agents. See the Semantic Cache documentation to set it up.
Prompt Guard catches prompt injection and unsafe inputs before they reach your model provider. You combine three kinds of guards: regular expressions for known patterns, CEL policies for rules based on the request, and an external classifier that scores each prompt. Every guard has a severity level, and you decide whether a request goes through or gets blocked when the external classifier is unavailable.
Because it runs in KrakenD, the same protection applies to every application and team sending traffic through the gateway, instead of each one implementing its own filters. See the Prompt Guard documentation to set it up.
Put together, these are the pieces teams have been building themselves in application code. 3.0 moves them into the gateway, where they apply to every service behind it at once.
KrakenD’s MCP Server gets the two features teams asked for most before exposing their APIs to agents:
MCP tools also gain dynamic routing, access to the incoming Host header, and support for binary responses.
KrakenD Community Edition 3.0 ships today too, as a leaner and cleaner gateway:
The KrakenD CE v3.0 release notes list every change, including the deprecated components this version removes.
3.0 is the first release in Series 3. Here’s what’s landing right behind it:
As a major version, 3.0 changes the configuration syntax version: every configuration file must now declare "version": 4 instead of the "version": 3 used across the 2.x series, or KrakenD refuses to start. It also removes components that were deprecated long ago. Most setups upgrade with a few configuration changes, not a re-architecture. The KrakenD CE v3.0 release notes and the changelog list every removal and its replacement, and the upgrade guide walks you through the changes.
Enterprise customers don’t have to upgrade on day one. KrakenD Enterprise 2.x enters maintenance mode with this release: it receives patches for security issues only, and no new features or other fixes, until it reaches its end of life on April 5, 2027. Plan your move to 3.0 before that date to keep receiving security updates.
The one change that needs planning is for Community Edition users with plugins, which move to Enterprise Edition in 3.0. We explained why in Dropping plugin support in KrakenD Open Source and Lura. If your setup depends on plugins, talk to us before you upgrade. We have a migration path, inclu