Your first AI agent talks to one model and calls two tools. Nobody worries about traffic. Then the team ships a second agent, a third tool server appears, someone wires in a CRM, and suddenly you cannot answer a basic question: who called what, with whose permission, and what came back? That is the moment people start searching for a gateway, and the moment the vocabulary gets muddy. An agent gateway and an MCP gateway sound like the same thing. They are not, and picking the wrong one leaves a real hole in your setup.
Here is the short version. An MCP gateway sits between agents and tools. An agent gateway sits between agents and other agents. Both are control points, both enforce identity and policy, and many teams end up running one of each. The rest of this article shows exactly where the line falls, which open source projects are worth testing, and how to keep the whole thing secure.
What Each Gateway Controls

Think of the old telephone switchboard. Callers did not dial each other directly. An operator connected them, checked who was allowed to reach whom, and kept a record. Gateways do the same job for AI systems, but each type watches a different kind of call.
The MCP Gateway
The Model Context Protocol (MCP) is the standard way an agent discovers and calls tools: search a database, open a ticket, read a file, send a message. An MCP gateway is a single front door for all the MCP servers behind it. Agents connect to the gateway, and the gateway decides which servers and which individual tools they can reach.
What it typically handles:
- Routing requests to the right MCP server
- Identity, so every tool call is tied to a user or an agent
- Tool level policy, such as "this agent may read tickets but never delete them"
- Aggregation, presenting many servers as one catalog
- Audit trails for every tool execution
Because it sees the actual payload of a tool call, an MCP gateway can inspect content in ways a plain API gateway cannot. It knows the difference between an agent asking to list invoices and an agent asking to export the whole customer table.
The Agent Gateway
Agent to agent traffic is a different animal. When one agent delegates a task to another, you get long running work, task state, status updates, and sometimes push notifications when a job finishes hours later. The A2A protocol exists for this: MCP frames the connection as agent to tool, while A2A frames it as agent to agent collaboration.
An agent gateway governs that second layer. It typically covers:
- Agent identity and permissions, meaning which agent may delegate to which
- Task lifecycle, including long running jobs and callbacks
- Memory and context sharing boundaries between agents
- Monitoring of multi agent workflows
- Coordination rules, such as limits on delegation depth
The practical difference is visibility. A tool gateway never sees the request one agent sent to another, and an agent gateway does not necessarily inspect every tool call. They watch different wires.
Where LLM Gateways Fit
A third layer often enters the conversation: the LLM gateway, sometimes called an AI gateway. It governs calls to models: prompts in, completions out, with routing across providers, cost tracking, caching, and logging. If your agents run on models such as Claude Sonnet 5, GPT 5.6 Sol or Gemini 3.1 Pro, the LLM gateway is what lets you swap one for another without rewriting the agent.
💡 Quick mental model: LLM gateway = calls to models. MCP gateway = calls to tools. Agent gateway = calls to other agents. Same control ideas, three different traffic types.
Side by Side Comparison

A parcel hub does not care what is inside every box, but it cares a great deal about where each box is going and who sent it. A table makes the three gateway types easier to compare.
| Aspect | MCP Gateway | Agent Gateway | LLM Gateway |
|---|
| Governs | Agent to tool calls | Agent to agent calls | Agent to model calls |
| Core protocol | MCP | A2A and similar | Provider APIs |
| Sees | Tool names, arguments, results | Tasks, delegations, status | Prompts and completions |
| Main risk handled | Over permissioned tools, data leaks | Rogue or confused delegation | Cost, provider outages, prompt abuse |
| Typical policy | Allow or deny per tool | Allow or deny per agent pair | Budgets, rate limits, fallbacks |
| State | Mostly stateless | Task state, long running jobs | Mostly stateless, optional cache |
| Audit value | Who ran which tool | Who asked whom to do what | Which model answered what |
Notice that the three are not rivals. They are layers. The mistake most teams make is buying or building one layer and assuming it covers the other two.
When You Need Each One

You do not need all three on day one. The right choice depends on how your agents actually behave.
Pick an MCP Gateway When
- You run more than two or three MCP servers and each team configures them differently
- Agents hold long lived credentials for tools, and you want to remove them from agent configs
- You need per tool permissions, not just "can connect or cannot connect"
- Compliance asks for a single audit log of every tool execution
- You want to expose internal systems to agents without handing out database access
For most teams, this is the first gateway worth deploying, because tools are where agents do real damage. A model that hallucinates a sentence is annoying. An agent that deletes a table is an incident.
Pick an Agent Gateway When
- Agents delegate work to other agents, especially across teams or vendors
- Some tasks run for minutes or hours and need status tracking
- You mix agents from different frameworks that need a shared way to talk
- You must prove which agent authorized which action in a chain of delegations
- External partners expose agents to you, or you expose yours to them
If you only run a single agent with many tools, an agent gateway is overkill. If you run a small fleet that hands work back and forth, it becomes the layer that keeps the chain accountable.
Running Both Together
In larger setups the two sit in sequence. A user request reaches an orchestrating agent. The agent gateway routes a subtask to a specialist agent. That specialist calls tools through the MCP gateway. Each hop is authenticated, logged, and checked against policy, and identity travels along the chain so the final tool call can still be traced back to the human who started it.
💡 Tip: Propagate the original user identity through every hop. If the MCP gateway only sees "the research agent," you lose the ability to enforce what that specific user is allowed to do.
Open Source Options Worth Testing

The ecosystem moves quickly, and most comparison posts come from vendors selling their own gateway, so treat any ranking with caution and test candidates against your own workload. These are the open source projects that come up most often.
agentgateway
Created by Solo.io and contributed to the Linux Foundation, agentgateway is written in Rust and released under the Apache 2.0 license. It is built for agent traffic, handling both MCP and agent to agent protocols, which makes it one of the few projects that straddles both layers. Rust gives it a performance edge for high volume proxying. Check its default authentication settings before exposing it (more on that below).
Docker MCP Gateway
Docker MCP Gateway takes a container first approach: each MCP server runs in its own isolated container, and the gateway manages them. It ships alongside Docker's MCP tooling, so it is a natural fit if your team already lives in Docker. The isolation story is its main selling point, since a compromised tool server stays boxed in. Licensing details have been reported inconsistently across roundups, so confirm the license on the repository before adopting it commercially.
IBM ContextForge
ContextForge, from IBM, is an open source registry and proxy that puts MCP servers, A2A agents, and REST or gRPC APIs behind one endpoint. It is released under Apache 2.0 and deploys through PyPI, Docker, or Kubernetes. That breadth is useful if you want a single control plane across tools and agents rather than two separate products.
MetaMCP and Others
MetaMCP works as an aggregator, orchestrator, and gateway for MCP servers, typically deployed with Docker and supporting SSE and streamable HTTP transports. Beyond those, you will see Microsoft's MCP Gateway (aimed at Kubernetes environments), Obot, Bifrost, and Lasso mentioned often. Some are fully open source, some are open core with paid tiers, so read the license and the feature split before committing.
| Project | Best for | Layer | Deployment |
|---|
| agentgateway | High performance, mixed protocols | MCP and agent traffic | Binary, container |
| Docker MCP Gateway | Isolated tool servers | MCP | Docker |
| IBM ContextForge | One endpoint for tools and agents | MCP and A2A | PyPI, Docker, Kubernetes |
| MetaMCP | Aggregating many MCP servers | MCP | Docker |
| Microsoft MCP Gateway | Kubernetes clusters | MCP | Kubernetes |
💡 Reality check: one third party security test reported that Docker MCP Gateway and ContextForge refused requests without a token, while agentgateway served tools to anyone who could reach the port in its default setup. Defaults change between releases, so run your own test instead of trusting any single result.
Security Risks and Controls

Security is the real reason gateways exist. Without one, every agent carries its own copy of every credential, and every tool server decides on its own who gets in. A gateway turns that sprawl into one place you can lock down and inspect.
Identity and Tokens

The first control is boring and essential: no anonymous access. A gateway that answers unauthenticated requests is an open door to every tool behind it.
- Require short lived tokens issued per user or per agent
- Use OAuth or an identity provider you already trust instead of shared secrets
- Keep upstream credentials inside the gateway, never in agent prompts or configs
- Apply least privilege: scope each agent to the smallest set of tools it needs
- Bind tokens to the original user, so delegation chains stay traceable
Treat agents as non human identities with their own lifecycle: created, scoped, rotated, and retired. Orphaned agent credentials are the same problem service accounts have always been, just faster.
Tool Poisoning and Injection
Agents trust the text they read, and that includes tool descriptions and tool results. A malicious or compromised MCP server can hide instructions inside a tool description, or a document the agent reads can contain text that redirects its behavior. This is the agent version of injection attacks.
Controls a gateway can apply:
- Allowlist approved servers and pin their versions, so a tool cannot silently change
- Inspect tool descriptions and results for suspicious instructions
- Block risky combinations, such as a tool that reads private data in the same session as a tool that sends email externally
- Require human approval for destructive or irreversible actions
- Rate limit unusual call patterns, like an agent suddenly enumerating every record
A safety classifier can help here. Models like Llama Guard 4 12B are built for content moderation and can sit in the inspection path to flag unsafe inputs and outputs before they reach an agent.
Audit and Logging

When something goes wrong, you need a timeline. A good gateway log answers five questions for every call: who, which agent, which tool or peer, with what arguments, and what happened. Store logs somewhere the agents themselves cannot edit, and redact sensitive fields before they are written.
Watch for these signals:
- Tools called outside an agent's normal pattern
- Sudden spikes in volume or data size
- Repeated denied requests, which suggest probing
- Delegation chains deeper than anything you designed
A Practical Rollout Plan

Rolling out gateways works best in small steps. Here is an order that avoids most pain.
- Inventory first. List every agent, every MCP server, and every credential in use. Most teams find tools nobody remembers approving.
- Put an MCP gateway in front of tools. Start in observe mode so you log traffic before you block anything.
- Move credentials into the gateway. Remove tool secrets from agent configs one by one.
- Write allow rules per agent. Begin with read only access and add write permissions deliberately.
- Add approval steps for destructive actions such as deletes, payments, and external sends.
- Introduce an agent gateway only when agents start delegating to each other or to outside partners.
- Test the defaults. Send unauthenticated requests, expired tokens, and malformed calls to confirm each gateway refuses them.
- Review logs weekly for the first month, then tune.
The choice of the model behind your agents matters less to this plan than the controls around it, but it still affects cost and behavior. Teams often route routine steps to a fast model like Gemini 3.5 Flash and reserve heavier reasoning for Deepseek V3.1 or Claude Fable 5. An LLM gateway makes that routing painless, and your MCP and agent gateways keep the tool and delegation side under control regardless of which model answers.
Common Mistakes to Avoid
- Treating one gateway as all three. An MCP gateway will not track agent delegation, and an LLM gateway will not authorize tool calls.
- Trusting defaults. Test authentication on day one.
- Skipping identity propagation. Without it, audit logs only say "the agent did it."
- Giving every agent every tool. Broad access is convenient until the first incident.
- Ignoring latency. Each hop adds time, so measure it, especially for chatty tool loops.

A toll plaza is a good final picture. Every vehicle slows down, gets checked, pays, and continues. Done well, the checkpoint adds seconds and prevents chaos. Done badly, it becomes the bottleneck everyone routes around. Design your gateways to be fast, boring, and impossible to bypass.
Try It With Picasso IA
Gateways keep agents in check, but the models behind them are where the interesting work happens. If you want to see how different models behave before wiring them into an agent stack, Picasso IA gives you one place to test them. Try Claude Sonnet 5 for coding tasks, compare it with GPT 5.6 Sol or Gemini 3.1 Pro, and see which one follows tool instructions most reliably.
Then go beyond text. Picasso IA also offers image, video, and audio generation, so you can create the visuals for your architecture docs, internal presentations, or product pages right alongside your language model experiments. Browse the full catalog at picassoia.com/en/all-models, pick a model, write a prompt, and generate your first image today. Experiment freely: the fastest way to learn what a model can do is to ask it for something specific and see what comes back.