One Gateway for Every Tool: How Uber Standardized Model Context Protocol (MCP) at Scale
How Uber unified internal microservices behind an enterprise MCP Gateway: automated tool discovery, protocol translation, and security sandboxing.
On this page
At a company with over 4,000 active microservices written across Go, Java, Python, and Node.js: communicating across a heterogeneous mix of gRPC, REST, and legacy TChannel protocols, enabling AI agents to access internal systems is an architectural minefield.
In early 2024, dozens of internal teams at Uber were building custom AI prototypes. Each team created its own proprietary function-calling wrappers, hardcoded internal API tokens into scripts, and manually pasted JSON schemas into system prompts. The result was massive tool sprawl, duplicated integrations, zero centralized security oversight, and no standardized way for an agent to discover which tools existed across the company.
Uber solved this by building an enterprise-grade Model Context Protocol (MCP) Gateway. The platform acts as a unified control plane and data plane proxy, automatically discovering internal APIs, translating legacy RPC protocols into standard MCP JSON-RPC, and enforcing strict enterprise authorization boundaries.
The Problem: Heterogeneous Services vs Standardized Agents
Anthropic’s open-standard Model Context Protocol (MCP) provides a clean client-to-server standard for AI tools:
// Standard MCP JSON-RPC 2.0 Request
{
"jsonrpc": "2.0",
"method": "tools/call",
"params": {
"name": "get_driver_telemetry",
"arguments": `{ "driver_id": "drv_8841a", "window_minutes": 15 }`
},
"id": 101
}
However, internal enterprise microservices do not speak MCP JSON-RPC. Uber’s core trip routing engines communicate via high-performance binary gRPC with Protobuf definitions; legacy billing services communicate via JSON-over-HTTP; older dispatch systems communicate over TChannel.
Expecting every microservice team to rewrite their production services into native MCP servers is impossible. The gateway must serve as the universal adapter.
The Dual-Plane Architecture: Registry vs Proxy Gateway
Uber’s platform decouples tool metadata discovery (Control Plane) from real-time request execution (Data Plane):
+-----------------------------------------------------------------------------+
| UBER MCP GATEWAY ARCHITECTURE |
+-----------------------------------------------------------------------------+
[Internal Microservices: gRPC / Protobuf / REST / TChannel]
|
| (1. Automated Service Scanning)
v
+---------------------------------------------------------------------------+
| CONTROL PLANE: MCP REGISTRY & AUTOCRAWLER |
| |
| [AutoCrawler Discovery Engine] ---> Scans Git IDLs & Service Schemas |
| [Tool Lifecycle Governance] ---> All discovered tools START DISABLED |
| [Owner Verification Portal] ---> Requires team lead approval to enable |
| [Semantic Tool Catalog] ---> Vector embedding index of all tools |
+---------------------------------------------------------------------------+
|
| (Synchronizes Approved Active Tools)
v
+---------------------------------------------------------------------------+
| DATA PLANE: MCP PROXY GATEWAY |
| |
| [Incoming MCP JSON-RPC 2.0] <=== Called by AI Agents & IDE Assistants |
| | |
| [Identity & Scopes Verification] ---> Validates User OAuth JWT |
| | |
| [Protocol Translation Engine] ---> Converts JSON-RPC into binary gRPC |
| | |
| [Credential Injection & mTLS] ---> Attaches internal service tokens |
+---------------------------------------------------------------------------+
|
v (Internal Private Network Forwarding)
[Production Microservice Mesh]
1. The Control Plane: AutoCrawler and Tool Governance
Rather than relying on engineers to manually register tools, an automated background service called AutoCrawler continuously scans Uber’s monorepos, extracting Protobuf IDL definitions and OpenAPI specifications:
- Default-Disabled Rule: Discovered tools start in a strict
DISABLEDstate. A tool cannot be accessed by any AI agent until the owning service team logs into the registry, reviews the exposed parameters, assigns an access policy, and explicitly clicksAPPROVE. - Schema Drift Detection: When an engineer merges a pull request altering an internal gRPC service parameter, AutoCrawler flags the schema change, generates a semantic diff, and alerts the team to update the tool documentation.
2. The Data Plane: Protocol Translation and Credential Injection
When an AI agent invokes an approved tool:
- The agent emits a standard MCP
tools/callpayload over a secure HTTP/SSE or WebSocket connection to the gateway. - The gateway validates the agent’s session token and verifies that the calling user possesses appropriate authorization scopes for the target service.
- The gateway’s Protocol Translator dynamically marshals the JSON arguments into strongly typed binary Protobuf structs.
- It injects internal mutual TLS (mTLS) certificates and service-to-service SPIFFE credentials, forwarding the gRPC call across the internal VPC network.
- Upon response, the gRPC response is converted back into standard MCP JSON-RPC and streamed to the agent.
Dynamic Tool Curation: Solving the 5,000-Tool Context Crisis
In a company with 4,000 microservices, the gateway index holds over 20,000 distinct API tools.
If an agent attempts to load all 20,000 tool schemas into its prompt, it consumes over 400,000 tokens of context before the user even types a greeting, crashing the model and triggering severe tool-selection confusion.
Uber implements Dynamic Context-Aware Tool Curation:
User Intent: "Investigate why trip #8912 was billed with an incorrect surge multiplier"
|
v
[Gateway Semantic Vector Matcher (Top-K Tool Retrieval)]
1. `get_trip_pricing_breakdown` (Score: 0.96)
2. `get_geohash_surge_multiplier` (Score: 0.93)
3. `get_driver_route_telemetry` (Score: 0.88)
|
v
[Agent Context Injected with ONLY 3 Tools Instead of 20,000!]
When an agent session initializes, it declares its broad task domain. The gateway executes a vector search over the tool catalog, injecting only the 3 to 6 most relevant tool definitions into the agent’s active session, slashing latency and eliminating tool hallucinations.
Architectural Takeaway
The Model Context Protocol (MCP) provides a clean standard for AI tool interaction, but deploying it in an enterprise requires industrial platform plumbing:
- Build a protocol translation proxy that bridges standard MCP JSON-RPC to internal gRPC and REST backends.
- Implement an automated discovery crawler that indexes service schemas but enforces a default-disabled governance gate.
- Inject internal enterprise credentials at the gateway proxy layer so that raw secrets are never exposed to model prompts.
- Use semantic vector search to dynamically curate small, relevant subsets of tools per request, preventing context window exhaustion.

/* Comments */
Comments are offline right now — we reconnect automatically, nothing is lost.