Rate Limits and Quotas for Internal MCP Servers

The pattern is well documented outside MCP. When nine agents shared one API quota, they opened 10 pull requests in 22 minutes, then hit 429 responses; every agent retried on the same schedule, triggering a second and third wave, and the shared hourly budget was exhausted in about 90 seconds 20. The author's conclusion is the right mental model: rate limiting across multiple agents is a coordination problem, not a retry problem 20.

Three distinct failures show up in that story, and each needs a different lever.

A fourth failure is specific to MCP and less discussed: the agent itself. A model in a tool-use loop can call the same tool with near-identical arguments repeatedly because it misreads an error or keeps retrying a validation failure. From the backend's point of view that looks exactly like load. Your limiter is also a loop breaker, provided the error you return tells the model to stop rather than try again.

The difference between a rate limit and a quota is worth stating plainly. A rate limit caps how fast. A quota caps how much over a longer window, such as requests per agent per day or bytes returned per tool per hour. You usually want both, because a patient loop can stay under a per-minute limit forever.

The key is the decision that outlives every library choice. A good default is three levels, checked in this order: agent identity, tool name, and any tenant or data-partition the tool touches.

The first level needs a trustworthy identity. Microsoft's guidance on MCP authentication separates shared authentication, where every user of the agent uses one identity, from individual authentication, where each user's context persists through OAuth identity passthrough 14. For budgets, shared identity means the budget is shared, which is usually wrong: one noisy team should not starve another. Issue a distinct credential per agent instance where you can, and fall back to per-user identity only for interactive use.

The second level is where most real damage is prevented, because expensive tools are not evenly distributed. A search_tickets call and a drop_partition call should not share a budget. Define a cost class per tool and weight the bucket accordingly.

The third level exists because your backend is shared with non-agent traffic. The limiter in the MCP server protects the MCP path; it does not protect the database from your batch jobs. Size the MCP budget from what is left after everything else, not from the backend's headline capacity.

A worked example makes the arithmetic concrete. Suppose a downstream CRM accepts about 20 writes per second sustained, and your batch jobs already use about 8. That leaves 12 writes per second for agents, or 720 per minute. With 30 engineers each running one agent, an equal split gives 24 writes per minute per agent. Set the bucket at 24 tokens per minute with a burst of 6, and a normal interactive session fits while a runaway loop is capped at 24 calls before it must wait. If that is still too generous, cut the burst first: the burst is what turns a loop into an outage.

The limiter goes at the top of the tool handler, before validation that touches the backend and before any downstream call. Token bucket is the right default because it tolerates natural bursts and still enforces a long-run rate. The skeleton below is deliberately framework-agnostic, so it drops into any Python MCP server.

import time, math, random, hashlib

class TokenBucket:
    def __init__(self, rate_per_min, burst):
        self.rate = rate_per_min / 60.0   # tokens per second
        self.burst = burst
        self.tokens = burst
        self.updated = time.monotonic()

    def take(self, cost=1.0):
        now = time.monotonic()
        self.tokens = min(self.burst, self.tokens + (now - self.updated) * self.rate)
        self.updated = now
        if self.tokens >= cost:
            self.tokens -= cost
            return 0.0
        needed = (cost - self.tokens) / self.rate
        return round(needed + random.uniform(0, 0.25), 2)  # jittered wait

BUCKETS = {}

def check_and_charge(agent_id, tool, cost_class):
    key = f"{agent_id}:{tool}"
    if key not in BUCKETS:
        rate, burst = BUDGETS[tool]      # e.g. (24, 6) per minute
        BUCKETS[key] = TokenBucket(rate, burst * cost_class)
    return BUCKETS[key].take(cost_class)

Three details carry most of the value. The jitter on the wait stops retry waves from re-forming. The cost class lets a search cost 1 and an export cost 20 from the same bucket. The dictionary is per process, which brings the first real architectural decision, covered next.

A note on MCP's own error surface: the protocol is built on JSON-RPC 2.0, where an error response carries a numeric code and a message alongside the request id 13. Use that shape, and put the retry hint where a client can read it without parsing prose.

In FastMCP, the handler is just the decorated function, so the check is a single call at the top:

from fastmcp import FastMCP

mcp = FastMCP("crm-tools")

@mcp.tool()
def search_tickets(query: str, limit: int = 10) -> list[dict]:
    """Search support tickets. Costs 1 unit."""
    wait = check_and_charge(CURRENT_AGENT_ID, "search_tickets", 1)
    if wait:
        raise ToolError(structured={"code": "RATE_LIMITED",
                                    "retry_after_seconds": wait})
    return backend.search(query, limit)

The CURRENT_AGENT_ID comes from your authentication layer, not from the tool arguments. Never let the model pass its own identity: that is an open invitation to self-issued exemptions.

A rate-limited call that returns "error" is a bug. A rate-limited call that returns a precise instruction is a feature, because the caller is a language model reading structured output.

Return, at minimum: the limit that was hit, the window, the number of seconds to wait, and a short statement that retrying immediately will not help. Compare the two responses below.

The last row is the one that saves your on-call engineer. The guidance row is the one that saves your quota, because a model given an explicit wait and a batching hint will often batch. This is also where constrained output helps: requiring structured output with a fixed schema rather than free text is a recommended mitigation against injected instructions in tool responses 6, and the same discipline makes your error responses machine-readable for clients.

There is a security reason not to leak detail here. Error text that names internal systems, hosts or query shapes becomes reconnaissance material. Return the scope and the wait, not the backend's stack trace.

A single MCP server process holding buckets in memory is fine until you run two replicas, at which point every limit doubles. Two options are worth comparing.

The middle row is the right default for a shared internal server. Fail open on store outage (admit the call) only if the downstream can survive it; otherwise fail closed with a retry hint. Write that choice down, because it is the kind of thing that gets flipped during an incident and never flipped back.

Then instrument the outcomes. You need four counters per key: admits, rejects, wait seconds handed out, and rejections by tool. The pattern to watch is not the reject count, it is the ratio of rejects to admits for a single agent. One agent at a high ratio is a loop. Every agent at a high ratio is an under-provisioned budget, and the fix is a capacity conversation, not a smaller bucket.

Observability practice for MCP servers is converging on the same signals as other services, with monitoring framed around common risks rather than only uptime 4. Rate-limit data is cheap to emit and expensive to reconstruct after the incident.

A word on where the traffic comes from. Most integrations now exist because MCP removes the N times M problem of custom connectors, reducing integrations from N×M to N+M 1. That is a good thing for adoption and a bad thing for blast radius: one shared server with a broad tool surface can be reached by every agent you run. Size budgets accordingly, and give the highest-damage tools the smallest buckets.

If your agents also call model endpoints directly, quota management does not stop at the MCP boundary. A gateway that presents text, image, video, audio and embedding models behind one OpenAI-compatible key, such as Quanta Inference, lets you hold one provider quota and see per-key usage, so an MCP server budget and a model budget are set from the same numbers. Build the tool surface with a reusable pattern library, or hand the work to an expert who ships deliverable code via Quanta Skills and Quanta Experts.

Work down this list in order. Each item closes a failure you can predict.

• Identity is stable and not supplied by the model: one credential per agent instance, mapped to an internal agent id.
• Every tool has a cost class, and expensive tools have their own budget line.
• The limiter runs before the downstream call, not after it.
• Rejections return a structured error with a numeric wait, and the wait is jittered.
• In-flight work is accounted for, not just completed calls, because MCP supports long-running calls with progress notifications 11.
• A daily quota exists per agent alongside the per-minute rate, so a patient loop cannot win.
• Every rejection is logged with agent id, tool, key, wait and request id.
• A load test runs one agent in a tight loop against the staging server and confirms the backend sees no more than the configured rate.

That last item is the one teams skip. It is also the only one that proves the limiter is on the path rather than in a code branch nobody executes.