Amazon Bedrock AgentCore Gateway にレート制限機能を追加

Amazon Bedrock AgentCore gateway is a fully managed, serverless AI gateway that provides a single, secure entry point for AI traffic. AgentCore gateway routes traffic to tools such as managed web search, managed knowledge bases, MCP servers, inference models (LLMs), agents (A2A, agents as tools, etc.), or HTTP endpoint. Today, we are announcing support for rate limiting on AgentCore gateway, giving you fine-grained control over how much traffic individual users can consume through your gateway.

Rate limiting in AgentCore gateway gives you per-user control over how users consume your tools, inference models, and agents. Define OAuth or IAM-based rules for requests per minute, concurrent connections, and token throughput, making sure downstream services remain available under heavy traffic spikes.

Centralized rate limiting for AI traffic with AgentCore gateway

AgentCore gateway provides three target types: MCP targets, inference targets, and HTTP passthrough targets. The following rate limiting metrics are supported on the targets.

• Request rate limits, measured in requests per second (RPS) and requests per minute (RPM), apply to all target types. Each limit defines a maximum count of requests permitted within the given time window, and the gateway measures every incoming request against it. A request counts as exactly one unit toward the configured limit, regardless of how long it takes to complete, a request that finishes in 50 milliseconds and one that streams for 90 seconds each consume exactly one unit from the per-second or per-minute limit.

• Token rate limits, measured in tokens per minute (TPM), apply to inference targets only. Token rate limiting accounts for both input tokens and output tokens. The full round-trip token cost of a request counts against the limit. AgentCore gateway uses a general-purpose tokenizer to estimate the incoming tokens for a request and deducts it from the rate-limit bucket upfront before the gateway dispatches the inference call. Once the inference call returns a response, which includes actual input and output token usage reported by the model provider, the gateway reconciles the limit by accounting for the true token consumption.

• Connection rate limits, measured in connections per second (CPS), apply to all target types. Unlike request rate limits, connection rate limiting tracks how long each request holds an open connection. For example, if a streaming inference call takes 100 seconds to complete, that request consumes one connection slot for the entire duration. CPS provides an additional mechanism for protecting targets against long-lived concurrent sessions particularly useful when you need to cap how many simultaneous connections a target sustains, rather than how many requests arrive in each window.

For this use case, assume three user groups: Basic, Advanced, and Beta. AgentCore Identity handles inbound authentication using JSON Web Tokens (JWT) with Microsoft Entra ID as the identity provider and also serves as the token vending service for outbound targets. Policy in Amazon Bedrock AgentCore enforces role-based access control (RBAC), scoping each group’s access to specific targets and models. The following diagram illustrates this configuration.

Figure 1: AgentCore gateway rate limiting architecture with user groups, identity, and policy enforcement

Basic users operate under more restrictive rate limits than Advanced users, while Beta users receive elevated limits on restricted models, enabling the organization to benchmark performance and suitability before rolling these models out to the broader organization. Before setting up rate limits for each user-group, review the rate limit structure.

Rate limit structure

A rate limit configuration consists of two parts: dimension keys and entries. Dimension keys define how the gateway groups incoming traffic into rate buckets. Entries define the allowed throughput for each bucket.

In this post, we use the AWS Command Line Interface (AWS CLI) to create the rate limit configuration. The following example demonstrates the relationship between dimension keys and entries. This rate limit uses targetName as the dimension key and defines two entries: a specific entry for the Booking target (MCP server), a high-traffic target, at 100 requests per second, and a wildcard entry that applies 10 requests per second individually to each remaining target, meaning every other target receives its own 10 RPS bucket.

Figure 2: Rate limit structure with dimension keys and entries

Dimension keys define how the gateway groups traffic into rate buckets. When a request arrives, the gateway resolves each dimension key to its value from the request context and uses the resulting combination to assign the request to the correct rate bucket. AgentCore gateway supports the following dimension keys: targetName, toolName, qualifiedModelId, $.context.jwt., $.context.iam.principal, and $.context.iam.sourceIdentity. We will explore each of these through examples in the sections that follow.

Entries are the rules within a rate limit. Each entry specifies a set of dimension keys to match, and the allowed throughput for that match. Entries support the special catch-all default value * that gives each distinct value its own independent bucket at the configured rate. When the gateway evaluates a request, it checks whether an entry matches by name before falling back to the wildcard. A named entry takes precedence because it refers to the value explicitly rather than relying on the catch-all.

Taking the preceding rate limit as an example, when a request arrives for the Booking target (MCP server), the gateway matches the first entry and allows up to 100 RPS. This entry takes precedence because the most specific value match wins over default value * as it refers to the Booking target by name. For any other target, no named entry exists, so the gateway falls back to the wildcard entry and allows up to 10 requests per second. Each target that matches the wildcard (Docs, BedrockMantle, CustomPlatform, and awsdocsagent) gets its own independent bucket.

You can combine multiple dimension keys for more granular control. For example, dimensionKeys: [“targetName”, “$.context.jwt.role”] groups traffic by both target and caller identity role claim, giving each user-group (Basic, Advanced, or Beta in the preceding example) their own independent rate bucket per target.

Types of rate limits and example configurations

AgentCore gateway enforces two layers of rate limiting: customer-defined rate limits and Service Quotas. Customer-defined rate limits are evaluated first. If the request passes, service quotas are evaluated. The following sections explain service quotas and the different types of customer-defined rate limits.

• Service managed quotas.

These are the limits enforced on AgentCore gateway per AWS account by the service. Service managed quotas define the ceiling that customer-defined rate limits cannot exceed. The effective rate for requests is the minimum of the customer-defined limit and the service-managed limit. You can request increases for some quotas using the Service Quotas console.

• Customer-defined user limits.

User-level limits use $.context.jwt., $.context.iam.principal, and $.context.iam.sourceIdentity as the dimension keys to control how much traffic individual users or entire user-group can consume. These limits enforce fair usage across your caller base and prevent any single caller from monopolizing gateway capacity. The following example assigns different request rates per user group. The JWT role claim is an array, so each unique combination requires its own entry.

aws bedrock-agentcore-control create-gateway-rate-limit \
  --gateway-identifier my-gateway-abc1234567 \
  --dimension-keys '["$.context.jwt.role"]' \
  --description "Per-role request and connection limit" \
  --entries '[
    {
      "dimensions": {"$.context.jwt.role": "[\"Basic\"]"},
      "requests": [{"rate": 100, "period": "minute"}],
      "connections": [{"rate": 50, "period": "second"}]
    },
    {
      "dimensions": {"$.context.jwt.role": "[\"Advanced\"]"},
      "requests": [{"rate": 300, "period": "minute"}],
      "connections": [{"rate": 150, "period": "second"}]
    },
    {
      "dimensions": {"$.context.jwt.role": "[\"Advanced\", \"Beta\"]"},
      "requests": [{"rate": 300, "period": "minute"}],
      "connections": [{"rate": 200, "period": "second"}]
    },
    {
      "dimensions": {"$.context.jwt.role": "*"},
      "requests": [{"rate": 80, "period": "minute"}],
      "connections": [{"rate": 10, "period": "second"}]
    }
  ]'

In this configuration, Basic users receive two buckets, 100 RPM and 50 CPS, meaning every request from any Basic user counts toward the same 100 RPM total, and every connection counts toward the same 50 CPS total. If one Basic user sends 80 requests in a minute, only 20 remain for all other Basic users in that window. Advanced users receive their own two buckets at 300 RPM and 150 CPS, governed by the same collective behavior. Users with [“Advanced”, “Beta”] group membership receive two buckets at 300 RPM and 200 CPS. The higher connection allowance accommodates their streaming-heavy benchmarking workloads.

However, within a group, a single user can still consume the entire group rate bucket, throttling everyone else in that group. For example, one Basic user sending 100 requests in a minute would leave zero capacity for all other Basic users. To prevent this, we create the following rate limit configuration as well.

aws bedrock-agentcore-control create-gateway-rate-limit \
  --gateway-identifier my-gateway-abc1234567 \
  --dimension-keys '["$.context.jwt.role", "$.context.jwt.sub"]' \
  --description "Per-user request and connection limit within each role" \
  --entries '[
    {
      "dimensions": {"$.context.jwt.role": "[\"Basic\"]", "$.context.jwt.sub": "*"},
      "requests": [{"rate": 20, "period": "minute"}],
      "connections": [{"rate": 10, "period": "second"}]
    },
    {
      "dimensions": {"$.context.jwt.role": "[\"Advanced\"]", "$.context.jwt.sub": "*"},
      "requests": [{"rate": 60, "period": "minute"}],
      "connections": [{"rate": 30, "period": "second"}]
    },
    {
      "dimensions": {"$.context.jwt.role": "[\"Advanced\", \"Beta\"]", "$.context.jwt.sub": "*"},
      "requests": [{"rate": 60, "period": "minute"}],
      "connections": [{"rate": 50, "period": "second"}]
    },
    {
      "dimensions": {"$.context.jwt.role": "*", "$.context.jwt.sub": "*"},
      "requests": [{"rate": 20, "period": "minute"}],
      "connections": [{"rate": 20, "period": "second"}]
    }
  ]'

With this configuration, each individual user is capped at their own rate regardless of how many users exist in their group. The $.context.jwt.sub claim from the JWT uniquely identifies each user, enabling the gateway to track and enforce limits at the individual level. Even if the group-level limit allows 100 RPM total for Basic, no single user can consume more than 20 RPM and 10 CPS of that shared pool. The same logic applies to Advanced and Beta users at their respective individual caps. Together, the per-group limit and the per-user limit create a two-layer enforcement model: the group ceiling helps prevent one group from starving another, and the per-user ceiling helps prevent one individual from starving their peers within the same group.

Both rate limits are evaluated independently using AND semantics. A request must pass both the group-level limit and the per-user limit to proceed. If either check denies the request, the gateway returns a throttling response. For example, if Arnav (Basic) has consumed 20 RPM individually, his next request is denied by the per-user limit even though the Basic group still has 80 RPM of remaining capacity. Conversely, if the Basic group has collectively consumed 100 RPM, all Basic users are throttled regardless of their individual consumption.

• Customer defined target-level limits.

Target-level limits use targetName, qualifiedModelId, or toolName as the dimension key to control throughput to specific downstream targets, models, or tools. These limits protect backend capacity and distribute load across your target resources. The following example limits traffic on a per-target basis.

aws bedrock-agentcore-control create-gateway-rate-limit \
  --gateway-identifier my-gateway-abc1234567 \
  --dimension-keys '["targetName"]' \
  --description "Per-target rate limit" \
  --entries '[
    {
      "dimensions": {"targetName": "Booking"},
      "requests": [{"rate": 20, "period": "second"}]
    },
    {
      "dimensions": {"targetName": "Docs"},
      "requests": [{"rate": 15, "period": "second"}]
    },
    {
      "dimensions": {"targetName": "awsdocsagent"},
      "requests": [{"rate": 10, "period": "second"}],
      "connections": [{"rate": 60, "period": "second"}]
    },
    {
      "dimensions": {"targetName": "BedrockMantle"},
      "tokens": [{"rate": 100000, "period": "minute"}],
      "connections": [{"rate": 250, "period": "second"}]
    },
    {
      "dimensions": {"targetName": "CustomPlatform"},
      "tokens": [{"rate": 50000, "period": "minute"}],
      "connections": [{"rate": 100, "period": "second"}]
    },
    {
      "dimensions": {"targetName": "*"},
      "tokens": [{"rate": 10000, "period": "minute"}],
      "requests": [{"rate": 10, "period": "second"}],
      "connections": [{"rate": 50, "period": "second"}]
    }
  ]'

You can also use qualifiedModelId to set connection rate limits (CPS) per model, or toolName to set request rate limits (RPS) per individual tool such as Booking___bookTool or Docs___searchDocsTool.

Note: We exclude customer-defined target-level rate limits from our use-case configuration. Beta users run heavy benchmarking workloads against restricted models, consuming a disproportionate share of a shared target-level limit. Because this limit dimensions only on targetName, all users share a single ceiling, meaning high traffic from one group or individual can starve everyone else on that target. When a subset of users is expected to dominate token or connection consumption on a specific target, scope the limit by identity instead (for example, [“targetName”, “$.context.jwt.role”]). See the following example.

• Customer defined target-user level limits.

Hybrid limits combine target and user dimensions in a single rate limit configuration, giving you the most granular control. Using multi-dimension keys, you can scope rate limits to a specific user or user group on a specific target, model, or tool.

The following example enforces token limits at the model level, scoped to each user within their group. The qualifiedModelId dimension is the fully qualified model identifier for inference targets. It uniquely identifies the mode