Amazon Bedrock で大規模な自律型 AI オペレーションを構築する方法

Amazon Bedrock powers generative AI for more than 100,000 organizations worldwide—from startups to global enterprises across every industry. It provides the proven infrastructure and comprehensive capabilities to confidently build applications and agents that work in production with the flexibility, enterprise security, and proven scalability you need to innovate boldly and deliver AI that drives real business impact. As organizations scale their generative AI applications powered by Amazon Bedrock across multiple foundation models and production workloads, proactive operational management becomes key to sustaining innovation velocity.

As generative AI adoption grows across teams, organizations can benefit from a purpose-built operational monitoring solution that delivers: 1) proactive, multi-layer monitoring that anticipates quota increase needs as adoption grows by tracking usage patterns and accelerates operational issue triage for generative AI workloads powered by Amazon Bedrock; 2) context-aware support case automation that accelerates mean time to resolution by equipping AWS support engineers with the information they need; 3) duplicate case prevention that suppresses new case creation when an unresolved case of the same alarm category already exists, avoiding distraction from active investigations; 4) contextualized notifications that empower AI SRE teams to act quickly; and 5) continued focus on innovation by reducing manual operational overhead.

In this post, we introduce Amazon Bedrock Ops Alert, a three-layer automated monitoring solution that proactively detects operational issues, dynamically adjusts alarm thresholds, classifies alarms by category, automatically creates context-aware support cases, helps prevent duplicate cases when an unresolved case of the same alarm category is already active, and delivers contextualized notifications to AI SRE teams. We walk through the solution architecture and how you can deploy it in your own environment.

Scaling operational maturity for generative AI workloads

Amazon Bedrock provides service quotas for requests per minute (RPM) and tokens per minute (TPM) to help manage resource allocation across customers. These quotas can be increased through AWS Support cases as workloads grow. A common initial approach uses third-party dashboarding solutions backed by Amazon CloudWatch metrics, combined with manual processes to monitor quota consumption and request increases when needed. This approach serves teams well during early adoption.

As adoption grows, organizations often discover that workload optimization addresses capacity needs more effectively than quota increases. Cross-region inference helps organizations manage unplanned traffic bursts by using compute across different AWS Regions. When using an inference profile tied to a specific geography, Amazon Bedrock automatically selects the optimal commercial AWS Region within that geography to process the inference request. Global cross-region inference extends this beyond geographic boundaries by routing inference requests to support commercial AWS Regions worldwide, optimizing available resources and providing higher model throughput. With global inference profiles, workloads are no longer constrained by individual Regional capacity, providing access to a much larger pool of resources and approximately 10% cost savings compared to geographic cross-region inference. In the post Unlock global AI inference scalability using new global cross-Region inference on Amazon Bedrock with Anthropic’s Claude Sonnet 4.5, we detail how global inference profiles dynamically route requests across the AWS global infrastructure to absorb demand that would otherwise require quota increases.

Prompt caching is an optional feature that reduces inference response latency and input token costs. By adding portions of the context to a cache, the model skips recomputation of inputs, allowing Amazon Bedrock to share in the compute savings and lower response latencies. Prompt caching helps when workloads have long and repeated contexts that are frequently reused for multiple queries, reducing costs by up to 90% and latency by up to 85%, which directly lowers tokens-per-minute consumption. In the post Effectively use prompt caching on Amazon Bedrock, we walk through how to structure prompts to maximize cache hits across multiple API calls. Additional techniques such as batch inference and Intelligent Prompt Routing further reduce per-request overhead by dynamically selecting the most cost-effective model for each call.

As organizations adopt these optimization strategies and expand across multiple foundation models and production workloads, AI SRE teams look to complement them with automated operational monitoring to sustain innovation velocity and reduce mean time to resolution. Specifically, teams commonly identify four areas for improvement:

• Reactive operations: AI SRE teams often learn of operational issues only when business users report impact. This forces the team to operate reactively, with limited time to investigate and respond before the impact escalates.

• Opportunity for case context enrichment: When quota issues arise, support cases can benefit from richer context, distinguishing straightforward quota increases from issues requiring deeper investigation, to help support engineers resolve cases faster.

• Multiplying operational effort: As organizations adopt new foundation models for different use cases, each new model requires its own monitoring setup and quota increase requests. This undifferentiated heavy lifting grows linearly with the model portfolio.

• Moving target for alarm thresholds: Each approved quota increase requires the AI SRE team to manually recalculate and update CloudWatch alarm thresholds, creating operational overhead and the risk of configuration drift.

Solution overview

Amazon Bedrock Ops Alert is an AWS CloudFormation-based solution that implements comprehensive generative AI observability through three complementary detection layers. Each layer provides different visibility into generative AI workloads, from immediate operational issue detection to predictive anomaly identification.

The solution uses Amazon CloudWatch alarms, AWS Lambda functions, Amazon Simple Notification Service (Amazon SNS), the Service Quotas API, and AWS Support API.

The following diagram illustrates the solution architecture.

The workflow steps are as follows:

• During deployment, a Lambda function (Quota Calculator) queries the Service Quotas API for current RPM and TPM quota values and calculates alarm thresholds by applying configured percentages.

• The calculated thresholds are stored in AWS Systems Manager Parameter Store, and AI SRE team email contacts are stored in AWS Secrets Manager.

• Amazon Bedrock publishes runtime metrics (invocations, token counts, errors, throttles, and latency) to CloudWatch. Three independent monitoring layers evaluate these metrics:

Layer 1 (Critical Error Detection) monitors throttles, client errors, and server errors for immediate alerting.

• Layer 2 (Usage Rate Monitoring) compares RPM, TPM, and latency against the dynamically calculated thresholds.

• Layer 3 (Anomaly Detection) uses CloudWatch machine learning to identify unusual patterns across metrics.

• When a child alarm triggers, a composite alarm aggregates the state.

• The composite alarm publishes to an SNS topic (Raw Alarm Topic).

• The SNS topic invokes a Lambda notification processor function, which polls the composite alarm to identify which child alarms triggered and determines alarm severity (critical or warning).

• The notification processor queries the Service Quotas API for current RPM and TPM quota values.

• The notification processor queries CloudWatch for current usage metrics, including steady-state and peak RPM/TPM over the past 14 days and average tokens per request. It also reads stored alarm thresholds from Parameter Store and compares peak usage against thresholds to determine the support case scenario.

• If automated support case creation is enabled, the function classifies the alarm as quota-related or non-quota, checks for existing unresolved cases using category-aware duplicate detection (configurable lookback window, default 60 days), and either appends a communication to the existing case or creates a new AWS Support case. For quota-related alarms, the case includes pre-filled quota data with usage-validated content. For non-quota alarm (such as persistent errors or latency anomalies), providing context to assist with root cause analysis.

• After support case processing completes, the function sends formatted email notifications to stakeholders through a second SNS topic (Formatted Notification Topic), filtered by notification preference (all, critical, or warning). If a support case was created, the email includes the case ID and a direct link to the AWS Support console.

• The formatted notification is delivered as email to subscribed stakeholders.

• On a configurable schedule, an Amazon EventBridge rule triggers a Lambda function (Alarm Updater).

• The Alarm Updater queries the Service Quotas API for current RPM and TPM quota values.

• The Alarm Updater recalculates alarm thresholds by applying configured percentages, and updates CloudWatch alarms with new thresholds.

• The updated thresholds are stored in Parameter Store with timestamps for tracking history.

Three-layer monitoring architecture

The solution implements three monitoring layers using CloudWatch alarms that work independently to detect operational issues at different stages.

Layer 1: Critical error detection

The first layer monitors error metrics that indicate operational issues:

• ClientErrors alarm: Monitors the InvocationClientErrors metric to identify requests rejected due to client-side issues such as exceeded quota limits, validation errors, or invalid parameters.

• ServerErrors alarm: Monitors the InvocationServerErrors metric to identify service-side errors that may require investigation.

• Throttles alarm: Monitors the InvocationThrottles metric to identify requests explicitly throttled when the rate limit is reached.

These alarms use configurable thresholds and evaluation periods. Setting the error threshold to 0 with a single evaluation period triggers immediate alerts when an error occurs, while higher values provide tolerance for transient issues.

Layer 2: Usage rate monitoring

The second layer monitors usage metrics against dynamically calculated thresholds, providing proactive alerts before reaching your quota limit:

• HighInvocationRate alarm: Monitors the Invocations metric and triggers when the API request rate breaches the configured RPM threshold percentage of your quota.

• HighTPMQuotaUsage alarm: Monitors the EstimatedTPMQuotaUsage metric and triggers when estimated tokens per minute quota consumption breaches the configured TPM threshold percentage of your quota (includes cache write tokens and output burndown multipliers).

• HighLatency alarm: Monitors the InvocationLatency metric and triggers when response time breaches the configured latency threshold.

The solution automatically calculates alarm thresholds by querying the Service Quotas API and applying configurable percentages. For example, with an 80% threshold and a 100 RPM quota, the RPM alarm triggers at 80 requests per minute. For TPM, the same 80% threshold on a 1,000,000 TPM quota gives an 800,000 effective tokens threshold. The TPM alarm uses the EstimatedTPMQuotaUsage metric that tracks estimated TPM quota consumption, including cache write tokens and output burndown multipliers.

Layer 3: Anomaly detection

The third layer uses CloudWatch anomaly detection as the threshold type to identify unusual patterns across metrics:

• InvocationAnomaly alarm: Monitors the Invocations metric using anomaly detection to identify unusual request volume changes.

• InputTokenAnomaly alarm: Monitors the InputTokenCount metric using anomaly detection to identify abnormal input token usage.

• OutputTokenAnomaly alarm: Monitors the OutputTokenCount metric using anomaly detection to identify abnormal output token usage.

• LatencyAnomaly alarm: Monitors the InvocationLatency metric using anomaly detection to identify performance degradation trends.

CloudWatch machine learning analyzes historical data to establish normal behavior baselines, then alerts when current metrics exceed the upper threshold of the expected range. The solution monitors only upward deviations: usage drops are positive signals that don’t require intervention. This approach detects issues that static thresholds miss, such as gradual quota consumption increases or unexpected usage surges.

Automated threshold management

The solution dynamically adapts to quota changes through automated threshold recalculation:

• Initial calculation: During deployment, a Lambda function queries the Service Quotas API and calculates alarm thresholds based on current quotas and configured percentages.

• Scheduled updates: An EventBridge rule triggers threshold recalculation on a configurable schedule (default: every 1 day).

• Automatic alarm updates: When approved quota increases change the quota values, the solution updates CloudWatch alarms with new thresholds.

• Threshold history: Calculated thresholds are stored in Parameter Store, a capability of AWS Systems Manager, with timestamps.

This automation alleviates manual threshold maintenance when further quota increase requests are approved. AI SRE teams no longer need to track quota changes and manually update alarm configurations: the system self-corrects.

The following table describes how alarm thresholds are derived from Service Quotas values.

Threshold

Formula

Example

RPM threshold

RPM quota × (RequestsPerMinuteThresholdPercent / 100)

10,000 RPM quota × 80% = 8,000

TPM threshold

TPM quota × (TokensPerMinuteThresholdPercent / 100)

6,250,000 TPM quota × 80% = 5,000,000

The TPM threshold percentage is applied directly to the TPM quota. The usage validation compares 14-day peak TPM against this threshold when determining the support case scenario.

Automated support case creation

The solution optionally automates AWS Support case creation when operational issues are detected. This feature requires an AWS Business or Enterprise Support plan for Support API access.

The workflow operates as follows:

• The composite alarm triggers when a child alarm enters ALARM state.

• A Lambda function polls the composite alarm status, checking for eligible child alarms.

• The function reads stored alarm thresholds from Parameter Store and compares 14-day peak usage against thresholds to determine the support case scenario.

• The function classifies the alarm as quota-related or non-quota and checks the Support API for existing unresolved cases using category-aware duplicate detection (configurable lookback window, default 60 days).

• If an unresolved case of the same category exists, the system appends a communication to the existing case with full alarm details, updated metrics, and urgency context. If no duplicate exists, the system creates a new support case with scenario-appropriate content, either a quota increase request with usage-validated details, or a service investigation request without quota details.

The system classifies alarms into two categories and determines the appropriate response.

Quota-related alarms trigger a “Quota Request” support case with usage-validated content:

• RPM-specific alarms (HighInvocationRate, InvocationAnomaly) request an RPM quota increase only.

• TPM-specific alarms (HighTPMQuotaUsage, InputTokenAnomaly, OutputTokenAnomaly) request a TPM quota increase only.

• Undetermined quota alarms (Throttles, ClientErrors) request both RPM and TPM quota increases, providing context to help identify which limit was reached.

Non-quota alarms (ServerErrors, HighLatency, LatencyAnomaly) trigger an “Investigation Request” support case providing alarm context and usage