Amazon Bedrock でプロンプトの移行と最適化を支援する新機能

The problem: Prompt engineering at scale is still painful

Migrating prompts to new models on Amazon Bedrock, or optimizing them for your current model, is still one of the most manual parts of building a generative AI application. Say you have built a deployed generative AI application. It works. Your prompts are tuned, your outputs are consistent, and your users are happy. Then a new model becomes available on Amazon Bedrock that is faster, cheaper, and more capable, and you must decide whether to migrate.

The decision should be straightforward, but it rarely is.

Customers spend days to weeks optimizing prompts and re-evaluating responses when they migrate to a new model. The same effort applies when they try to improve performance on their current model. The cycle looks the same every time. Rewrite the prompt, run it against test cases, compare results, tweak, and repeat. Now multiply that work by every prompt template in production and every model candidate worth evaluating. The result is a problem that scales with your ambition.

Today, Amazon Bedrock introduces Advanced Prompt Optimization, a tool that optimizes prompts for up to 5 models on Bedrock while comparing original and optimized performance. In this post, we show you how to use Amazon Bedrock Advanced Prompt Optimization to migrate and optimize prompts across multiple models in a single job. This approach replaces days of manual iteration with a guided, metrics-driven workflow.

The challenge: How this breaks the generative AI software development lifecycle

Prompt migration and optimization sit at a critical chokepoint in the generative AI development lifecycle. When this step is slow or manual, you feel it in 4 areas:

• Model lock-in. Teams avoid migration because the time and effort of re-tuning is too high, even when newer models would reduce latency and inference cost.

• Underperformance. When you write prompts for one model, you don’t use the full capability of another.

• Regression blindness. Without systematic evaluation against ground truth, you can’t distinguish “different” from “worse.” Teams ship degraded outputs without knowing it.

• Slow iteration cycles. Every prompt change requires manual A/B testing, which keeps engineers in evaluation loops instead of building features.

You need a prompt optimizer with built-in evaluations that test multiple models at once. It should also let you guide how prompts and responses change, grounding the optimization in real use cases and data.

The architecture: Metric-driven feedback loop

Amazon Bedrock Advanced Prompt Optimization works backward from your evaluation metric. Instead of rewriting prompts heuristically, the system operates in a reinforcement learning-style feedback loop, without changing model weights:

• You provide a prompt template, example user inputs (text or multimodal), optional ground truth answers, and an evaluation metric.

• The optimizer sends your template and inputs to your chosen inference models. It evaluates responses against your metric and then rewrites the prompt.

• The loop repeats (evaluate, rewrite, evaluate), iteratively optimizing your prompt against your evaluation metric.

• The output is the original and optimized prompts. For each, you get evaluation scores, time to first token (TTFT) per sample, and inference cost estimates at on-demand pricing.

This architecture is model-agnostic. You can use your choice of model on Amazon Bedrock. A single job can target up to 5 models at once. You get an optimized prompt for each candidate and a direct comparison of quality, latency, and cost, all from one submission.

The following diagram shows how these pieces fit together.

What is TTFT? TTFT (time-to-first-token) measures how quickly the model starts responding: the time from sending a request to receiving the first token. It’s a useful proxy for perceived latency, like a chatbot beginning to type.

TTFT captures when the response starts, so pair it with output-token counts for the full latency picture. Since it also reflects real-time infrastructure conditions, read single-run values as a directional signal.

Advanced Prompt Optimization optimizes for your quality metric (the score) and reports TTFT and cost alongside every result. This gives you a complete view of quality, latency, and cost to choose the right prompt with confidence.

Solution: Amazon Bedrock Advanced Prompt Optimization

Amazon Bedrock introduces Advanced Prompt Optimization, a tool that optimizes prompts for up to 5 models on Bedrock while comparing original and optimized performance. You can use it in two ways:

• Model migration. Select your current model as a baseline plus up to 4 candidates. You get optimized prompts for each and a side-by-side comparison.

• Same-model improvement. Select only your current model. You get a better prompt without changing anything else.

Three evaluation modes

You choose one evaluation method per prompt template, and each method serves a different use case. The following table summarizes when to use each one.

Mode

Best for

What you provide

AWS Lambda function

Concrete metrics (accuracy, F1, ROUGE, JSON match, and so on)

An AWS Lambda function with compute_score logic

LLM-as-a-Judge

Open-ended tasks (summarization, reasoning, generation)

A rubric prompt and a judge model selection

Steering criteria

Brand voice, format, safety constraints

Up to 5 natural language criteria

If you omit all evaluation fields, the system uses a built-in default that combines accuracy, answer completeness, and writing style scoring. We recommend you define your own evaluation metric for the best results. If you have more than one objective to optimize for, you can create a composite metric with weights for each of your criteria in the scoring algorithm. Your metric output must be a single number.

Multimodal support

Advanced Prompt Optimization supports multimodal inputs: PNG, JPG, JPEG, GIF, WebP, and PDF files referenced through Amazon Simple Storage Service (Amazon S3) URIs. You can optimize prompts for document analysis, image classification, visual question answering, and other tasks that combine text with visual input.

Getting started: Console

This section walks you through the console flow step by step: you create a job, select models, upload your input dataset, submit the optimization, and read the results. The Advanced Prompt Optimization landing page lists your jobs and their status and provides a button to start a new one, as shown in the following screenshot.

Advanced Prompt Optimization landing page

Step 1: Create a prompt optimization job

In the Amazon Bedrock console, in the left navigation pane, choose Advanced Prompt Optimization. On the landing page, choose Create prompt optimization. The next screenshot shows the creation page.

Step 1: Create a prompt optimization job

Step 2: Select inference models

In the Model selection section, choose Add model. Select your current model as a baseline. To add candidates, choose Add model again and select up to 4 additional models. If you are not migrating, select only your current model.

Step 2 Select Inference Models

Step 3: Prepare your input dataset (JSONL)

Your input file uses JSONL format, with one JSON object per line, and each line is one prompt template to optimize. For full details, see the technical documentation. The following snippet shows the core schema.

{
    "version": "bedrock-2026-05-14",
    "templateId": "unique-id-for-this-template",
    "promptTemplate": "Your prompt with {{variableName}} placeholders",
    "evaluationSamples": [
        {
            "inputVariables": [
                {"variableName": "value"}
            ],
            "referenceResponse": "Expected output (optional but recommended)"
        }
    ]
}

Then add one of the evaluation method fields. For more details and overall guidance, see the technical documentation. The following snippet shows the AWS Lambda metric option.

{
    "evaluationMetricLambdaArn": "arn:aws:lambda:us-west-2:111122223333:function:my-metric",
    "customEvaluationMetricLabel": "my_score_name"
}

The following snippet shows the LLM-as-a-Judge option.

{
    "customLLMJConfig": {
        "customLLMJPrompt": "Your rubric. Use {{prompt}}, {{response}}, {{referenceResponse}} as placeholders.",
        "customLLMJModelId": "us.anthropic.claude-opus-4-6-v1"
    },
    "customEvaluationMetricLabel": "my_judge_metric"
}

Authoring your judge rubric. In your customLLMJPrompt, reference 3 placeholders with double curly braces: {{prompt}} (the fully rendered prompt, which is the prompt template combined with the evaluation sample), {{response}} (the model output), and {{referenceResponse}} (the ground truth). Advanced Prompt Optimization substitutes these values for each candidate and sample before sending the rubric to the judge model. For details, see Use an LLM as a judge.

The following snippet shows the steering criteria option.

{
    "steeringCriteria": [
        "be concise",
        "use professional tone",
        "avoid speculation"
    ]
}

Step 4: Upload and submit

In the Prompt templates source section, choose Upload files directly from your local machine to upload your JSONL file. Alternatively, choose Import prompt templates from Amazon S3 and enter the S3 URI. In the S3 Output location section, enter your S3 output path. Choose Create optimization to submit the job.

Step 5: Interpret results

The results page shows 4 things: evaluation scores before and after optimization (per model), the optimized prompts ready to copy and deploy, TTFT per sample across models, and cost estimates based on on-demand inference pricing.

The following screenshots show the actual results of the examples we ran for this post: NESTFUL and XSum. We submitted them through the API and SDK (see the next section) using the datasets from the tutorial GitHub repo. Each job used 5 evaluation samples. The console renders the same results whether the job was created from the console or the API. You can use the repo as a reference for the expected output format and reproduce these views in your own account. Your results will vary based on your dataset, prompt complexity, and model choice. The summary view gives you the headline before and after scores. The following screenshot shows the NESTFUL run (function calling, Amazon Nova 2 Lite), where accuracy increased from 0.267 to 0.647.

NESTFUL: Summary Table Optimization Results

The details view shows the original and optimized prompt side by side, along with per-sample metrics and the full token and latency breakdown, as shown in the following screenshot.

NESTFUL: Detailed Results view

The following screenshot shows the summary view for the XSum run (summarization, Claude Haiku 4.5). The Quality Score increased from 0.550 to 0.743 while output tokens dropped from 51 to 28.

XSum: Summary Table Optimization Results

The following screenshot shows the corresponding details view for XSum.

XSum: Detailed Results view

Getting started: API / SDK (boto3)

You can also drive the whole workflow with boto3. The following code creates a job that targets two candidate models.

import boto3

bedrock = boto3.client("bedrock", region_name="us-west-2")

response = bedrock.create_advanced_prompt_optimization_job(
    jobName="my-migration-job",
    modelConfigurations=[
        {"modelId": "us.amazon.nova-2-lite-v1:0"},              # candidate 1
        {"modelId": "us.anthropic.claude-haiku-4-5-20251001-v1:0"}, # candidate 2
    ],
    inputConfig={
        "s3Uri": "s3://amzn-s3-demo-bucket/inputs/prompts.jsonl"
    },
    outputConfig={
        "s3Uri": "s3://amzn-s3-demo-bucket/outputs/migration-job/"
    },
)

job_arn = response["jobArn"]
print(f"Job submitted: {job_arn}")

Poll for completion

A job runs asynchronously, so poll its status until it reaches a terminal state. The following code checks every 60 seconds.

import time

while True:
    info = bedrock.get_advanced_prompt_optimization_job(jobIdentifier=job_arn)
    status = info["jobStatus"]
    print(f"Status: {status}")

    if status in ("Completed", "Failed", "Stopped"):
        break
    time.sleep(60)

Download results

When the job completes, it automatically writes a JSONL results file to the Amazon S3 output location that you selected when you started the job. The following code uses boto3 to download it.

# Results land at: s3://output-uri/job-id/advanced_prompt_optimization_results.jsonl
s3 = boto3.client("s3")
job_id = job_arn.rsplit("/", 1)[-1]
result_key = f"outputs/migration-job/{job_id}/advanced_prompt_optimization_results.jsonl"
s3.download_file("amzn-s3-demo-bucket", result_key, "results.jsonl")

Full example: Steering criteria with multimodal input

The following example builds a steering-criteria dataset that pairs a question with a PDF document, writes it to JSONL, and uploads it to Amazon S3.

import json

Build your input dataset

record = {

"version": "bedrock-2026-05-14",

"templateId": "document-qa-v1",

"promptTemplate": "Answer the question based on the document.\n\nQuestion: {{question}}",

"steeringCriteria": [

"State the answer first, then provide supporting evi