Amazon Nova Forge におけるハイパーパラメータ最適化の芸術と科学

Large language models (LLMs) deliver strong results on general tasks, but they often struggle with specialized work that requires understanding proprietary data, internal processes, or domain-specific terminology. Amazon Nova Forge addresses this by enabling you to build your own frontier models using Amazon Nova. You can start development from early model checkpoints, blend proprietary data with Amazon Nova-curated training data, and host custom models securely on AWS. A key capability is data mixing, which blends your training data with curated datasets. This helps the model absorb your domain while retaining broad reasoning, instruction-following, and language capabilities. This prevents catastrophic forgetting that typically undermines domain customization.

Successful customization requires careful hyperparameter tuning. Learning rate, data mixing ratio, checkpoint selection, and training techniques all interact in ways that can silently undermine a training run. If any of them are wrong, you trade one problem for another. This post covers the art (strategic trade-offs) and science (metric-driven decisions) of hyperparameter tuning on Amazon Nova Forge to help you avoid expensive failed training runs.

Fine-tuning for domain-specific tasks means improving performance in one area without degrading the model’s general capabilities, and getting that balance right is harder than it looks. This post walks through how to navigate that balance, from selecting the right customization strategy for your data and task, to configuring the training parameters that most influence outcomes, like learning rate, batch size, and checkpointing. We also cover the common mistakes that lead to wasted training runs and how to catch them early, so you can improve domain performance without degrading general capabilities or burning through compute on avoidable failures.

By the end, you will know how to improve domain performance without degrading general capabilities and how to avoid the expensive failures that come from getting the balance wrong.

The hyperparameter tuning challenge

Achieving this balance is harder than it appears. Three fundamental challenges make hyperparameter tuning particularly difficult on domain-specialized models.

Challenge 1: Catastrophic forgetting

When you train a model on narrow domain data, the model can overwrite general capabilities it learned during pre-training. This phenomenon, called *catastrophic forgetting*, shows up as degraded performance on tasks outside your training domain. The model becomes highly specialized but loses instruction-following ability, reasoning capability, and broad knowledge. In production, this means a customer service model fine-tuned on your support tickets may no longer reason about ambiguous requests or maintain coherent multi-turn conversations.

This creates a stability-flexibility tradeoff. Ideally, the model is flexible enough to learn about an organization’s domain but stable enough to retain general capabilities. Nova Forge addresses this through data mixing, which blends your training data with curated datasets during training, and checkpoint selection, which lets you choose how much existing alignment to preserve.

Challenge 2: Finding the right learning rate

The learning rate controls how much the model’s weights change in response to each batch of training examples. It’s the most sensitive hyperparameter across all customization techniques. A learning rate that’s too high causes the model to overshoot the optimal state, destabilize during training, or forget base capabilities rapidly. A learning rate that’s too low wastes compute on very slow convergence. The right value depends on your data distribution, mixing ratio, and training technique.

Nova Forge provides calibrated service defaults for each training technique that account for these interactions. When you use data mixing, the sensitivity increases further. Deviating from the default learning rate when mixing Nova data with your own data is the most common source of training instability, so these service defaults are the recommended starting point.

Challenge 3: Baseline performance constraints

Reinforcement fine-tuning (RFT) is a technique that improves model behavior by generating multiple candidate responses and scoring them against quality criteria. The model learns by comparing its own outputs and reinforcing the better ones. RFT works at its full capacity within a specific range of baseline task accuracy, measured by how often the model produces correct or high-quality responses before fine-tuning. If baseline accuracy is too low (the model rarely produces correct responses), there aren’t enough good examples for reward-guided exploration to learn from. If baseline accuracy is already very high, additional training yields diminishing returns and risks degrading existing performance. This means RFT can’t close large competence gaps where the model fundamentally lacks the knowledge or reasoning ability to attempt a task. It refines and strengthens behaviors the model can already partially demonstrate, rather than teaching entirely new capabilities from scratch.

The Nova Forge pipeline addresses both bounds. For low-baseline scenarios, run supervised fine-tuning (SFT) first to establish the foundational capabilities needed for effective reward-based learning. For high-baseline tasks, make sure that your reward function has discriminative power across the model’s quality range. If most responses already score highly, RFT has no meaningful signal to optimize against.

The Nova Forge customization pipeline

Understanding these challenges frames how the Amazon Nova Forge customization pipeline is designed to address them. Nova Forge provides three complementary customization techniques, each serving a distinct purpose in the model development lifecycle.

Technique

What it does

When to use

Input data

Continued pre-training (CPT)

Expands foundational model (FM) knowledge through self-supervised learning on large quantities of unlabeled, domain-specific proprietary data. CPT teaches the model domain terminology and patterns from your text corpus.

You need the model to understand specialized vocabulary, industry concepts, or organizational knowledge that does not exist in the base model.

Large volumes of unlabeled domain text. Nova Forge supports CPT with data mixing and three checkpoint options (pre-trained, mid-trained, and post-trained), each suited to different data scales and downstream requirements.

Supervised fine-tuning (SFT)

Customizes model behavior using a training dataset of input-output pairs specific to your target tasks. SFT teaches the model “given X, output Y” behavior through demonstrations.

You need the model to follow specific response formats, adopt particular tones, or perform structured tasks like classification or extraction.

1,000–10,000 high-quality demonstrations per task. Quality, consistency, and diversity matter more than volume. Nova Forge supports SFT with data mixing using Amazon Nova-curated datasets, including reasoning-instruction-following categories that preserve general capabilities.

Reinforcement fine-tuning (RFT)

Steers model output toward preferred outcomes using reward signals. RFT optimizes the model within a behavioral neighborhood established by prior training for single-turn or multi-turn conversational tasks.

You have a clear reward function that can evaluate response quality and want to push performance beyond what SFT alone achieves.

Prompts and a reward function. Nova Forge supports bringing your own external reward environment through AWS Lambda, enabling custom verification logic for domain-specific quality assessment.

When all three stages are used together (CPT, then SFT, then RFT), they produce the strongest results. However, with the right pipeline, each stage can be optional. It depends on your data availability, task type, and starting point. CPT is only needed when the base model lacks domain vocabulary or knowledge your task requires. SFT and RFT can be used independently or combined depending on what your task demands.

*Figure 1: The Amazon Nova Forge customization pipeline. CPT teaches domain knowledge from unlabeled text, SFT teaches task-specific behavior from demonstrations, and RFT optimizes performance using reward signals. Each stage is optional, and the full pipeline (CPT, then SFT, then RFT) produces the strongest results when all three are applicable to your use case.*

Amazon SageMaker AI offers different environments for customization: SageMaker Serverless provides a UI-driven experience with automatic compute provisioning, SageMaker AI training jobs (SMTJ) provide a fully managed experience without cluster management, while Amazon SageMaker HyperPod offers specialized environments for advanced distributed training scenarios.

Strategic decisions

With the customization pipeline in view, the next step is understanding the qualitative trade-offs that shape your configuration. These strategic decisions matter as much as any individual hyperparameter value: checkpoint selection, data mixing, and training mode.

Checkpoint selection (most impactful decision)

For CPT, checkpoint selection is more impactful than any hyperparameter. Amazon Nova Forge provides three checkpoint options, each suited to different data scales and downstream requirements.

• Pre-trained checkpoints are the most flexible and offer the fastest convergence. These checkpoints accept new patterns readily and work best for large-scale CPT with substantial token budgets exceeding 100 billion tokens. When using pre-trained checkpoints with large datasets, you can use a higher learning rate (such as 1e-4) to accelerate knowledge absorption. You then need to gradually reduce the learning rate back to approximately 1e-6 for model stability before running SFT to let the model “settle” into what it learned without overshooting. Be aware that pre-trained checkpoints have no instructions for tuning. After CPT, you must run SFT to make the model useful for downstream tasks.

• Mid-trained checkpoints balance flexibility and alignment. They accept domain knowledge while retaining some instruction-following behavior. Use mid-trained checkpoints for medium-sized datasets where you want faster domain adaptation than post-trained but more stability than pre-trained. Mid-trained checkpoints work well for full rank training, which updates every parameter in the model during fine-tuning, with large, structured datasets.

• Post-trained checkpoints are the most resistant to new patterns but preserve instruction-following and general capabilities. Use post-trained for smaller-scale CPT where preserving alignment matters more than maximizing domain knowledge absorption. Post-trained checkpoints are the recommended starting point for LoRA (Low-Rank Adaptation), which freezes the original model weights and trains small adapter matrices on top, and other parameter-efficient fine-tuning methods, as they maintain the model’s existing capabilities while allowing targeted adaptation. For small datasets or later-stage checkpoints, use conservative learning rate values from the service defaults.

*Figure 2: Checkpoint selection for continued pre-training. Pre-trained checkpoints offer maximum flexibility for large datasets but require SFT afterward to restore instruction-following. Post-trained checkpoints preserve alignment and suit smaller datasets or parameter-efficient methods like LoRA.*

Data mixing strategy

Without data mixing, training on narrow domain data can cause the model to become unstable, resulting in erratic training behavior (gradient instability or loss spikes) or a sudden degradation in performance.

When configuring data mixing, balance your customer data around 50 percent of the total mix for most use cases. For SFT, always include the “reasoning-instruction-following” category in your Nova data mix. This single category significantly improves generic benchmark performance after fine-tuning. Skipping this category is a common cause of degraded reasoning performance in fine-tuned models.

Data mixing is very sensitive to learning rate. Deviating from the default learning rate when using data mixing causes instability. This is the most common mistake practitioners make. If you observe training instability with data mixing, the learning rate is the first suspect.

Finding the optimal mixing ratio requires experimentation. Hold your domain data constant and vary the Nova data proportion across several runs. Domain performance typically stays constant while general capabilities keep improving the more Nova data is mixed in. Place your highest-quality data toward the end of training for better convergence.

Training mode: Low-Rank Adaptation (LoRA) vs Full Rank

Amazon Nova Forge supports two training modes that determine how model parameters are updated during training:

• LoRA updates only adapter layers, offering lower compute costs, faster iteration, and compatibility with on-demand inference. LoRA achieves near Full Rank performance for most tasks while being more forgiving of suboptimal hyperparameters. The default alpha scaling factor of 64 works for most tasks. Increase alpha if LoRA is under-adapting to your data or decrease it if LoRA is over-adapting and losing general capabilities. Use post-trained checkpoints as your starting point for LoRA training.

• Full Rank updates all model parameters, providing maximum adaptation capacity. Full Rank requires Amazon Bedrock Provisioned Throughput for deployment (On-Demand is only available for LoRA-based customization) and higher compute during training. Use Full Rank when you have validated your pipeline and your deployment architecture justifies the additional cost. Mid-trained checkpoints work well for Full Rank training with large, structured datasets.

Start with LoRA to validate your pipeline, data quality, and reward function (for RFT). Graduate to Full Rank when you have confirmed the approach works, and your production requirements justify it (for example, model performance or cost constraints).

Recommended workflow

Applying these strategic decisions to your specific situation depends on what data and objectives you have. The following paths map your starting conditions to the right sequence of techniques.

If you have labeled demonstrations and a verifiable reward function (SFT then RFT):

• Start with SFT using LoRA to teach the target behavior and establish baseline competency.

• Enable data mixing with “reasoning-instruction-following” included to preserve the model’s ability to follow structured prompts and produce well-formatted outputs during domain adaptation.

• Use default learning rates without modification.

• Monitor validation loss to select the best SFT checkpoint.

• Graduate to RFT on the SFT checkpoint to optimize further through reward signals.

• Consider Full Rank training only after validating the approach with LoRA.

• Test thoroughly on both your domain task and general benchmarks before production deployment (see the Experiments and insights section for an example).

If you can define verifiable outcomes but cannot easily label responses at scale (RFT only):

• Evaluate base model performance on a representative sample of your task first.

• Proceed with RFT directly if the base model achieves more than approximately 5 percent positive reward.

• Fall back to SFT if reward scores are consistently near zero. The model needs baseline competency before reward-guided learning can take effect.

If the base model lacks domain vocabulary or knowledge your task requires, start with CPT:

• Run CPT to absorb domain knowledge from unlabeled text.

• Follow with SFT. Pre-trained checkpoints used for CPT have no instruction tuning, so SFT is required after CPT to make the model useful.

• Optionally follow with RFT to further optimize performance.

Parameter configuration

With strategic decisions made, you can now optimize specific hyperparameters that govern how each technique executes. This section provides guidance for each technique.

Learning rate configuration

Learning rate controls how quickly the model updates based on training signals. Service defaults represent tested configurations that work across diverse use cases.

• For CPT: Start at service defaults. For large datasets exceeding one trillion tokens, you can use a higher learning rate (such as 1e-4) to accelerate knowledge absorption, but you need a ramp-down stage to reduce the learning rate back to approximately 1e-6 for model stability before SFT. The constant_steps parameter controls how many steps the model trains at the peak learning rate before this ramp-down stage begins. Increase constant_steps for very large token runs where more steps at full learning rate help domain absorption. For smaller datasets or later-stage checkpoints, use the default (lower) learning rate from the start.

• For SFT: Stick to service defaults, especially with data mixing. The recommended learning rate is 1e-5 for LoRA and 5e-6 for full-rank SFT. Deviating from the default learning rate when mixing Nova data causes instability. If you observe training instability with data mixing, the learning rate is the first suspect.

• For RFT: Start at service defaults. Adjust in small multiplier increments only if needed. If reward drops suddenly and does not recover, the learning rate is likely too high. Even a small multiplier increase can drop performance below baseline.

Configure warmup steps to approximately 15 pe