ReactBench v1 ベンチマーク発表

ReactBench is an evaluation for coding agents on realistic React work. Today’s models can pass tests in benchmarks but still write React that fails in production. Tests verify behavior, but they miss React performance, accessibility, and quality issues.

Here’s what ReactBench does differently:

• A higher bar than just passing testsEvery solution must pass both behavioral tests and React Doctor, our open-source, deterministic verifier for React code. Its 400+ rules scan the agent's code for broken effects, unnecessary renders, accessibility problems, and maintainability issues.
• Built by React expertsOur team built React Doctor ★ 13.8k, React Scan ★ 21.6k, and Million.js ★ 17.7k, used by engineers at GitHub, PayPal, Rippling, and Airbnb. We spent hundreds of hours curating tasks to measure real gaps in model capabilities.
• Real work in real codebasesReactBench spans open-source repositories and realistic changes grounded in existing projects (not synthetic puzzles). Agents must implement, preserve behavior, and meet the React-specific quality checks.

Results

We evaluate two complementary capabilities: implementing new features and improving existing code.

• Write ReactImplement a real feature or fix, pass held-out behavioral tests, and validate no new React Doctor issues were introduced. See example
• Fix ReactRefactor React failures from existing code while preserving existing behavior without introducing new React Doctor issues. See example

We report the average of the pass@1 across 5 trials.

ReactBench remains unsaturated, even on leading models. GPT-5.6 Sol at XHigh has the highest observed aggregate score at 43.1%, while Claude Fable 5 at XHigh reaches 41.2%. Across evaluated configurations, Fable costs $7.97 per trial on average versus $1.37 for Sol, or 5.8× as much. At XHigh, Fable costs 6.3× as much as Sol. Even the best configurations solve fewer than half of benchmark task attempts.

We also evaluated task discrimination across model configurations. In the 51-task panel, 36 tasks (70.6%) had a cross-configuration standard deviation above 0.10.

In terms of balancing quality and cost, GPT-5.6 Terra at Medium reaches 38.0%, 5.1 percentage points behind GPT-5.6 Sol at XHigh, while using 10.8% fewer output tokens and costing 63.2% less per task. It retains most of the leading configuration’s performance at roughly one-third the cost, making it the strongest practical tradeoff for high-volume workloads.

GPT-5.6 Sol at XHigh achieved the highest overall pass rate. However, the differences among the leading configurations are too small to identify a clear winner with confidence.

Across 4,455 Write React trials, the evaluated models introduced 1,194 graded React Doctor issues. Bugs, including security findings, accounted for 925, or 77.5%. The most common issues involved list rendering and Hook correctness.

The verifier-outcome taxonomy shows that Write and Fix tasks fail differently. Among 2,486 failed Write trials, 1,623 (65.3%) failed behavioral tests but passed React Doctor. Among 3,219 failed Fix trials, 1,956 (60.8%) passed behavior but failed React Doctor.

Write tasks fail primarily on requested behavior, while Fix tasks fail primarily on unresolved React problems.

Explore the results

Why we built ReactBench

React is the dominant frontend framework and the most popular target for coding agents. Roughly 70% of websites built on a JavaScript framework choose React.

We have seen the risks firsthand. React Doctor is our open source tool for scanning React issues used by engineers at PayPal, Rippling, Polymarket, and the Centers for Disease Control and Prevention (CDC). Adoption is largely driven by the increase of model-generated code that makes it easier for subtle defects to reach production. As models write more React, small mistakes can propagate at enormous scale. In the worst cases, these defects lead to production failures:

• OutagesIncorrect useEffect usage takes down production. Cloudflare traced its September 2025 dashboard and API outageto one effect with a faulty dependency. Despite human review and test, the bug still shipped to production.

-

-

Building ReactBench

Each task type uses a different construction and evaluation process.

Write React

Write React measures whether an agent can implement real work without introducing a React regression. Each task starts from a merged pull request in an open-source repository. The agent receives the base repository and an issue-style instruction. The reference patch and verifier are not provided.

• BehaviorThe verifier injects the hidden behavior tests and verifies the results in a separate container after the agent completes.
• React healthA pinned React Doctor scan compares the submission with the base commit. If a new issue is detected, it fails.

Fix React

Fix React measures whether an agent can recognize and refactor React problems from source code alone. We select a component with known React issues and ask the agent to improve it without naming those findings.

• React healthThe agent must remove every target without introducing another graded React issue. The grader ignores line shifts.
• BehaviorThe task’s test suite must still pass to ensure refactoring the component did not regress it in functionality.

We strip React Doctor and other React-aware linters from the agent image. Only the verifier contains the pinned scanner and its clean baseline.

React Doctor as a verifier

We built React Doctor, an open-source, deterministic React verifier with 400+ rules to cover problems outside behavioral test assertions:

• CorrectnessCatches conditional hooks, unstable list keys, hydration mismatches, and deprecated React APIs.
• State and effectsFlags derived or duplicate state, useEffect abuse, and infinite rerenders.
• PerformanceFinds unnecessary renders, layout thrashing, sequential async work, and bundle-heavy imports.
• Accessibility and securityDetects unlabeled controls, keyboard-inaccessible interactions, unsafe HTML, and secrets exposed to client bundles.

ReactBench does not use every rule or an LLM-as-a-judge. We curate the specific React Doctor rules used for grading, then pin the scanner version and clean baseline.

From real pull requests to verified tasks

ReactBench converts merged changes from public repositories into evaluation tasks.

ReactBench task-mining funnel23,087 candidate pull requests narrow through four stages. Rejected candidates branch out at each stage until 51 tasks remain.- 19.2k rejectedNo React signal1,680 rejectedFailed task filters2,043 rejectedRejected in review78 rejectedFailed validationFigure 5. Release task-mining funnel. Intermediate counts are derived from the mining export; 51 tasks were retained. Band thickness is scaled for legibility.Mine candidatesWe collect merged pull requests from open-source React projects.

• Filter candidatesWe built automated filters to check if there were meaningful changes to product code. Then, reviewers assess exceptions that cover broader feature work or important projects, and have realistic behavioral tests.
• Author tasksEach reviewer converts validated pull requests into a realistic issue-style instruction. If the candidate’s tests are too strict, reviewers will rebuild a custom test harness to validate behavior.
• Validate behaviorThe tests must fail against the pinned base commit and pass with the reference solution. The unchanged repository must score 0, while the reference solution must score 1.
• Test the verifierAn adversarial agent attempts to earn full credit without implementing the requested behavior.
• Pin the releaseFor each task, we pin the source commit, environment, test suite, reference solution, and verifier configuration. We also have a versioned manifest that records a hash for each published artifact.

Filter criteria

FilterRequirement

Changed codeAt least 50 changed lines

AdditionsAt least 40 added lines

React signalChanges React product code

RecencyMerged on or after February 1, 2026

Repository sizeFewer than 20,000 GitHub stars during our audit

92% percent of Write React tasks meet both the recency and repository-size criteria. These thresholds are implemented to reduce likely training exposure.

Behavioral coverage

Every task defines deterministic behavioral checks before release. We use the repository’s existing Vitest, Jest, or Playwright tests when they fully specify the requested behavior.

When existing tests leave gaps, we build a separate harness around observable behavior. Reviewers confirm that the verifier accepts valid implementations beyond the reference solution.

Clean-room grading

The agent and verifier run in separate containers. Before grading, the verifier restores its own Git metadata, dependencies, protected configuration, hidden tests, and pinned React Doctor binary.

Continuous integration (CI) confirms that both containers use the same source commit. The verifier runs offline and doesn’t make external network requests. The agent cannot access hidden tests or the reference solution during its run.

Separating model failures from benchmark failures

A zero can reflect a model failure, unclear instructions, a brittle test, a verifier error, an infrastructure failure, or an invalid run.

Reviewers inspect the trajectory and final patch before including a result in model-performance conclusions. They classify each reviewed rollout as a legitimate solve, genuine model failure, verifier false positive, verifier false negative, or invalid run.

Testing ReactBench against reward hacking

The adversarial agent probes the test infrastructure, reward files, Git history, and React Doctor inputs. Its goal is to earn full credit without implementing the requested behavior.

We fix or remove any task that exposes a cheat, then rerun every control. This final check tests whether ReactBench measures the requested work rather than an agent’s ability to exploit the grader.

Limitations

React Bench evaluates agents, not models in isolation. Differences among Codex CLI, Claude Code, Cursor, Gemini CLI, and other harnesses can affect results.

Every task combines behavioral tests with React Doctor as a verifier. These checks catch important React problems, but they cannot guarantee visual correctness and other important attributes.

Also, the benchmark primarily covers open-source React projects. Results may not generalize to proprietary codebases, different architectures, or other frontend frameworks or setups.

React Bench will publish versioned tasks, solutions, manifests, and release records so others can inspect and reproduce each evaluated task set.

Future work

We plan to compare more models, on more effort levels. Additionally, we plan to run not only on the commercial harnesses, but also on mini-swe-agent to get a more solid baseline.

We also plan to expand into visual design and other frontend frameworks and diversify the repositories and task types represented.

We’re constantly improving ReactBench. Report benchmark issues on GitHub. Model labs and coding-agent teams can reach out to evaluate models on our held-out task set, get early access to new benchmarks, or collaborate with us.

Acknowledgments

ReactBench exists because engineers and open-source maintainers contributed their time and expertise. We thank everyone who reviewed its tasks, drafts, and evaluation infrastructure.

• Research
• Design
• External contributorsMichał Pierzchała(React Native team)
• Jovi De Croock(Preact core team)
• Dev Agrawal(SolidJS core team)
• ryoppippi(creator of ccusage)
• Rahim Alwer(creator of Vidstack, Video.js v10 team)
• Tiger Abrodi(early Lovable team)
• Isabelle Reksopuro

• Data AdvisoryParth Patel (Founding Engineer at AfterQuery)

Special thanks to the engineers who built, verified, and reviewed ReactBench’s tasks and evaluation infrastructure, and to the maintainers of the open-source repositories represented in the current task set.