AI製品に共通するモデル選択の課題と限界について

Open almost any AI product and you will find the same dropdown in the corner. Before you can get anything done, you have a job: pick a model. Maybe decide how hard it should think too.

The choice matters. One model may be great at tracing a difficult bug and strangely bad at design. Another can make beautiful interfaces but lose the thread on a long build. Then a new model ships, the rankings move again, or prices change.

Calling one model “the best” assumes the frontier has a crown. It doesn’t. If choosing the model changes whether your application works, you should not have to guess correctly before the work even begins.

At Lovable, that is what model independence means. We don’t treat models as interchangeable. We are model-independent because we are deeply opinionated about them.

Model-independent, not model-indifferent

Model indifference means putting every model behind the same interface, giving it the same instructions, and swapping one name for another. Real model independence means learning how each model works best.

We have a team at Lovable dedicated to this work. For each model, they shape the instructions, tools, and project context around what it does best. They study where it gets stuck, whether that’s in a long debugging loop, while configuring the backend, while polishing the interface, or somewhere else entirely. Then they test the whole setup across complete builds.

In one of those evals, for example, a frontier model finished tasks 15% faster than its predecessor, took 40% fewer turns, and scored 2–3% higher. Those were meaningful gains and made the new model the stronger option at the time.

We can invest heavily in making a model work well inside Lovable and still move on when something better arrives. Portability gives us that freedom without throwing away what we’ve learned about how differently models behave.

The control plane is the product

“Add payments to my app” sounds like one instruction. Inside Lovable, that request sets an app-building agent in motion. The agent carries the build from your request to a working change and reacts to whatever it finds along the way.

It has to understand what you meant, find the relevant parts of the project, plan the change, write the code, run the app, and see whether it works. If something breaks, the right recovery path is not always obvious. Should the system retry, change the plan, use another tool, bring in a model that reasons differently, or ask you a question?

The answer depends on what the build uncovers along the way. A simple request may reveal an architectural issue. Or the first attempt may show that the model understood the goal but struggled with one of the tools.

That is why the control plane watches the work as it unfolds: what you are trying to do, how difficult the job has become, and whether the agent is making progress or starting to circle. The control plane can then assign different parts of the build to different models instead of asking one model to own the whole thing.

The control plane also adapts the system around the model. In one eval, a model kept re-reading files that were already in context and checking them again after successful edits. We gave it different instructions: trust the context, trust the edit result, and skip the redundant tool calls. We may also change the tools, how we explain them, how much project context the model sees, and how a long build gets summarized before the next turn.

This isn’t model roulette. We do not send the same prompt through five models and pick our favorite response. We give each model the instructions, tools, and context that fit it, then judge whether the app gets better.

Independence does not mean constant switching

Changing models mid-build can make things worse because a long-running project develops history. The agent has explored files, tried approaches, hit errors, and learned what matters. The model is only one part of that agent. When the model changes, we may have to compress that history and rebuild its cached context. Some details from the conversation may survive only as a summary.

That means a model might be better in isolation and still be the wrong choice for the build in front of it. Our control plane has to weigh the possible gain against the context we may lose and the work we may repeat. The goal is not to switch models as often as possible. A useful switch has to be worth the cost. Otherwise, we are just creating model churn.

Deciding whether to switch starts with knowing why the system failed. Sometimes the app-building agent tells us directly through a vent tool, surfacing problems that would otherwise be hard to see.

If a model provider is overloaded, Lovable can send the next call to the same model through another provider. That can fix an availability problem, but not a bad approach. If the context is wrong, a tool is confusing, or the model misunderstood the job, switching providers gives up the cache without addressing the failure. A retry with user feedback may help. Otherwise, a different model can try another approach.

Recovery should change the thing that failed, whether that is the context, plan, tool, or model.

The application is the benchmark

A model can produce an impressive answer and still leave you with a broken app. It can write convincing code that never runs or create a finished-looking checkout page that saves the wrong data. On a benchmark, the response may look excellent. In the browser, it may be another story.

So the unit of optimization at Lovable is the finished application. We care about the whole trajectory: what the system tried, where it recovered, how long the build took, what it cost, and whether the application at the end does what you asked.

That changes what “fast” and “cheap” mean. A model can respond quickly and still be slow if it takes three times as many rounds to finish. A cheap call gets expensive when it sends the build down the wrong path. Even a brilliant plan only matters if the system can turn it into working software.

Public leaderboards can point us toward promising models, but they cannot tell us whether those models will produce better apps inside Lovable. We test each candidate inside Lovable using our prompts, tools, context, and agent loop. We run every build more than once because one beautiful result can be luck. A model that performs well starts with a little work and gets more only if it keeps delivering.

The tests have to earn our trust too. We look for agreement between human judgment, our LLM judges, and the model rankings we would expect based on outside evidence. When those signals disagree, we inspect the builds and find out why. In one bake-off, that led us to recalibrate a judge that ranked a hollow build near the top and drop another that gave nearly identical builds opposite scores.

Models that consistently produce better apps earn more work, with human judgment as the final check.

Sometimes the right model is ours

There is a limit to how much improvement comes from adding another paragraph to a prompt. Exceptions pile up, and one instruction fixes a failure only to distract the model from something else. Eventually, another prompt edit stops helping. Some jobs happen often enough that a specialist makes more sense.

That is where we started training models of our own. Routing requests, summarizing responses, and writing commit messages gave us clearly defined jobs on which to learn how to train, evaluate, and ship models.

Now our post-trained models are handling a meaningful share of app-building work in production. We will be training the next ones to handle harder problems and more of each build.

Our own models still go through the same control plane as outside models. If an external model becomes better at a job, it can replace one of ours. Even models we train ourselves have to earn their traffic. Our ambition is to build the best models in the world for the work Lovable understands best.

What we learn carries forward

Any company can buy access to a frontier model. The hard part is building a system that can turn its uneven, fast-changing strengths into software people can depend on. Every failure helps improve it. Each one becomes a test that tells us whether the prompt, tool, context, orchestration, evaluator, or model needs to change. We rerun the build, then feed what we learn back into the control plane.

That learning carries forward when a new model is released. Lovable puts it through a system that already knows what good looks like and can find where its capabilities fit.

A model company can improve its model. Lovable can improve the whole system.

The whole frontier should work for you

Foundation models will keep getting better. They just will not get better at the same things, at the same speed, or in the order anyone predicts.

When a model leaps ahead at reasoning, speed, or a repeated task, Lovable should put it to work where it helps. When it falls behind, Lovable should move on.

The model picker is a dead end because it freezes the decision before the system has enough information about the problem. Lovable should keep watching the work, use the model the job calls for, and change course when the evidence changes.

You tell Lovable what you want to build. The frontier can keep moving underneath you. Our job is to keep up so you can build software people love.