マイクロソフト、AI エージェントの大量運用手法を公開

Debugging SSO, managing users, adjusting auth policies, configuring branding: every configuration task has lived behind a UI that only a human can drive.

The WorkOS MCP server gives agents the same access as your dashboard login. Hundreds of operations, discoverable at runtime. Connect in one command via OAuth, with scoped tokens instead of a master API key. Pass a screenshot of your marketing site and ask your agent to match the login page. If a human had to do it before, an agent can do it now.

Connect your agent →

Microsoft operates at an enormous scale. More than 80,000 enterprises now build on Microsoft Foundry, the company’s platform for building, deploying, and running AI agents and applications. Microsoft’s own copilots run on the same platform, including Microsoft 365 Copilot, which alone serves over 20 million users and has a monthly active usage of first-party agents growing 6x year-to-date.

To understand what it actually takes to ship agents at that scale, we spoke with Marco Casalaina, VP of Products for Microsoft Core AI. He walked us through what his team has learned from running these systems in production, the engineering challenges that come with it, and where he thinks enterprise AI is headed next.

In this article, you’ll learn:

• Why a prototype agent doesn’t survive in production
• What’s in a production agent harness, and why Microsoft believes context is the key
• Two engineering ideas behind Foundry: retrieval-as-a-subagent, and giving agents their own identity and a place to act
• How Microsoft evaluates production agents with rubric-based evals and an auto-improvement loop
• Lessons for other teams, and what’s next

Production agents fail for reasons that aren’t visible in a prototype. The model is rarely the problem. What breaks is everything around the model, including the data the agent retrieves, the tools it calls, the way it handles real users, and the way quality drifts as the world around it changes. Enterprises trying to ship agents this year are running into a different engineering problem than the one they were solving last year.

To see why, it helps to start with what’s actually changed about what enterprises are trying to build. Marco framed the shift this way:

We are leaving the question-answering phase of AI. In 2026, we are seeing a huge increase in the number of our customers that are using voice as a front-end, so we’re also leaving the chatbot era of AI.

The old shape was a chatbot. The user types, the agent types back, and it can only answer questions. The new shape is an agent that does meaningful work on the user’s behalf. It books the meeting, runs the analysis, sends the email, files the ticket. The user might not type at all because the front end can be voice. For example, Foundry’s Voice Live lets a team turn an existing text agent into a voice agent without rebuilding it.

This shift is what makes the engineering problem different. A chatbot returning a wrong answer is a bad experience. An agent taking the wrong action is a business incident. The bar for what’s good enough to ship has moved.

That’s where the gap between a prototype and a production agent opens up. The first prototype is easy. You can vibe-code one in an afternoon. The model is smart, your test prompts work, the demo is impressive, and the pilot ships in a week.

Production is where the cracks open. Real users ask things you didn’t anticipate. Documents the agent depends on go stale. New edge cases emerge that never appeared in your eval dataset. A model update changes the agent’s behavior subtly, and nobody notices until a customer complains. Without identity controls, the agent runs as a shared system principal and there’s no audit trail when something goes wrong. Without guardrails, it confidently says something it shouldn’t. Without observability, you can’t tell whether quality is improving or degrading. None of these problems showed up in the prototype. All of them show up in production.

When we asked Marco for the single biggest lesson the Foundry team has learned from running these systems at scale, his answer was “the harness matters as much as the model.”

The harness is everything around the model. The runtime, the tools, the context retrieval, the identity layer, the guardrails, the evaluators, the deployment pipeline. Models change constantly, and you cannot treat them like database versions. With Postgres, you change versions and you pretty much expect it to straight up work. Models are not like that. Each one has different properties that the harness has to adjust to. When Anthropic released Claude Opus 4.8, Microsoft’s GitHub Copilot CLI team had to re-tune their harness and re-run their evaluations before they could ship it.

AI is reshaping infrastructure, but not how most teams expected. The teams getting the most from it aren’t adopting the fastest. They’re taking advantage of the platform, governance, and automated pipelines that allow AI-generated infrastructure to move safely to production.

The 2026 Infrastructure Automation Report surveyed 406 infrastructure and platform engineering leaders to reveal how Pioneers are benefiting the most from AI. You’ll learn:

• Why platform engineering is the critical component of AI adoption
• New AI-specific signals and metrics that need monitoring beyond traditional metrics
• 5 tactical steps to reach the Pioneer segment of the Al Maturity index

Download the report

If the harness matters as much as the model, the next question is what’s in it. Walking the harness from the bottom up shows why each layer exists and what breaks if you try to do without it.

At the bottom is the inference layer, the single interface the harness uses to reach the models. The models themselves sit outside the harness and stay swappable. Different agents need different models, and the right one changes every few weeks. Foundry supports more than 11,000 of them, from OpenAI, Anthropic, xAI, DeepSeek, and Microsoft’s own MAI family.

Above the model is the agent runtime, which is what turns a model into an agent. The runtime handles the orchestration loop, the tool calls, the conversation state, and the protocol the rest of the harness speaks.

Not every step in that loop should run through the model. A well-built agent sends only the parts that need reasoning to the LLM and leaves the rest to ordinary code, since a database lookup or a purpose-built extraction model is faster, cheaper, and more reliable than asking a model to do the same work.

The number of agent frameworks keeps growing, from open-source options like LangChain, LangGraph, and CrewAI to vendor-built runtimes. The principle that matters is framework neutrality. An agent built on one framework should be portable to another without rewriting the surrounding harness. Foundry, for example, lets agents run on any framework interchangeably.

Once agents are in production, an organization needs observability and governance. It needs a single view of every agent running across every project, with health scoring, token usage, latency metrics, drift detection, and the kind of cross-project rollups that let a platform team govern a fleet. Without this layer, regressions are invisible and cost is uncontrolled. In Microsoft’s case, this is Foundry Control Plane, which provides cross-project fleet visibility and routes agent telemetry into Azure Monitor and Application Insights, the same pipeline that already handles infrastructure alerts.

Once agents start taking real actions inside an organization, they need their own identities. They need their own role assignments and their own audit trails, because a misbehaving agent has to be bounded by the same access controls that bound a misbehaving employee. The industry is still converging on how to do this well, and the answer most platforms are landing on is to extend an existing enterprise identity system to treat agents as a new class of principal rather than inventing a parallel system. For example, Foundry extends Entra, Microsoft’s enterprise identity platform, to treat agents this way. This is the access-control primitive; later in the article, the section on giving agents a place to act shows what an agent does once it has one of these identities.