Microsoft builds synthetic enterprise tenants to evaluate Copilot without using customer data

Microsoft has built a platform that generates full-scale synthetic enterprise environments to evaluate Microsoft Copilot, avoiding the use of real customer data. The company calls it the Industry Evaluation Data Generation Platform, and says it establishes the first standardized evaluation baseline for Copilot.

The problem Microsoft is solving

According to Microsoft, evaluating enterprise AI requires test data that captures reporting structures, project histories, communication patterns and domain-specific workflows. Real enterprise data contains sensitive information that cannot be used for evaluation under any circumstances. Microsoft says it replaced small, manually curated test fixtures with generated environments to resolve that tension.

Three pillars

The platform rests on three components:

• Synthetic enterprise environments: simulated Copilot environments with realistic users, meetings, emails, documents and organizational structures, built without customer data.
• Automated query generation pipelines: rapid, reproducible creation of evaluation datasets.
• Structured quality frameworks: checks that synthetic data meets the bar for meaningful evaluation.

Each simulated workspace is called a "tenant" in Microsoft terminology: an isolated organizational environment with its own users, data and configurations. Microsoft describes the environments as scalable baselines that teams can build on.

Grounding in public data

The environments draw on three publicly available sources, which Microsoft says were reviewed and approved by its legal and policy organization:

• O*NET (U.S. Department of Labor occupational database): makes employee roles and responsibilities realistic, calibrates the mix and frequency of work activities by role, and tailors workflows such as clinical trial reviews or portfolio updates.
• Online job postings: supply role titles, reporting hierarchies and departmental compositions.
• SEC filings: inform organizational scale, business unit structures and financial contexts.

The environments are not static. They simulate time-evolving organizations, with meeting histories, project timelines, email threads that develop over weeks, and organizational changes. Microsoft says this matters because many Copilot queries are time aware, such as "What did the team decide last Thursday?" or "What's changed since the last quarterly review?"

Three industry environments

Microsoft generated three synthetic environments:

1. Pharmaceutical: regulatory submission workflows, clinical trial discussions and cross-functional safety review meetings.
2. Capital markets: deal pipelines, compliance communications and client portfolio reviews.
3. IT/software company: sprint planning, code review discussions and product launch coordination.

Microsoft says this specificity lets evaluation queries reflect what real users in those domains ask, rather than generic test cases that may miss domain-specific failure modes.

Six-stage query pipeline

Microsoft says writing thousands of grounded, industry-specific queries by hand is prohibitively expensive and slow. Its automated pipeline starts with a large library of common workplace tasks and narrows them to those relevant to a given industry. It then adapts them with domain-specific language and context, uses AI to generate queries grounded in the synthetic environment's data, and rewrites them to sound more natural and human. Each query is validated for accuracy and uniqueness.

Quality and safety controls

Microsoft assesses environment quality with an internal semantic evaluation framework that measures three dimensions. The first is structural fidelity: whether the organizational structure reflects realistic reporting hierarchies. On safety and privacy, Microsoft says every environment is fully synthetic and automated content-safety checks keep generated data free of harmful or personally identifiable material.

What this means

The approach addresses a constraint any enterprise assistant faces: the most representative test data is the data that cannot be used. By grounding synthetic tenants in public sources and adding a time dimension, Microsoft aims to test time-aware and domain-specific queries that generic benchmarks do not cover, according to its own description. The work covers only three industries so far, and Microsoft has not published benchmark scores or Copilot results in the material above, so the platform's value depends on what evaluations it produces. Its role as a baseline for Copilot is Microsoft's own claim.