AI Center of Excellence: Models, Roles and a Charter

An AI center of excellence earns its place when it helps business teams deliver useful AI with clear ownership. For a company of 100 to 2,000 people, start with the decisions you need to coordinate; if those fit with two named owners, run an AI desk before creating a department.

The useful output is a working agreement: which problems get attention, which standards apply, who supplies common tools, and who remains responsible after launch. A new reporting line cannot answer those questions by itself.

Verified on 26 October 2026: the organizational guidance below was checked against Microsoft, IBM and NIST's current publications. The charter, staffing recommendation and 90-day sequence are proposed operating arrangements, not industry staffing benchmarks or promised delivery times.

What Does an AI Center of Excellence Do?

An AI center of excellence, or AI CoE, coordinates the expertise and shared services that help people use artificial intelligence across a company. It connects business priorities to delivery, evaluation, standards and training. Its authority comes from an executive-approved mandate, not the word “excellence.”

Think of a teaching conservatory. It supplies shared equipment, growing methods and expert help; the gardener responsible for each bed still has to tend it. Your CoE can supply a tested approach to support drafting, but the support leader still owns the accuracy of customer replies and the work of reviewing them.

Microsoft's CoE guidance includes intake, pilots, standards, skills, reusable assets and outcome reporting. For this audience, organize that work into four responsibilities.

Pick Use Cases With Business Owners

Maintain a prioritized list of proposed work. A use case is a specific task in a specific workflow, such as drafting a support reply from approved help articles. “Use AI in customer service” is too broad to assign or evaluate.

Require an owner, a current performance baseline and an observable result. Favor work with accessible data and outputs a person can check. A frequent task with a recoverable mistake is a better first candidate than an impressive demonstration with no path into daily operations.

Your AI strategy supplies the priorities. The CoE translates them into funded work and makes dependencies visible; it should not silently invent a different business strategy.

Set Standards People Can Apply

Define permitted data, access rules, review requirements, change records and the conditions for stopping a system. Turn those standards into practical defaults and short checklists.

For support drafting, the rule might be: use approved knowledge sources, show the source to the reviewer, and require a support agent to approve every outgoing reply. A general instruction to “use AI responsibly” does not tell that agent what to do.

Connect the technical rules to your AI acceptable use policy, which tells employees what they may do day to day.

Own Shared Tools and Evaluation

Give common capabilities an accountable owner: approved access, reusable integrations, a system inventory, example workflows and a way to run evaluations. An evaluation is a repeatable check of how the complete workflow performs on representative cases.

The CoE owns the shared method. The business owner defines what an acceptable result means for their task. A test set for support answers cannot certify a hiring workflow, even if both use the same underlying model.

Train Teams in Their Actual Work

Teach a support reviewer to spot unsupported claims, an engineer to test permission boundaries, and a manager to distinguish usage from useful results. Have people demonstrate the task and its failure response.

A prompt workshop can introduce a tool. It cannot establish that someone can safely operate a consequential workflow.

Choose Among Three Operating Models

Choose a model by where delivery and operating responsibility can reliably sit. Headcount alone is a poor rule: a large company can have scarce AI expertise, while a smaller software company can have capable product teams.

The labels overlap. Microsoft's operating-model guidance uses centralized, hybrid and federated, and also describes a central platform with federated delivery as hub-and-spoke. IBM's CoE explanation draws the distinctions somewhat differently. Use the definitions below when assigning work.

Central works best as a deliberate concentration of scarce expertise. The business owner still decides whether the result helps the workflow. Keep an intake queue visible so a small technical group does not become an invisible waiting room.

Hub-and-spoke is a useful default once several departments need the same foundations. In a hypothetical 700-person company, the center might maintain access and evaluation tooling while support owns response quality and finance owns its document workflow. Write down who fixes a failed integration and who pauses each service.

Federated requires local operating capability, not just local enthusiasm. For a hypothetical 1,800-person company, ask each participating unit to demonstrate its owner, testing records, monitoring and incident response before delegating routine release decisions. A department that cannot meet that bar stays on a more supported path.

Different workflows can use different models. Keep a sensitive shared service central while allowing a capable department to manage a bounded internal assistant. Delegation follows evidence about the team and task, not a company-wide graduation ceremony.

AI CoE vs AI Governance Committee

An AI CoE supplies delivery capability; an AI governance committee exercises the policy and risk authority delegated to it. They work together, and some people may sit in both, but their decisions should be distinguishable.

A committee might approve the company's risk tolerance, prohibited uses and exception process. The CoE turns those decisions into configurations, evaluation requirements, training and operating support. Your AI governance framework should make the relationship explicit.

Use this suggested division of authority:

The committee does not have to review every approved internal drafting workflow. Equally, the CoE should not accept a material risk merely because it built the system. Record who can approve an exception, its scope and when it expires.

NIST's AI RMF Core calls for documented responsibilities in GOVERN 2.1 and executive responsibility for AI risk decisions in GOVERN 2.3. Those outcomes matter more than whether you create a new committee. An existing leadership or risk forum can hold the authority if its remit is clear.

Roles and Decision Rights: Assign Work Before Hiring

Cover the responsibilities before creating job titles. Some require dedicated work; others need reliable access to an existing specialist. A list of roles is not a recommendation to hire a person for every row on an organization chart.

Executive sponsor: funds the agreed priorities, resolves conflicts between departments and appoints the decision authorities. A COO can sponsor operational change; a CTO can sponsor shared technical capability. Name one accountable sponsor even when both participate.

CoE lead: owns the mandate, intake process, shared backlog and reporting. This person must be able to defer proposals that lack an owner or readiness, within the authority granted by the charter.

Business workflow owner: supplies the baseline, defines acceptable results, assigns reviewers and owns the operating outcome. For support drafting, this is the leader accountable for support quality, not whoever knows the most about models.

Technical and evaluation owner: maintains shared integrations, access controls, repeatable tests and change records. Separate the ability to modify a system from approval of its business results where the stakes warrant independent review.

Data and risk partners: data owners decide whether information is suitable and permitted for the purpose. Security, privacy, legal and other relevant specialists assess matters within their remit. The CoE coordinates their input; it does not inherit their authority by default.

Enablement owner and departmental champions: turn approved workflows into training, collect problems from users and update guidance. Champions need protected time and a route to technical help, not an honorary title.

For every production workflow, also name the person who receives an incident, can stop the service and arranges recovery. AWS's CoE guidance includes formal incident responsibilities and response actions. Carry those into the operating agreement before the pilot becomes routine work.

When a Two-Person AI Desk Is Enough

Use an AI desk when the coordination work fits with two named working owners and existing specialist support. This is a proposed starting arrangement, not a universal minimum or a promise that two people can operate every AI system.

Assign an operations or product owner to intake, workflow design, training and measures. Assign a technical owner to approved access, integrations, evaluation and support. Give them an executive sponsor and access to existing data, security and legal owners. They may start with allocated time rather than new full-time posts, but their managers must protect that time.

For a hypothetical 150-person services company trying a supervised drafting workflow, this can be enough to establish common practices and learn whether a larger function is justified. The two owners coordinate the work; the relevant department still reviews the output.

Stay with the desk while:

• The active workload is small enough to support and monitor properly.
• Departments can use common tools without substantial custom engineering.
• Existing owners can answer data and risk questions promptly.
• Routine support and evaluations fit within the capacity explicitly allocated.

Expand when repeated cross-team dependencies, unresolved requests or operational support needs exceed that capacity.

Treat higher-consequence work as a separate capacity question. A small company building systems that affect people's rights or safety may need specialist oversight before it needs a broad enablement program.

One-Page AI CoE Charter Template

Start with this compact charter, then replace the bracketed fields with names and commitments. It is designed to fit one page; check the layout after adding your details. The same charter works for a two-person desk if you keep the scope narrow.

AI Center of Excellence Charter: [Company]

Sponsor and term. Sponsor: [name]. CoE lead: [name]. Effective: [date]. Review: [date, initially day 90]. Allocated people and working time: [commitment].

Mandate. Help [business functions] improve [named outcomes] through appropriate AI use. Prioritize use cases, maintain shared standards and tools, coordinate evaluation, and teach teams to operate approved workflows.

Scope. Cover [workflows, users and environments]. Start with [bounded pilot]. Exclude [prohibited uses and work outside this mandate]. Business owners remain accountable for their processes and results.

Decision rights. The sponsor approves priorities and capacity. The CoE lead manages intake and approves supported technical patterns within policy. The data owner authorizes each proposed data use. The named service owner authorizes release after required evidence and specialist reviews. [Risk authority] decides exceptions and their expiry. [Operator] may pause an unsafe or failing service; [owner] authorizes restart after review. No role may waive another owner's required approval.

Intake. Submit [form/location] with the workflow, business owner, baseline, desired outcome, users, data, proposed actions, foreseeable harm, evaluation method and support owner. Review weekly. Record proceed, prepare, escalate or stop, with a reason, responsible person and next date. Escalate unresolved priority conflicts to the sponsor and policy exceptions to the risk authority. Silence is not approval.

Shared services. Maintain approved access, the AI inventory, reusable patterns, evaluation methods, training and incident contacts at [location]. Each asset has an owner and review date. Workflow-specific acceptance criteria belong to its business owner.

Measures. Track accepted task outcomes, quality failures, intake decision time, reuse of maintained assets and demonstrated user competence. Record baseline, target, evidence source, reporting owner and stop condition for each pilot. Count review and rework in performance measures.

Cadence and handover. Hold weekly intake and monthly sponsor reviews. Before routine use, confirm operating ownership, monitoring, incident response and fallback. At the review date, expand, revise or retire this mandate based on evidence.

Keep detailed procedures outside the charter and link them. The charter should let someone settle an ownership dispute without reading a handbook.

Intake Should End With a Decision

Make intake a short route to an accountable next step. Collect enough information to decide what happens next, then request deeper evidence in proportion to the workflow's consequences.

Use the AI readiness assessment to examine the proposed task, its data and its people. A company-wide enthusiasm score cannot establish that a particular workflow is ready.

Consider a support manager who proposes drafting replies from approved help articles. The useful intake record names the manager, the source collection, the reviewer, the current handling process and the errors that would make the result unacceptable. It also specifies that the draft cannot send itself.

Route the proposal to prepare if nobody can own the source collection. Route it to escalate if the requested data use falls outside policy. Route it to a pilot only when its owner, permitted data, review process and evaluation are ready. Stop it if the expected benefit does not justify the work or the proposed use is prohibited.

Define the pilot's acceptance criteria before generating attractive examples. For support drafting, assess factual support, source relevance, inappropriate commitments, disclosure and the review effort needed to obtain an acceptable reply. Include awkward cases such as conflicting articles and questions the knowledge base cannot answer.

The CoE should make those checks repeatable. The support owner should judge the results against the actual service requirements. A technically successful generation can still fail the business test if reviewing it takes longer than doing the original task.

How to Set Up an AI Center of Excellence: The First 90 Days

Use the first 90 days to establish an operating agreement and gather evidence from one bounded pilot. Treat the dates as review points. A workflow that fails its acceptance criteria does not earn a production release because the calendar reached the final phase.

1. 1

   Days 1-30: Define the work and authority

   Name the sponsor and working owners. Inventory existing AI use, including AI features already embedded in business software. Identify duplicated work, unclear data permissions and unsupported services.

   Agree the charter and choose the operating model for the first workflow. Establish its baseline and complete the intake record. Publish approved-use guidance, the intake location and the incident contact.

   Exit evidence: a signed mandate, a usable inventory, a named workflow owner and an agreed evaluation plan. If data access remains unresolved, preparation is the next phase's work.

2. 2

   Days 31-60: Test a bounded workflow

   Run the pilot within the approved scope. Record the relevant system configuration, instructions and source material so you can explain what produced a result. Train the people who will review the output.

   Evaluate routine cases and failure cases against the agreed criteria. Record review time and rework alongside output quality. Rehearse pausing the workflow and returning to the existing process.

   Exit evidence: reviewed results, an issue log, trained operators and a workable fallback. Use the evidence practices in the NIST AI RMF guide to connect risks, tests and decisions.

3. 3

   Days 61-90: Decide and assign ongoing ownership

   Have the workflow owner and required reviewers decide whether to continue, change or stop. Record the reason. A stopped pilot with clear evidence is a useful decision.

   If the workflow proceeds, assign monitoring, support, change review and retirement responsibilities. Publish only the reusable parts with named maintainers. Decide whether the next request fits the AI desk or exposes a specific need for more CoE capacity.

   Exit evidence: a documented disposition, accepted operating ownership for anything continuing, and a prioritized next backlog. Expand the scope only when the evidence supports it.

Measure Work That Survives Review

Measure the outcome after review and correction. Counting generated drafts hides the work needed to make them usable.

For the support pilot, compare the time to an accepted reply with the baseline process, including review and rework. Check quality alongside speed so a faster workflow cannot appear successful by sending worse answers. Treat a small pilot as local evidence, not a company-wide productivity estimate.

Keep the CoE's measures separate from the workflow's measures:

• Decision time: median elapsed time from a complete intake to a recorded decision. Track missing-information delays separately.
• Quality: unacceptable outcomes and critical failures against criteria agreed before the pilot. Record severity, not only an average score.
• Reuse: workflows using a maintained shared asset, with its version and owner known.
• Competence: whether a user can perform the task, identify a failure and follow the escalation path.

Training attendance and active accounts can help diagnose adoption. They do not establish that the work improved. Avoid turning estimated time savings into financial benefits unless the business can show how that capacity was used.

Who Should Act Now, and Who Should Wait?

Act now when coordination problems are visible. Wait on a new department when the work still fits within existing ownership.

For COOs and operations leaders: sponsor the mandate when teams duplicate effort, proposals lack business owners or nobody can explain which pilots should continue. Your first decision is which outcome matters and who owns it.

For CTOs and builders: centralize common access, evaluation methods and operating support when maintaining them separately causes repeated problems. Delegate delivery only to teams able to run what they build.

For buyers and procurement leaders: use the CoE to clarify approved uses, technical requirements and operating ownership before expanding a commitment. Procurement authority and contract review remain with their existing owners unless formally reassigned.

For teams with a working approved process: keep using it. A CoE announcement does not require replacing every tool, reopening every decision or migrating a workflow that already meets the agreed standards.

What's Overhyped About an AI CoE?

A center does not create value simply by existing. Its value is the work it makes possible and the bad decisions it helps prevent.

A bigger org chart is not a maturity measure. You can have a substantial committee and no person responsible for restoring a failed workflow. Name the operator before adding another forum.

Approved tools do not approve every use. Permission to draft internal notes does not authorize the same system to send customer commitments. Approval needs a purpose, data scope and action boundary.

Shared evaluation does not produce universal assurance. The reusable part is the method and infrastructure. The domain-specific cases, harms and acceptance criteria still need the business owner's judgment.

A CoE need not become a permanent delivery monopoly. Move routine work outward as teams demonstrate competence, and keep central the services that benefit from common ownership. Review the mandate as seriously as you review a pilot.

Frequently Asked Questions

What is an AI center of excellence?

An AI center of excellence is a coordinated function that helps a company prioritize AI use cases, set standards, supply shared tools and evaluation, and train people. It may deliver systems centrally or support business teams that build and run their own.

What to not ask an AI?

Do not submit information prohibited by your company's data rules or ask a system to take actions beyond its approved scope. For workplace use, the CoE or AI desk should publish permitted uses and route uncertain cases to the relevant data or risk owner.

What are the potential risks of AI?

Depending on the workflow, risks include incorrect outputs, disclosure of sensitive information, unequal treatment and unauthorized actions. Identify the consequences for the particular task, then assign controls, a reviewer and a recovery path. A generic risk list cannot replace that assessment.

The Monday Move

Bring the sponsor, an operations owner and a technical owner together around one recurring workflow. Fill in the charter's mandate and decision rights, name the person accountable for the outcome, and write down what evidence would make you stop. If that group can manage the agreed scope, begin as an AI desk and let demonstrated workload justify expansion.

For practical guides to putting AI into business operations, join the newsletter.