Mixture of Experts Explained: Active vs Total Parameters

A mixture of experts model can do less work per token while remaining enormous to host. Moonshot AI's Kimi K3.1 Pro language model has 2.8T total parameters but 104B active per token, according to its official launch post. The smaller number helps explain computation; the larger number still matters when you have to hold the weights.

Parameter counts and source claims verified on 24 October 2026. The active-share calculations below are ours; they are not measured speedups, memory savings or quality scores.

What Is Mixture of Experts?

Mixture of experts, or MoE, gives a model many internal processing networks while selecting only some of them for each input. A parameter is a learned numerical value in the model. The stored parameters are commonly called its weights. A token is a small unit of input or output, often a word fragment.

Think of a consultation building with many rooms. A receptionist sends each visitor to the rooms needed for that visit. Fewer rooms do work for that visitor, but the building still contains every room. In an MoE layer, the router plays the receptionist and the experts are the rooms.

The analogy has a limit: these are learned numerical networks, not a roster of human job titles. You cannot assume there is a legal expert, a coding expert and an accounting expert waiting to be selected.

How Does Mixture of Experts Work?

The router selects paths inside the model, then combines their results. A transformer processes contextual relationships between tokens. In common MoE transformers, multiple expert networks replace a feed-forward block. A feed-forward block transforms a token's internal numerical representation; attention is the separate mechanism that lets it use surrounding context.

The flow is:

1. A token's current representation reaches an MoE layer.
2. A learned router, also called a gate, scores the experts.
3. It selects a subset, often the highest-scoring experts. This is top-k routing, where k is the number selected.
4. The selected experts process that representation. Their outputs are weighted and combined before processing continues.

This happens at the model's MoE layers, so different tokens and layers can select different experts. PyTorch's architecture explanation describes the weighted combination; Mistral's model paper shows how choices vary across layers.

Sparse Mixture of Experts

Sparse means that only a subset of experts runs for a given token. That is the form relevant to the total-versus-active counts in modern model cards. The remaining experts may run for other tokens, including tokens being processed in the same batch.

A dense feed-forward block applies its full network to each token. Sparse expert layers offer more stored capacity without making every token traverse every expert. Shared components, including attention in these examples, still do work.

A Mixture of Experts Model Is Still One Model

MoE routing is different from choosing between API models or asking several chatbots to vote. The router and experts are trained as components of one network. Your application normally sends a request to that network; it does not select an expert as if selecting a product from a menu.

For a builder, this distinction prevents a misleading design assumption: choosing an MoE model does not give your application a controllable team of specialist agents.

Mixture of Experts Models: Read the Two Counts Together

Read total parameters as stored model size and active parameters as the reported per-token computation path. Neither is a complete hardware specification. B means billion; T means trillion.

The last column is active divided by total, rounded. It describes each maker's published counts, not a cross-model efficiency ranking. Architectures and parameter-accounting conventions can differ.

Kimi K3.1 Pro: A Small Fraction of a Very Large Model

Kimi K3.1 Pro illustrates why “active” cannot be used as the size of the model you need to host. Converting Moonshot's total to the same units gives 2,800B. Then 104 ÷ 2,800 × 100 = 3.7%, rounded.

That means the reported active count is a small fraction of the total. It does not mean a serving installation can discard the other weights, or that the API runs at that fraction of a dense model's latency. The route changes as tokens and layers change.

Moonshot's launch blog describes Pro as a post-trained K3 and uses the rounded label “3T-class”. Its official launch post gives the exact 2.8T / 104B pair. Use that explicit specification when interpreting the model's size.

For an API builder, the model is a candidate to evaluate. For a hosting operator, availability is a separate gate: on the verification date, Moonshot promised full weights by November 4, 2026. A promised release is not yet a deployment artifact. The Kimi K3.1 Pro explainer covers the release-specific decision.

Qwen3-235B-A22B: The Active Count Does Not Replace the Total

Qwen3-235B-A22B makes the distinction visible in its name. Qwen's original model card states 235B total and 22B activated, with 128 experts and 8 activated experts.

The parameter calculation is 22 ÷ 235 × 100 = 9.4%, rounded. For a private document assistant, that is a clue about how the model limits per-token computation. Your memory plan must still accommodate the complete chosen checkpoint and the state of the requests it serves.

Do not turn “22B active” into “fits like a dense 22B model.” Also keep the checkpoint identity attached to the numbers: a family name alone is insufficient when choosing weight files, precision and serving configuration.

Mixtral 8x7B: Why Multiplying the Name Gives the Wrong Answer

Mixtral 8x7B contains shared components, so its name is not a multiplication instruction. Mistral's launch blog states 46.7B total and 12.9B used per token. Multiplying the numbers in the name treats shared components as though they belonged to separate complete models.

Mistral's paper describes expert feed-forward networks alongside shared attention. At each layer, the router chooses 2 of 8 experts. Those are not eight independent, complete models.

Using the published parameter pair gives 12.9 ÷ 46.7 × 100 = 27.6%, rounded. Selecting 2 of 8 experts is 25% of the experts, a different denominator. Shared parts are one reason the expert fraction and parameter fraction differ.

For deployment planning, use the card's total and the actual checkpoint files. Neither a product name nor a selected-expert count tells you the full weight footprint.

Why Less Computation Does Not Guarantee Lower Latency

Sparse routing reduces expert computation relative to executing a comparably large dense network; it does not promise the response time of a smaller dense model. That is how MoE can make large-capacity models cheaper per token to serve, while leaving substantial memory and communication work.

Arithmetic is only part of inference, the process of running a trained model. The server also reads weights, runs shared components, routes token representations, and sometimes transfers them between devices. A lower active count does not make those operations disappear. Shazeer and colleagues identify small expert batches and network bandwidth as performance constraints in the original sparse MoE paper.

Throughput and latency answer different business questions. Throughput is how much work the service completes over time. Latency is how long an individual request waits. A nightly document-processing queue may benefit from a configuration that handles large batches efficiently; an interactive support assistant needs the first useful answer promptly.

Batching, processing multiple tokens or requests together, can improve hardware utilization. But different tokens can select different experts, so a batch may use much more of the model than one token does. The active count is not the number of weights the server touches across an entire batch.

Prompt processing and token generation also stress the system differently. Prefill processes the input; decode generates subsequent tokens. vLLM, software for serving language models, documents separate expert-communication options for high-throughput prefill and low-latency decode. Our LLM inference guide explains these stages and the measurements to separate.

For API selection, compare observed behavior on your requests. A parameter pair cannot tell you how heavily a provider batches traffic, how long its queue is, or how well its hardware suits the model.

The Trade-Offs Your Business Feels

MoE moves part of the infrastructure burden from repeated computation to holding and coordinating a larger model. The operational question is whether that exchange works for your workload.

Memory: Every Expert Needs Somewhere to Live

Inactive for this token does not mean removable from the deployment. In normal resident serving, the full weight set stays loaded across the serving system, even though each token uses a subset. PyTorch's explanation makes this memory requirement explicit.

A useful starting calculation is weight storage ≈ total parameters × bytes per stored parameter. It is only a starting point: precision can vary across tensors, and file metadata, padding, replicas and runtime allocations add to the footprint.

The weights are not the whole memory budget. The KV cache stores attention keys and values from previous tokens so they can be reused. A document assistant with longer conversations or more simultaneous requests has additional request-state memory to accommodate. See KV cache explained for that separate constraint.

Quantization, storing weights at reduced numerical precision, can reduce the weight footprint. It does not turn total parameters into active parameters or remove the need to validate output quality. The LLM quantization guide covers the format and precision decision; the mechanism is documented in Hugging Face's quantization overview.

Throughput: Holding the Model Is Only the First Gate

A model that loads successfully can still miss your service target. Expert parallelism distributes experts across devices, letting the deployment hold a model larger than one device's memory. It also requires moving token representations to the devices that host the chosen experts and combining the results.

Those transfers make interconnects, the links between devices, part of the serving design. Capacity spread across poorly connected machines is not equivalent to the same capacity on a well-connected system. This follows directly from the communication pattern described in PyTorch's expert-parallel implementation.

For a low-traffic internal assistant, the practical concern is fixed capacity sitting underused. For a busy batch service, the concern may be keeping expert computation supplied with work. Neither outcome is determined by the active count alone.

Routing Quirks: Popular Experts Can Become Bottlenecks

A learned router does not guarantee perfectly even hardware use. NVIDIA's Mixtral routing analysis found that traffic concentrated on a topic could produce uneven expert loads. Some workers finish early while others remain busy.

This is why a mixed demonstration workload is insufficient for a service that mainly processes contracts or code. Evaluate that concentration of requests, including the slow requests, rather than relying only on an average.

Serving software can compensate, but compensation has its own footprint. vLLM documents expert load balancing with redundant experts, and warns that those extra copies consume GPU memory that might otherwise hold KV cache. Model weights fitting exactly into memory can leave too little room for the configuration that serves them well.

Choosing a Mixture of Experts LLM: API or Self-Hosting

API builders should judge the delivered service; hosting operators must also judge the complete deployment. Both need acceptable outputs, but only one of them directly manages where the experts live.

For builders using an API, the architecture is useful context, not a reason to switch. Evaluate representative tasks, instruction following, tool use where relevant, time to first token and completion time at expected concurrency. Record failed attempts and retries. The provider's serving efficiency does not dictate your bill or the amount of work needed to obtain an acceptable result.

For operators self-hosting, start with the exact downloadable checkpoint, its license and a supported serving implementation. Then evaluate weight precision, full-model memory, cache headroom, device connections and performance under load. Our open weight models guide separates access to the files from the rights and responsibilities of operating them.

For buyers responsible for a service commitment, ask whether the candidate finishes accepted work within the response-time target on the planned infrastructure. “Trillions total, billions active” is an architecture description, not a service-level guarantee.

1. 1

   Define the workload before comparing models

   Use representative requests with clear acceptance criteria. Separate interactive work from queued work, and include the long inputs and concentrated topics your application really handles.

2. 2

   Evaluate the route you intend to operate

   For an API, test the intended provider and endpoint. For self-hosting, record the exact checkpoint, precision, runtime and hardware. A different serving route is a different operational result.

3. 3

   Test capacity and quality together

   Increase concurrent work while checking memory use, waiting time, output quality and failures. A fast isolated response does not establish useful throughput, and high throughput does not establish acceptable interactive latency.

4. 4

   Switch only when the workload earns it

   Adopt the candidate when accepted work and operational requirements improve together. Keep the current model when those gains do not appear. Wait on self-hosting when the required weight files or supported deployment path are still unavailable.

A builder whose current API meets quality and response-time requirements is unaffected by the architecture label. A hosting operator who sized a deployment from active parameters should revise that plan now.

What Is Overhyped About MoE?

The overclaim is that sparsity gives you all the benefits of a huge model with all the operating characteristics of a small one. It gives you a useful separation between capacity and computation, with conditions attached.

Experts are not guaranteed domain specialists. Mistral's analysis of Mixtral found routing patterns that do not cleanly separate academic subjects. A learned expert is not a credentialed specialist, and router selection is not an explanation of why an answer is correct.

More total parameters do not prove better answers. Nor does a smaller active fraction prove a better model. Training, architecture and the task all matter. The table above explains reported sizes; it is deliberately not a quality leaderboard.

A smaller active fraction is not a matching discount or speedup. It says nothing by itself about provider charges, queueing, network transfers or how many generated tokens your task needs.

The Mixture of Experts Paper Behind the Idea

Shazeer et al.'s 2017 paper formalized an influential sparsely gated MoE layer that increased model capacity without a proportional increase in computation. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer is the primary reference for that mechanism. Its discussion of load balancing and distributed performance is also a reminder that the operational trade-offs were part of the idea from the outset.

MoE Questions Builders Ask

What is the mixture of experts in LLM?

It is a way to build a large language model with multiple internal expert networks and a learned router. In sparse MoE, each token uses only selected experts at each MoE layer. The application still calls one model.

Is GPT-4 a mixture of experts?

The GPT-4 technical report withholds further architecture details, including model size. It does not establish a maker-verified MoE parameter breakdown, so rumored counts should not be used as deployment facts.

Mixture of experts finetuning

Fine-tuning adapts a model's learned behavior; it does not remove the hosting requirements of its full weights. Evaluate the adapted model's task quality and routing behavior under representative traffic. Hugging Face's MoE guide discusses the different overfitting and balancing concerns involved.

Your Next Deployment Decision

On Monday, attach a serving note to each shortlisted model. Record its maker-stated total and active counts, the source, and the exact checkpoint or API endpoint. For hosted access, add your task results and latency measurements. For self-hosting, add the full weight footprint, cache allowance and performance under expected traffic. That note is more useful than ranking candidates by their largest advertised number.

For practical explanations of the infrastructure behind AI products, join the newsletter.