← All papers
How Modular Is a Frontier Mixture-of-Experts? A Pre-registered Causal Test in Which Apparent Expert Modularity Mostly Dissolves
A pre-registered causal test of whether the experts in a frontier MoE (Command A+, 218B total / 25B active, 128 experts) form functional modules tied to capabilities or languages. Of six pre-registered expert families ablated at inference time against a size-matched random-expert null, only one — the Arabic-language family — is a clean selective module that survives an independent corpus and a conservative statistical bar; every other family has a real causal effect but its apparent modularity flips with the corpus, the metric, or the threshold. A positive control on Qwen3-30B-A3B recovers its known disjoint structure, and the verdict reproduces on the un-quantized BF16 model: robust expert modularity is rare and measurement-dependent.