AI Research

Scaling Laws for Looped Mixture of Experts

Medium Severity Global
Date Occurred Sep 30, 2026 17:53 UTC
Event Type AI Research
Source arXiv
Recorded Oct 01, 2026
Full Description

arXiv: Scaling Laws for Looped Mixture of Experts Looped transformers and Mixture-of-Experts (MoE) offer complementary routes to efficient scaling: recurrence increases computational depth at fixed parameters, while MoE sparsity expands total capacity at fixed active compute. Yet existing scaling laws model recurrence or sparsity in isolation. In this work, we introduce Loop Scaling Laws, the first scaling law to jointly model recurrence and sparsity alongside model size and data. At its core is a bounded, sparsity-conditional recurrence mappin

AI Intelligence Layer

AI Categories

performance
Event Metadata
  • ID #35869
  • Type AI Research
  • Region Global
  • Severity Medium
  • Indexed Oct 01, 2026