Blog

Is Mixture of Experts (MoE) Really More Energy Efficient? This post explores the energy efficiency claims around Mixture of Experts (MoE) models. While selectively activating parts of a model sounds like a power saver, the reality is more nuanced. We dig into experiments on smaller models to see when — and if — MoEs actually deliver efficiency gains. Spoiler: it depends on model size and how many experts you activate.

"Agent Bedlam" explores a future where autonomous AI agents multiply uncontrollably, triggering runaway energy consumption. Drawing parallels to email storms, the post introduces a custom Monte Carlo simulation to model energy impacts and defines a 'Bedlam Threshold'—a point where AI energy demand skyrockets. It’s a warning and a call for responsible growth in the age of intelligent automation.

Evaluating inference hosting configurations to optimize for power efficiency. Concurrent requests have significant impact on Tokens/Watt.

We began evaluating Deep Seek last week (like everyone else in the world) to assess its energy impact and optimization capabilities for our platform.