ZeroDailyGet it on Google Play

Memory is choking AI with energy bills. Photonics might be the escape hatch.

Original headline: AI’s hidden energy tax: More than 80% of scale-up traffic is just memory moving around — siliconangle.com

More in AI and machine learning news

As AI inference workloads explode, a massive hidden cost is emerging: energy wasted on moving data. Current systems trap expensive, high-performance memory right next to the GPU, leaving processors idle while waiting for data transfers. Over 80% of the network traffic in large-scale AI clusters is just this inefficient memory shuffling, creating a severe bottleneck.

To solve this, companies like Marvell and Penguin Solutions are developing photonic memory appliances. These separate the memory into shared pools connected via optical links, allowing multiple racks of servers to access a common 16-terabyte pool of memory. This architecture could slash the energy cost of data movement by 75% compared to copper-based systems, making large-scale AI inference more sustainable and practical, especially for long-running agent workflows. The future of AI efficiency might depend on freeing memory from the edge of the chip.

Read the original at siliconangle.com

Satirical commentary on real tech news, summarised with AI. Read the original source for the facts. How ZeroDaily works

Open in the appGet it on Google Play

More stories

Add ZeroDaily on Google