← The feed Many Minded · The brief
KNOWLEDGE · forward · impact 2/5 · 2026-11-16 · Marvell

Pooled photonic memory aims to ease the AI inference squeeze

Marvell and Penguin Solutions are building a shared-memory appliance linked by light, aiming to cut the memory cost of AI inference. Delivery is expected in 2027.

Marvell Technology and system integrator Penguin Solutions are working together on an appliance that pools memory over a photonic fabric so many servers can share it. Marvell supplies the memory modules, built on technology it bought from Celestial AI for $3.5 billion. Penguin builds the appliance itself, including power management, software and the management layer.

Each module carries two HBM3e stacks totaling 72 GB, plus support for eight DDR5 channels, on a silicon photonic interposer with 7.2 Tbps (900 GB/s) of bidirectional bandwidth. Penguin's first appliances take 16 modules, giving up to 16 TB of DDR5 and 1 TB of HBM, and later systems may double the DDR5 capacity. An appliance offers 57.6 Tbps of bandwidth through a Marvell-designed optical fiber shuffle and connects up to 16 servers through NICs, co-packaged optics or near-packaged optics. Penguin's software lets each server reach the pool and supports tiering modes, including one that uses HBM as a read/write cache in front of DDR5. CXL 3.1-capable NICs and host CPUs such as AMD Venice make this work today, and Marvell's Preet Virk says optics-based memory can bypass CXL.

The pitch is an alternative to offloading AI inference KV cache to storage. If memory can be shared, each server may need less DRAM of its own, which could lower the cost of inference capacity at a time when memory is a bottleneck. That bears on how cheaply AI-driven knowledge services can be delivered.

The caveats are real. Savings from pooling are possible, not demonstrated. The performance and utilization advantages are Marvell's own claims. Reported capacity figures are inconsistent across the coverage. Marvell expects Penguin's first-generation systems in the second half of 2027, and that is only an expectation.

Source: The Register