5 stories tagged inference.
Apple's Neural Engine, born from its failed car project, became the backbone of on-device AI. The M7 Ultra with 1.5TB of RAM could put Apple in the server inference market by 2027.
Claude on GB300 is now generally available in Microsoft Foundry, giving Azure teams more inference headroom but a tighter cloud bet.
Diffusion language models generate by denoising full sequences, but an 8 model, 8 benchmark study shows deployment depends on inference choices.
AgentPerf is a benchmark for concurrent AI agents. Its first results put NVIDIA GB300 NVL72 at up to 20x Hopper efficiency.
A typical AI text prompt uses roughly 0.24 to 0.34 watt-hours and a fraction of a milliliter of water, not a bottle per email. Training is huge but rare, and inference is what adds up.