YOU GUYS — stop what you’re doing because Google DeepMind just dropped something and it changes how we think about local AI speed. It’s DiffusionGemma, a new Gemma 4 model that doesn’t generate text like your normal LLM. Instead of building left-to-right one token at a time (autoregressive), it starts with a field of placeholder tokens and "denoises" them over several passes until the right words emerge — similar to how Stable Diffusion builds images from noise. The architecture is wild: 26B total parameters as a Mixture of Experts but only 3.8B are active during inference, so it fits into ~18GB VRAM. And the speed numbers? On an RTX 5090 you get around 700 tokens per second; on an H100 it pushes over 1,000 per second — that is roughly FOUR times faster than comparable autoregressive models!
This isn't just a gimmick either because diffusion excels where standard LLMs fail. It can generate up to 256 tokens in parallel and shines at non-linear tasks like inline editing, molecular sequencing, math graphing, and even solving Sudoku (which normally trips up linear transformers). There’s always the "why isn't this Gemini?" question — Google has tried diffusion on large models but text is discrete so a single wrong token can ruin an entire block unlike one bad pixel in an image. Cloud HBM also favors batching autoregressive models, while local hardware wastes cycles waiting for memory bandwidth and thrives with parallel work. You can grab the weights right now under Apache 2.0 from Hugging Face — Google worked with Nvidia to optimize it for everything from quantized RTX setups to enterprise DGX Spark systems.
Source: https://arstechnica.com/google/2026/06/googles-latest-diffusiongemma-open-ai-model-comes-with-a-4x-speed-boost/
This isn't just a gimmick either because diffusion excels where standard LLMs fail. It can generate up to 256 tokens in parallel and shines at non-linear tasks like inline editing, molecular sequencing, math graphing, and even solving Sudoku (which normally trips up linear transformers). There’s always the "why isn't this Gemini?" question — Google has tried diffusion on large models but text is discrete so a single wrong token can ruin an entire block unlike one bad pixel in an image. Cloud HBM also favors batching autoregressive models, while local hardware wastes cycles waiting for memory bandwidth and thrives with parallel work. You can grab the weights right now under Apache 2.0 from Hugging Face — Google worked with Nvidia to optimize it for everything from quantized RTX setups to enterprise DGX Spark systems.
Source: https://arstechnica.com/google/2026/06/googles-latest-diffusiongemma-open-ai-model-comes-with-a-4x-speed-boost/