2026年6月10日
|
6分钟阅读
我们最新的开放实验模型在专用GPU上推理速度提升高达4倍，为探索速度关键型交互式本地工作流打开了大门。
Brendan O'Donoghue
研究科学家
Sebastian Flennerhag
研究科学家
今天，我们推出DiffusionGemma，一个探索文本扩散的实验性开放模型，这是一种异常快速的文本生成方法。该模型采用Apache 2.0许可发布，是一个260亿参数的混合专家（MoE）模型，超越了典型自回归大语言模型（LLM）逐token顺序处理的方式。相反，它同时生成整个文本块，在GPU上实现高达4倍的文本生成速度。
基于我们Gemma 4系列行业领先的每参数智能水平以及前沿的Gemini扩散研究，DiffusionGemma集成了一个旨在最大化生成速度的新型扩散头。虽然自回归的Gemma 4模型仍是高质量生产输出的标准，但DiffusionGemma专为探索速度关键型交互式本地工作流的研究人员和开发者设计，例如内联编辑、快速迭代以及生成非线性文本结构。
为开发者解锁新价值
构建实时交互式AI应用的开发者常常面临本地推理的延迟瓶颈。DiffusionGemma直接应对这些挑战，并伴随一些关键权衡：
- 极速推理： 通过将解码瓶颈从内存带宽转移到计算，DiffusionGemma在专用GPU上生成token的速度提升高达4倍。（在单个NVIDIA H100上每秒超过1000个token，在NVIDIA GeForce RTX 5090上每秒超过700个token）。1
- 可访问的硬件占用： 作为一个总参数量260亿的混合专家（MoE）模型，推理时仅激活38亿参数，DiffusionGemma在量化后能轻松适配高端专用消费级GPU的18GB显存限制。
- 双向注意力： 每次前向传播并行生成256个token，使每个token都能关注其他所有token。这为非线性领域（如内联编辑、代码填充、氨基酸序列或数学图）提供了显著优势。
- 智能自我修正： 模型迭代优化自身输出，使其能一次性评估整个文本块，实时修正错误。
- 实验状态与生产建议： 由于优先考虑速度和并行布局生成，DiffusionGemma的整体输出质量低于标准Gemma 4。对于追求最高质量的应用，我们建议部署标准Gemma 4。
你可以通过微调来提升DiffusionGemma在特定任务上的性能。在下面的示例中，Unsloth微调了DiffusionGemma来玩数独——这是自回归模型难以胜任的任务，因为每个token都依赖于未来的token。DiffusionGemma的双向注意力使这变得容易得多。
微调后的DiffusionGemma解决数独问题。
为何对文本采用扩散？
尽管AI研究社区多年来一直在探索基于扩散的文本生成，但将其应用于大模型仍是一个挑战。DiffusionGemma通过改变模型使用硬件的方式改变了这一局面。
传统模型的权衡
大多数语言模型像打字机一样，从左到右逐token生成。在云端，这很高效，因为服务器可以将数千个用户请求批量处理，共享硬件负载。但当在本地为单个用户运行时，这种逐词生成过程会使你的专用GPU或TPU利用率不足——大部分时间只是在等待下一次“按键”。
DiffusionGemma扭转了这种低效。它不按顺序预测单词，而是同时草拟整个256个token的段落。通过一次性给计算机处理器分配更大的工作块，DiffusionGemma充分发挥了硬件的潜力。它将模型推理从单个顺序的打字机升级为一台巨大的印刷机，同时印出整个文本块。
Hugging Face的DiffusionGemma文本转3D SVG演示。逐步生成过程。
这意味着DiffusionGemma的速度提升专为本地和低并发推理设计。在高QPS的云服务中，自回归模型可以通过部署有效饱和计算资源，因此DiffusionGemma的并行解码带来的收益递减，并可能导致更高的服务成本。在单个加速器上的低到中等批量大小下，吞吐量优势最为显著。
文本扩散的工作原理
类似于AI图像生成器从视觉静态开始并迭代优化为清晰图像，DiffusionGemma将其应用于文本：
- 画布： 模型从随机占位符token的画布开始。
- 迭代优化： 模型进行多次传递，锁定正确的token，并将其作为上下文线索来优化其余部分。
- 最终润色： 文本收敛为高质量输出。
由于模型在生成过程中可以处理整个段落，它解锁了新的模型行为模式，例如完美闭合复杂的Markdown格式，或近乎实时地生成和渲染代码。
立即开始
- 下载权重： 立即在 Hugging Face 上获取实验性模型权重（基于宽松的 Apache 2.0 许可证发布）。
- 集成与学习： 查阅我们的 DiffusionGemma 开发者指南了解更多信息。或深入阅读 DiffusionGemma 可视化指南，了解其底层机制。
- 使用你喜爱的开发工具： 通过 MLX、vLLM（由 Red Hat 支持集成）和 Hugging Face Transformers 高效部署模型。为快速实验，我们发布了使用 Hackable Diffusion（一个专为可组合性设计的模块化 JAX 工具箱）的微调教程。你还可以通过 Unsloth 和 NVIDIA NeMo 探索微调。此外，对 llama.cpp 的官方支持即将推出。
- 体验优化性能： 我们与 NVIDIA 合作，在其硬件栈上进行了优化，确保与消费级配置（针对 GeForce RTX 5090 和 4090 GPU 量化）的兼容性，同时在企业级系统（使用高级 NVFP4 内核的 Hopper 和 Blackwell）上实现高性能，包括用于本地桌面部署的 NVIDIA DGX Spark 和 DGX Station，以及面向 AI 专业人士的 RTX PRO。对 NVFP4（4 位浮点数）的原生支持加速了计算吞吐量，使模型能够以更快速度运行，同时保持近乎无损的精度。
- 按需尝试： 在桌面专用 GPU 上运行，或通过 Gemini Enterprise Agent Platform Model Garden 或 NVIDIA NIM 在云端运行。
Jun 10, 2026
|
6 min read
Our newest open experimental model delivers up to 4x faster inference on dedicated GPUs and opens the door to exploring speed-critical, interactive local workflows.
Brendan O'Donoghue
Research Scientist
Sebastian Flennerhag
Research Scientist
Today, we’re introducing DiffusionGemma, an experimental open model that explores text diffusion, an exceptionally fast approach to text generation. Released under an Apache 2.0 license, this 26B Mixture of Experts (MoE) model moves beyond the sequential token-by-token processing of typical autoregressive Large Language Models (LLMs). Instead, it generates entire blocks of text simultaneously, delivering up to 4x faster text generation on GPUs.
Built upon the industry-leading intelligence-per-parameter of our Gemma 4 family and cutting-edge Gemini Diffusion research, DiffusionGemma integrates a novel diffusion head designed to maximize generation speed. While autoregressive Gemma 4 models remain the standard for high-quality production outputs, DiffusionGemma is designed for researchers and developers exploring speed-critical, interactive local workflows such as in-line editing, rapid iteration, and generating non-linear text structures.
Unlocking new value for developers
Developers building real-time interactive AI applications often struggle with the latency bottlenecks of local inference. DiffusionGemma addresses these challenges directly, with some key trade-offs:
- Blazing fast inference: By shifting the decode bottleneck from memory-bandwidth to compute, DiffusionGemma generates up to 4x faster token output on dedicated GPUs. (1000+ tokens per second on a single NVIDIA H100, 700+ tokens per second on NVIDIA GeForce RTX 5090). 1
- Accessible hardware footprint: Operating as a 26B total Mixture of Experts (MoE) model that activates only 3.8B parameters during inference, DiffusionGemma fits comfortably within 18GB VRAM limits of high-end dedicated consumer GPUs when quantized.
- Bi-directional attention: Generating 256 tokens in parallel with each forward pass allows every token to attend to all others. This provides significant advantages for non-linear domains such as in-line editing, code infilling, amino acid sequences or mathematical graphs.
- Intelligent self-correction: The model iteratively refines its own output, allowing it to evaluate the entire text block at once to fix mistakes in real-time.
- Experimental status & production recommendations: Because it prioritizes speed and parallel layout generation, DiffusionGemma’s overall output quality is lower than standard Gemma 4. For applications that demand maximum quality, we recommend deploying standard Gemma 4.
You can improve DiffusionGemma's performance on specific tasks through fine-tuning. In the example below, Unsloth fine-tuned DiffusionGemma to play Sudoku — a task autoregressive models struggle with because each token depends on future tokens. DiffusionGemma's bi-directional attention makes this much easier.
Fine-tuned DiffusionGemma solving Sudoku.
Why diffusion for text?
While the AI research community has explored diffusion-based text generation for years, applying it to large models has remained a challenge. DiffusionGemma changes this by shifting how models use hardware.
The trade-off with traditional models
Most language models act like a typewriter, generating one token at a time from left to right. In the cloud, this is efficient because servers can batch thousands of user requests together to share the hardware load. But when run locally for a single user, this word-by-word process leaves your dedicated GPU or TPU underutilized — it spends most of its time simply waiting for the next "keystroke."
DiffusionGemma reverses this inefficiency. Instead of predicting words sequentially, it drafts an entire 256-token paragraph simultaneously. By giving the computer's processor a larger chunk of work at once, DiffusionGemma utilizes your hardware to its full potential. It upgrades your model inference from a single, sequential typewriter to a massive printing press that stamps the entire block of text simultaneously.
DiffusionGemma text-to-3D SVG demo by Hugging Face. Step-by-step generation.
This means DiffusionGemma's speedup is designed for local and low-concurrency inference. In high-QPS cloud serving, autoregressive models can be deployed to saturate compute efficiently, so DiffusionGemma's parallel decoding offers diminishing returns and can result in higher serving costs. The throughput advantage is strongest at low-to-medium batch sizes on a single accelerator.
How text diffusion works
Similar to AI image generators that start with visual static and iteratively refine it into a clear picture, DiffusionGemma applies this to text:
- The canvas: The model starts with a canvas of random placeholder tokens.
- Iterative refinement: The model makes multiple passes, locking in correct tokens and using them as context clues to refine the rest.
- Final polish: The text converges into high-quality output.
Because the model can process the whole paragraph while generating, it unlocks new patterns of model behavior, like perfectly closing complex markdown formatting or generating and rendering code in near real-time.
Get started today
- Download the weights: Access the experimental model weights (released under a permissive Apache 2.0 license) right now on Hugging Face.
- Integrate & learn: Learn more in our DiffusionGemma developer guide. Or deep dive into A Visual Guide to DiffusionGemma to understand the mechanics under the hood.
- Use your favorite development tools: Serve the model efficiently using MLX, vLLM (with integration supported by Red Hat), and Hugging Face Transformers. For rapid experimentation, we are releasing a fine-tuning tutorial using Hackable Diffusion, a modular JAX toolbox designed for composability. You can also explore fine-tuning with Unsloth and NVIDIA NeMo. Additionally, official support for llama.cpp is arriving soon.
- Experience optimized performance: We worked with NVIDIA to optimize across their hardware stack, ensuring compatibility with consumer setups (quantized for GeForce RTX 5090 and 4090 GPUs) alongside high performance on enterprise systems (Hopper and Blackwell using advanced NVFP4 kernels), including NVIDIA DGX Spark and DGX Station for local deskside deployment, and RTX PRO for AI professionals. Native support for NVFP4 (4-bit floating-point) accelerates compute throughput, allowing the model to run at faster speeds with near-lossless accuracy.
- Try your way: Run on your desktop dedicated GPU or in the cloud through Gemini Enterprise Agent Platform Model Garden or NVIDIA NIM.
本文内容采集自官方网站，排版和翻译可能与原页面存在差异。
阅读官方全文