我们设计了 GPT‑5.6 模型系列，旨在平衡用户使用我们模型完成各类任务时的能力与成本。我们的旗舰模型 GPT‑5.6 Sol（最大推理模式）在人工智能分析编码智能体指数上超越了 Claude Fable 5，而成本不到后者的一半。Terra 在智能基准测试中表现与 GPT‑5.5 相当，但价格仅为一半；Luna 是我们最快且最实惠的模型，定价比 Sol 低 80%。为了实现这些效率提升，我们的研究和技术团队在技术栈的每个主要层面都进行了重大优化。这些改进涵盖了我们的模型、推理（我们运行模型以生成输出的方式）以及智能体框架（Codex 和 ChatGPT Work 均使用该框架）。
过去四年间，随着我们的模型扩展到 10 亿活跃用户和超过 200 万家企业，效率一直是让智能惠及所有人的核心。我们的使命是确保通用人工智能造福全人类。这些年来，我们不断在技术栈中解锁更大的优化空间，以便在成本-智能曲线上每个点都提供性能最优的模型。通过 GPT‑5.6，我们实现了迄今为止最高的每 token 智能效率，该模型经过训练可在每个 token 上完成更多工作。在训练中，我们同时优化任务成功率和效率，引导模型在完成任务时走更直接的路径。
本文不仅关注模型本身，还将分享我们如何通过技术栈中另外两个主要部分的进步来设计效率，包括：1）推理，通过优化负载均衡、推测解码、缓存和内核优化等流程，从相同硬件中获得更多输出；2）我们的智能体框架，包括更好地管理上下文膨胀、工具使用和重复工作。我们还将介绍 GPT‑5.6 Sol 在自主实现其中多项增益方面的作用。虽然任何孤立的改进可能看似有限，但这些成果叠加起来，使我们能够在智能和效率的前沿同时取得突破。
使用 GPT‑5.6 Sol 加速推理
在计算资源受限、模型需求增长快于容量增长的世界中，效率是每个系统设计的核心。这一点在我们的推理栈中尤为突出，推理栈负责运行训练好的模型以生成响应。我们的主要目标是用相同的硬件提供更多 token，同时保持用户期望的智能、延迟、可用性和可靠性。
实现这一目标需要优化整个系统。一个模型单独来看可能非常高效，但如果请求分配不当、硬件闲置或数据移动拖慢计算速度，其服务成本仍然可能很高。每一层的改进都会叠加，收益来自路由（请求发送到哪里）、调度（请求何时发送）、内核（在 GPU 上运行的软件）、缓存（保存并重复使用的工作）和模型实现（GPU 代码的执行顺序）等方面的优化。Codex 中的 GPT‑5.6 Sol 在这些优化中发挥了关键作用。
第一个重要例子是负载均衡。在全球范围内，我们根据地理位置、可用容量和加速器类型（运行模型的 GPU 或专用芯片类型）等因素路由请求。在集群内部，我们根据负载、上下文长度、缓存可用性和其他请求属性，将工作分配到各个模型实例。在每个实例内部，工作必须高效地分配到加速器、模型的子网络和计算核心。Codex 中的 GPT‑5.6 Sol 帮助我们分析生产流量，识别之前被忽视的不平衡来源，测试新的路由策略，并不断调整这些启发式规则。仅这些负载均衡改进就显著降低了模型服务成本。
我们还使用 GPT‑5.6 Sol 优化了模型的前向传播：将输入转换为下一个 token 预测的计算过程。即使单个操作很快，过多的内存移动、同步和低效的数据布局也可能导致 GPU 闲置。为避免这种情况，GPT‑5.6 Sol 找到了可以预计算、避免或并行化的工作。借助 Codex，GPT‑5.6 Sol 自主重写并优化了我们的生产内核，即执行构成模型的数学运算的核心代码。这在一定程度上是因为我们训练了 GPT‑5.6，使其能够高效地编写和优化 Triton（在新窗口中打开） 和 Gluon（在新窗口中打开） 中的内核，这两种开源 GPU 编程语言由 OpenAI 维护。这些努力，加上 GPT‑5.6 Sol 带来的更广泛内核改进，将端到端服务成本降低了 20%。我们还大力投资了验证工具，例如开源工具 FpSan（在新窗口中打开）（浮点清理器），以帮助验证 GPT‑5.6 Sol 编写的内核的正确性。
推测解码是提高速度和效率的另一个杠杆。该技术涉及在主模型旁边运行一个较小的草稿（或“推测器”）模型，提出多个 token 供主模型并行验证。当这些提议被接受时，系统可以通过一次主模型前向传播生成多个输出 token，从而减少昂贵的顺序计算量。GPT‑5.6 Sol 通过在其架构上设计并运行数百次实验，测试大小、结构和特征的变化，改进了自身的草稿模型。此外，GPT‑5.6 Sol 启动并监控了推测器训练过程，在出现问题时（包括硬件故障和训练不稳定）自主干预。由此产生的改进使 token 生成效率提高了 15% 以上。
在处理未缓存的输入 token 时，模型通过一次计算密集型的前向传播构建键值缓存；在生成输出时，它会反复读取并扩展该缓存。服务的最佳配置（如批处理、分片和键值管理）在很大程度上取决于工作负载——提示和输出长度、批大小、缓存命中率、查询特征等。然而，配置空间以前太大，无法系统性地调整，迫使工程师依赖宽泛的启发式规则。借助 Codex 中的 GPT‑5.6 Sol，我们能够分析生产工作负载，生成并评估候选配置，并针对每种场景超优化引擎和模型的配置方式。这使得一种新的工作负载特定优化成为可能，从而从相同硬件中提取更多有用的推理结果。
推理优化是一个持续的反馈循环。我们测量生产行为，识别最大差距，实施变更，并验证它们能改善整个系统而非孤立基准。GPT‑5.6 Sol 和 Codex 加速了这一循环的每个环节。这意味着我们的团队可以探索更多想法，更快响应变化的工作负载，并构建一个延迟更低、容量更大、成本更低的推理栈。
我们的智能体框架如何简化重复工作
ChatGPT Work 和 Codex 通过一系列模型请求和工具调用来完成复杂任务。在单次交互中——从用户请求到最终响应——Codex 可能检查源代码、搜索部署历史、阅读事件报告、编辑文件并运行测试。每个步骤都可能需要一次请求。
准备上下文、传输数据、运行推理、调用工具和启动进程都需要时间和计算资源。如果一个任务需要30次模型请求，每次请求多花一秒就会累积起来。提升整体性能意味着减少系统中重复的工作，而不仅仅是让模型更快。
一次用户交互可能包含多次模型和工具迭代。重复区域内的任何成本都可能被多次支付。
这些乘数效应指导了我们如何设计智能体框架，这是一个用 Rust 编写的编排层，连接我们的模型、工具和用户环境。接下来，我们将介绍如何通过避免上下文膨胀、加载工具和复用工作来提高每次请求的效率。
避免上下文膨胀
随着智能体被授予更多工具、技能、插件和对话历史，上下文窗口很容易扩大。这会增加成本，分散模型注意力，并引发不必要的推理。该框架可以通过延迟发现来减少这种开销，使集成、自定义 MCP 工具、技能和插件仅在需要时才可访问。该框架还能防止单个工具和 MCP 集成意外消耗上下文窗口。默认情况下，工具输出限制为10,000个 token，除非模型请求不同的限制。
保留精确前缀以支持提示缓存
如前所述，智能体循环可能在单次交互中多次向 GPU 发送相同的指令、对话历史、工具定义和先前结果。处理这些重复输入成本高昂，因此提示缓存会复用与先前处理过的提示前缀相关的计算。为了保留该前缀，框架将所有模型可见的历史记录视为仅追加：新消息、工具结果和环境更新被添加到末尾，而不是插入到较早的上下文中。工具也以确定性顺序呈现，而运行时设置（如审批策略）在执行期间应用，而非嵌入工具定义中。这一设计选择有助于 Codex 和 ChatGPT Work 实现较高的整体提示缓存命中率。
增量传输改变了网络传输的内容；提示缓存改变了模型可能避免重新计算的内容。宽度为概念性示意，未显示额外的压缩层。
智能曲线上的效率提升
我们通过 GPT‑5.6 实现的效率提升是多年来跨栈（涵盖研究、推理和智能体框架）复合改进的结果。GPT‑5.6 在实现这些改进中的角色让我们对优化速度的加速感到乐观。我们将继续在核优化等领域进行更大改进，同时对我们栈进行基础性提升。我们期待将这些持续进行的底层改进以更广泛可用、更具成本效益的智能形式回馈给用户和客户。
特别感谢技术团队成员 Matthew Ferrari、Philippe Tillet、Ahmed Ibrahim、Joe Gershenson 和 Steve Coffey 对本文的贡献。
We designed the GPT‑5.6 model family to balance capability and costacross the spectrum of tasks people use our m odels for. Our flagship model, GPT‑5.6 Sol, with max reasoning outperforms Claude Fable 5 on the Artificial Analysis Coding Agent Index at less than half of the cost.Terra performs as well as GPT‑5.5 on intelligence benchmarks at half the price, and Luna is our fastest and most affordable model, priced 80% less than the cost of Sol. To deliver these efficiencies, our research and technical teams have made significant optimizations at every major layer of our stack. These improvements span our models, inference (how we run models to generate output), and our agentic harness, which is used by both Codex and ChatGPT Work.
As we’ve scaled our models to 1 billion active users and more than 2 million businesses over the past four years, efficiency has been central to distributing the benefits of intelligence to everyone. Our mission is to ensure that artificial general intelligence benefits all of humanity. Over these years, we’ve worked to continuously unlock greater optimizations across our stack in order to offer the most performant models at every point in the cost-intelligence curve. We achieved our greatest intelligence-per-token efficiency yet through GPT‑5.6, which is trained to achieve more work per token. In training, we optimize for both task success and efficiency, shaping the model to take a more direct path through a task.
This post looks beyond our models to share how we designed for efficiency through advancements in two other major parts of the stack, including 1) inference, by optimizing processes such as load balancing, speculative decoding, caching, and kernel optimization, to get more output from the same hardware, and 2) our agentic harness, including better managing context bloat, tool usage, and repeated work. We’ll also share the role of GPT‑5.6 Sol in landing several of these gains autonomously. While any isolated improvement may seem limited, these wins compound to allow us to deliver on the frontier of both intelligence and efficiency.
Accelerating inference with GPT‑5.6 Sol
In a compute-constrained world where model demand is growing faster than capacity, efficiency is core to every system design. That’s especially true in our inference stack, which runs trained models to generate responses. Our primary objective is to serve more tokens with the same hardware, while preserving the intelligence, latency, availability, and reliability us ers expect.
Achieving this requires optimizing the entire system. A model can be highly efficient in isolation, but still be expensive to serve if requests are distributed poorly, hardware sits idle, or data movement slows down computation. Improvements at every layer compound, with gains coming from optimizations in routing (where requests are sent), scheduling (when requests are sent), kernels (software that runs on GPUs), caching (saved and reused work), and model implementation (the ordering of GPU code). GPT‑5.6 Sol in Codex played an instrumental role in all of these optimizations.
The first important example is load balancing. Globally, we route requests based on factors such as geography, available capacity, and accelerator type (the type of GPU or specialized chip running the model). Within a cluster, we distribute work across model instances based on load, context length, cache availability, and other request properties. Within each instance, work must then be partitioned efficiently across accelerators, the model’s sub-networks, and computing cores. GPT‑5.6 Sol in Codex helps us analyze production traffic, identify previously overlooked sources of imbalance, test new routing strategies, and constantly tune these heuristics. These load balancing improvements alone dramatically reduced the cost of serving our models.
We also used GPT‑5.6 Sol to optimize the model’s forward pass: the computation that transforms inputs into next-token predictions. Even when individual operations are fast, excess memory movement, synchronization, and inefficient data layouts can leave GPUs idle. To avoid this, GPT‑5.6 Sol found work that could be precomputed, avoided, or parallelized. With Codex, GPT‑5.6 Sol autonomously rewrote and optimized our production kernels, the core code that executes the mathematical operations that make up the model. This worked in part because we’ve trained GPT‑5.6 to be effective at writing and improving kernels in Triton(opens in a new window) and Gluon(opens in a new window), two open-source GPU programming languages maintained by OpenAI. These efforts, combined with broader kernel advancements from GPT‑5.6 Sol, reduced end-to-end serving costs by 20%. We’ve also heavily invested in verification tooling, such as the open-source tool FpSan(opens in a new window) (Floating-Point Sanitizer), to help validate the correctness of the kernels written by GPT‑5.6 Sol.
Speculative decoding is another lever for improving speed and efficiency. The technique involves running a smaller draft (or “speculator”) model alongside the primary model, proposing several tokens for the primary model to verify in parallel. When those proposals are accepted, the system can produce multiple output tokens from a single primary-model pass, reducing the amount of expensive sequential computation. GPT‑5.6 Sol improved its own draft model by designing and running hundreds of experiments on its architecture, testing changes in size, structure, and features. Additionally, GPT‑5.6 Sol launched and monitored the speculator training process, autonomously intervening when issues arose, including hardware failures and training instability. The resulting improvements increased token-generation efficiency by more than 15%.
When processing uncached input tokens, the model builds the key-value (KV) cache in one compute-intensive pass; when generating output, it repeatedly reads from and extends that cache. The optimal configuration for serving, such as batching, sharding, and KV management, depends heavily on the workload—prompt and output length, batch size, cache hit rate, query characteristics, and more. However, the configuration space was previously too large to tune systematically, forcing engineers to rely on broad heuristics. With GPT‑5.6 Sol in Codex, we were able to analyze production workloads, generate and evaluate candidate configurations, and hyper-optimize how the engine and model are configured for each scenario. This makes a new level of workload-specific optimization practical, extracting more useful inference from the same hardware.
Inference optimization is a continuous feedback loop. We measure production behavior, identify the largest gaps, implement changes, and verify that they improve the whole system rather than an isolated benchmark. GPT‑5.6 Sol and Codex accelerate every part of that loop. This means our team can explore more ideas, respond faster to changing workloads, and create an inference stack with lower latency, more capacity, and lower costs for users.
How our agentic harness streamlines repeated work
ChatGPT Work and Codex complete complex tasks through a series of model requests and tool calls. In a single turn—from the user’s request to the final response—Codex might inspect source code, search deployment history, read incident reports, edit a file, and run tests. Each step can require a request.
Preparing context, transmitting data, running inference, calling tools, and starting processes all take time and compute. If a task requires 30 model requests, an extra second per request adds up. Improving overall performance means reducing repeated work throughout the system, not just making the model faster.
One user turn can contain many model and tool iterations. Any cost inside the repeated region can be paid many times.
These multipliers have informed how we designed our agentic harness, which is a Rust orchestration layer connecting our models, tools, and the user’s environment. Next, we’ll cover how avoiding context bloat, loading tools, and reusing work make each request more efficient.
Avoid context bloat
As agents are granted access to more tools, skills, plugins, and conversation history, context windows can easily expand. This increases cost, distracts the model, and prompts unnecessary reasoning. The harness can reduce this overhead through deferred discovery, which makes integrations, custom MCP tools, skills, and plugins only surfaceable when needed. The harness also prevents individual tools and MCP integrations from unexpectedly consuming the context window. Tool output is capped at 10,000 tokens by default unless the model requests a different limit.
Preserve exact prefixes for prompt caching
As previously mentioned, an agent loop can send the same instructions, conversation history, tool definitions, and earlier results to the GPUs multiple times within a single turn. Processing these repeated inputs is expensive, so prompt caching reuses the computation associated with a previously processed prompt prefix. To preserve that prefix, the harness treats all model-visible history as append-only: new messages, tool results, and environment updates are added at the end rather than inserted into earlier context. Tools are also presented in a deterministic order, while runtime settings, such as approval policies, are applied during execution instead of being embedded in tool definitions. This design choice contributes to Codex’s and ChatGPT Work’s high overall prompt-cache hit rates.
Incremental transport changes what crosses the network; prompt caching changes what the model may avoid recomputing. Widths are conceptual, and the additional compression layer is not shown.
Efficiency across the intelligence curve
The efficiency gains we delivered with GPT‑5.6 are the result of years of compounding improvements across the stack, spanning research, inference, and our agentic harness. The role of GPT‑5.6 in delivering many of these improvements makes us optimistic about how the pace of optimizations will accelerate. We’ll continue making greater optimizations in areas such as kernel optimization, alongside foundational improvements to our stack. We look forward to transferring these ongoing, under-the-hood improvements back to our users and customers in the form of more widely available, cost-efficient intelligence.
Special thanks to Matthew Ferrari, Philippe Tillet, Ahmed Ibrahim, Joe Gershenson, and Steve Coffey, Members of Technical Staff, for their contributions to this post.
本文内容采集自官方网站，排版和翻译可能与原页面存在差异。
阅读官方全文