Olmo 3 - The Architecture of "Fully" Open Sourced Reasoning Models

State-of-the-Art Open AI Model

Peeking Under the Hood of AI

Every week, it seems another new AI model is announced, smashing benchmarks and promising incredible new capabilities. We see the impressive performance charts, but the model itself often remains a black box. The data it was trained on, the code that built it, and the specific recipes used are kept behind closed doors, making it impossible for the wider research community to truly understand, reproduce, or build upon the work.

AI2's Olmo 3 is a rare and powerful exception. It’s a "fully-open" model, which means its creators released not just the final model weights, but the entire ecosystem used to build it: the training data, the code, the evaluation tools, and every checkpoint along the way.

This radical transparency gives us a unique opportunity to peek under the hood and learn what it really takes to build a powerful language model. The lessons learned are often fascinatingly counter-intuitive. Here are five of the most surprising takeaways from the Olmo 3 technical report.

Great AI Isn't Just About Good Data, It's About the Gap Between Good and Bad.

To align a model with human preferences, a standard technique like Direct Preference Optimization (DPO) relies on a simple principle: show the model a "chosen" (good) response and a "rejected" (bad) one, and it learns to prefer the former. The common method, popularized by datasets like UltraFeedback, is to generate these pairs from a diverse pool of models. The more varied the models, the richer the preference data.

But what happens when such a pool doesn't exist? In the specialized domain of open models that show their reasoning steps, the Olmo team faced this exact problem. Lacking a diverse set of open-reasoning models, they couldn't generate the necessary high-quality preference pairs. A straightforward attempt to simply train on good responses from a single better model (Qwen-3-32B) actually made their model worse.

Their ingenious solution was the "Delta Learning" hypothesis. Instead of seeking two slightly different high-quality responses, they created a massive quality chasm. They paired a "chosen" response from the powerful Qwen-3-32B with a "rejected" response from the much weaker Qwen-3-0.6B. The results were dramatic. This strategy transformed a scarcity of data into a powerful learning signal, proving that the model learns most effectively from clear, unambiguous contrast, not just absolute quality.

“The intuition behind delta learning is that the quality of preference data depends primarily on the quality of the delta between chosen and rejected responses; the quality of either response individually is less important.”

Crafting a 'Data Diet' for an LLM is a Rigorous Science.

In the high-stakes world of LLM training, where a single run can cost millions, creating the pretraining dataset isn't just about vacuuming up the internet. The creators of Olmo 3 treated it as a scientific optimization problem to de-risk their investment.

They pioneered a "swarm optimization" approach. First, they trained dozens of small, inexpensive proxy models on different data mixtures. The results from these tiny models were then used to train predictive models—specifically, generalized linear models—that could forecast how a given data mix would perform at full scale. This allowed them to scientifically engineer the optimal data diet before committing to the final, massive training run.

Their scientific rigor didn't stop there. Instead of just filtering out bad data, they implemented "quality-aware upsampling." This meant the absolute best data—the top 1%—was intentionally repeated up to seven times in the training corpus. This lesson is clear: building frontier models requires a level of data curation that resembles a sophisticated resource allocation problem, using predictive modeling to ensure every training dollar is spent on the most impactful data possible.

“We found that quality-aware upsampling improves performance in data-constrained settings… We achieved better results by upsampling the highest-quality data: including multiple copies of the top 5% and single copies of the remaining data to reach the target token count.”

'Frankensteining' Models Together Works Surprisingly Well.

One of the most unintuitive techniques in the Olmo 3 toolkit is "model merging"—the seemingly strange but effective process of averaging the weights of multiple different models to create a superior hybrid.

The Olmo team used this as a clever engineering shortcut to save immense amounts of compute. During pretraining, accurately evaluating a checkpoint typically requires an expensive process called "learning rate annealing." The team discovered they could get a remarkably accurate simulation by simply merging four recent checkpoints, a trick that delivered vital insights at a fraction of the cost.

This "Frankensteining" approach wasn't a one-off. For their midtraining stage, they improved performance by merging two models that were trained independently with different random seeds. For long-context training, they merged three adjacent checkpoints from the end of a single run. These practical, cost-saving tricks challenge the intuition of a model as a single, monolithic artifact, revealing it as a more malleable object that can be cleverly combined and improved.

“We demonstrate that merging checkpoints trained with constant learning rates not only achieves significant performance improvements but also enables accurate prediction of annealing behavior. These improvements lead to both more efficient model development and significantly lower training costs.”

The Biggest Bottleneck in Advanced AI Training Isn't What You Think.

For the final, crucial stage of training a reasoning model, developers turn to Reinforcement Learning (RL). One might assume the computational heavy lifting is the "learning" itself—the policy updates that adjust the model's billions of parameters.

The Olmo 3 report reveals the surprising reality: the true bottleneck was inference. The process of generating responses (the "rollouts") to learn from was vastly more expensive than the learning. The numbers are staggering: inference consumed 5 to 14 times more compute than the training updates. For the 32B model, the powerful learner GPUs spent 75% of their time idle, simply waiting for the inference GPUs to produce data.

To overcome this, the team engineered OlmoRL, a sophisticated system built for efficiency. Its key components—a fully-asynchronous architecture that prevents waiting, continuous batching to eliminate GPU idle time, and inflight updates that refresh the inference model's weights without pausing generation—slashed the total RL training time from over 15 days to just 6. This highlights that frontier AI progress is often gated by gritty engineering challenges, where overcoming practical bottlenecks is as important as algorithmic breakthroughs.

The problem with existing RL-Zero experiements use modesl with no data transparancy. This makes it impossible to rule out data contamination when unexpected results occur.A fully open RL-Zero setup combining the transparent Olmo 3 Base model with the new decontaminated Dolci RL-Zero dataet allows researchers to confidently draw conclusions from RLVR experiments and investigate phenomena without the data leakage. 

True Openness Finally Allows Us to Bust AI 'Myths'.

AI research occasionally produces bizarre findings that stump the community. One such mystery emerged when researchers found that Reinforcement Learning with Verifiable Rewards (RLVR) could improve a model's performance even when the rewards were completely random. This effect, observed primarily in the Qwen-2.5 model series, sparked debate. Was it data contamination—evaluation data leaking into the pretraining set? Or was it something more complex, like "entropy collapse" during RL, where the process unintentionally reinforces a model's useful latent behaviors (like generating code for math problems)?

With closed models, these hypotheses were untestable. The Olmo 3 team, however, could create the ultimate "clean room" environment. Using their fully decontaminated dataset and transparent training process, known as the RL-Zero setup, they re-ran the experiment. Their setup controlled for both data contamination and applied modern algorithmic fixes like "clip higher" to prevent entropy collapse.

The result was definitive: in their clean environment, the performance benefit from random rewards completely disappeared. This masterclass in scientific validation demonstrates the immense value of fully-open models. They allow researchers to untangle complex, interacting phenomena and build a more robust, evidence-based understanding of how these systems truly work.

“[A lack of data transparency] can lead to a myriad of issues with benchmark evaluations being contaminated; e.g. midtraining data containing the evaluation which makes spurious rewards as effective as true reward or improvements from fixing prompt templates outweighing the improvements from RL.”

The Dawn of an Open Renaissance

The journey to build Olmo 3 reveals a consistent theme: creating a state-of-the-art language model is a deeply scientific and clever engineering endeavor, filled with non-obvious challenges and surprising solutions. We only know these fascinating details because of AI2's commitment to full transparency.

Ultimately, the value of Olmo 3 isn't just the final model. It’s the entire ecosystem it provides—the tools, data, and reproducible research artifacts that serve as "a starting point for any aspect of open LLM research." Building such a foundation from scratch would "require millions of dollars in experiments," but now it is a shared resource for the entire community.

This begs a final, exciting question: as more researchers build upon these open foundations, what other long-held assumptions in AI are about to be overturned?