Less is More: Recursive Reasoning with Tiny Networks

Ghe research paper "Less is More: Recursive Reasoning with Tiny Networks," which introduces the Tiny Recursive Model (TRM). TRM is a novel, highly efficient approach to complex reasoning that significantly outperforms most Large Language Models (LLMs) on difficult puzzle benchmarks. The model, which consists of a single, 2-layer network with only 7 million parameters, achieves superior generalization and accuracy on tasks like ARC-AGI compared to both a prior recursive model (Hierarchical Reasoning Model, or HRM) and established LLMs such as Gemini 2.5 Pro. The core takeaway is that architectural innovation, specifically recursive reasoning with small networks, can be a more effective and efficient solution for hard reasoning problems than the prevailing strategy of scaling model size, demonstrating that "less is more."

Analysis of Recursive Reasoning Models

The research positions recursive reasoning with small networks as a powerful alternative to the massive scale of contemporary LLMs for solving specific, challenging problems.

Hierarchical Reasoning Model (HRM)

The paper first introduces the Hierarchical Reasoning Model (HRM) as a novel, biologically inspired approach that established the potential of this methodology.

• Architecture: HRM utilizes two small neural networks that recurse at different frequencies.
• Scale: The model was trained with 27 million parameters on a small dataset of approximately 1000 examples.
• Performance: HRM was shown to outperform LLMs on hard puzzle tasks, including Sudoku, Maze, and the Abstraction and Reasoning Corpus (ARC-AGI).
• Limitations: Despite its promise for solving hard problems with small networks, HRM is described as "not yet well understood and may be suboptimal."

Tiny Recursive Model (TRM)

The paper proposes the Tiny Recursive Model (TRM) as a superior and much simpler alternative to HRM.

• Architecture: TRM employs a single tiny network with only 2 layers.
• Simplicity: It is presented as a "much simpler recursive reasoning approach" compared to the dual-network HRM.
• Efficiency: The entire model contains only 7 million parameters.
• Generalization: TRM is stated to achieve "significantly higher generalization than HRM."

Performance Benchmarking and Key Results

TRM's performance demonstrates a substantial leap in capability and efficiency, particularly when benchmarked against much larger models on the ARC-AGI intelligence tests.

Comparative Performance on ARC-AGI

TRM's test accuracy surpasses that of most leading LLMs while using a fraction of the computational resources.

Model / Benchmark	Parameters	ARC-AGI-1 Test Accuracy	ARC-AGI-2 Test Accuracy
TRM	7M	45%	8%
Most LLMs*	>70B+	Lower than TRM	Lower than TRM

*Including Deepseek R1, o3-mini, and Gemini 2.5 Pro.

Key Performance Insights:

• Superior Accuracy: With 45% accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, TRM's performance is explicitly stated to be higher than that of most LLMs.
• Extreme Parameter Efficiency: TRM achieves these results with less than 0.01% of the parameters used by the LLMs it is compared against. This highlights a profound efficiency advantage.

Core Thesis: Efficiency Over Scale

The central argument of the research is that for certain classes of hard reasoning problems, architectural design is more critical than sheer model scale. The success of TRM indicates that small, specialized, recursive networks present a promising path forward for artificial intelligence, challenging the paradigm that ever-larger models are the only way to advance capability.