Google researchers have developed a system called Dream-RSI that significantly reduces the number of discovery-agent calls needed for exploration tasks. In tests, it required up to 162 times fewer calls than the existing SimpleTES system.
The innovation lies in letting agents “dream” their way through exploration loops, learning from past attempts to guide future searches. This approach, detailed by a team of 17 researchers from Google, Google DeepMind, the University of Maryland, and the University of Virginia, aims to cut down on repetitive and costly reruns.
How Dream-RSI Works
Dream-RSI operates through a lightweight orchestration layer that manages branching, parallel exploration, and stopping decisions. It builds “historical discovery trees” that record every agent decision and outcome. This allows the system to evaluate new strategies by reading past records instead of rerunning the discovery agent.
The process unfolds in three stages: “online explore,” where policies guide an agent to build a discovery tree and log historical traces; a “construct replay simulator,” where the tree and its traces are added to a reusable repository; and “dreaming-based policy improvement,” where the agent essentially dreams up alternative policies.
Across several domains, Dream-RSI matched or improved discovery quality while substantially reducing compute costs.
Benchmark Performance
In tests, Dream-RSI demonstrated significant efficiency gains. Using Gemini-3.1-Pro, it cut average runtime from 3,587.1 to 2,931.0 milliseconds and reduced discovery-agent calls from 550 to 317. With Gemini-3.7-Flash, runtime dropped from 2,516.7 to 2,350.6 milliseconds, and calls decreased from 3,200 to 1,879.
Read Also: Cohere encrypts AI inference for enterprises
The researchers evaluated Dream-RSI in GPU kernel engineering, where agents are tasked with discovering high-performance kernel implementations without sacrificing numerical correctness. On the ConvDiv GPU-kernel task, Dream-RSI’s performance successively improved, cutting evaluated attempts from 110 to 50 before spending again, up to around 90, when progress plateaued. On the GPU programming tasks VGG16 and LayerNorm, Dream-RSI reached comparable performance using 2.43x and 1.79x fewer generations. On ConvDiv and ConvMax, it achieved 2.09x and 1.44x higher performance under comparable budgets.
For teams already investing heavily in agent calls, Dream-RSI offers a way to maximize the value of their existing records. The code is publicly available, making it accessible for broader adoption.
This development comes amid ongoing debates about the pace of AI advancement, with figures like Dario Amodei calling for caution. However, Dream-RSI exemplifies how self-exploring agent loops can be harnessed for practical, efficiency-driven applications rather than existential risks.
Addressing Exploration Bottlenecks in RSI
Researchers identify “meta-level feedback” as a key bottleneck in recursive self-improvement (RSI). This feedback is delayed, expensive, and often redundant, as builders repeat feedback they’ve already obtained. Exploration strategies remain static, unable to learn from past failures, leading to wasted computation and time. The project site notes that this inefficiency severely limits RSI’s scalability and effectiveness, particularly as discovery loops expand to thousands of cycles.
Real-World Applications and Benchmarks
In Lasso discovery tests, Dream-RSI outperformed Recursive Fixed Exploration (RFE) by delivering a better quality-to-compute tradeoff.
