Replacing traditional retrieval systems like RAG with a direct corpus interaction approach can reduce AI retrieval costs by 30%, according to researchers. This approach, which involves low memory usage, lets agents bypass embedding models and search raw corpora directly using standard command-line tools.
The current retrieval systems have limitations, as they rely on vector representations of documents and filter results based on semantic similarity. However, modern agentic applications require more precise search capabilities, such as exact string matching and sparse combinations of clues.
Direct corpus interaction addresses these limitations by providing direct access to the corpus, allowing agents to reason over the current state of the workspace rather than relying on outdated vector indexes. This approach is particularly useful in enterprise environments where data is constantly changing.
How Direct Corpus Interaction Works
It operates in a terminal-like environment, where agents use commands like “find” and “glob” to navigate directory structures and locate files. For exact matching, they use “grep” and “rg” to locate specific keywords, regex patterns, and exact strings.
Agents can combine these tools via shell pipelines to execute complex search logic in a single step. This approach allows for strict lexical constraints, such as searching a file for one term and piping the output to search for a second term.
The researchers propose two versions of this system: DCI-Agent-Lite and DCI-Agent-CC. DCI-Agent-Lite is a lightweight, low-cost setup built on the GPT-5.4 nano model, while DCI-Agent-CC is a higher-performance version designed for teams with more compute budget.
Performance and Cost Savings
Direct corpus interaction systematically outperformed traditional retrieval systems in agentic search benchmarks, with a 30.7-point improvement in accuracy on multi-hop QA benchmarks. The approach also reduced API costs by $600 in some cases, which is a significant reduction in network costs.
On the complex BrowseComp-Plus benchmark, swapping a traditional semantic retriever for direct corpus interaction improved accuracy from 69.0% to 80.0% while reducing costs from $1,440 to $1,016.
Direct corpus interaction has lower overall document recall than dense embedding models but extracts more value from relevant documents once found.
Use Cases and Limitations
It is most useful in tasks that require exact evidence localization in a dynamic workspace, such as debugging production incidents, searching large codebases, and analyzing logs.
However, direct corpus interaction has limitations, including lower broad document recall and higher latency and compute costs due to the high volume of iterative tool calls required, which can be mitigated by using models like Alibaba Qwen3.7-Max that can run for extended periods.
Granting an agent expressive tools like an unrestricted bash shell also creates significant context-management and security challenges for IT departments.
Future Implications
Direct corpus interaction changes how we think about enterprise data, which will need to be organized for agents that can inspect, compare, grep, trace, and verify. File names, timestamps, stable identifiers, metadata, version history, and machine-readable structure become part of the retrieval interface.
The researchers have released the code for direct corpus interaction under the permissive MIT license, allowing for further development and deployment of this technology.
