Reducing Agent Token Usage with Filesystem-Based Recursive Language Models
Production investigations can return more logs, traces, and command output than an agent needs in its prompt. We changed Foam’s internal agent to keep full results in files and delegate focused questions to smaller models. Across 63 tasks in 25 evaluation runs, token spend fell by roughly 90% with no observed loss of accuracy.
Recursive Language Models keep large inputs in an external environment. The model inspects selected portions and can make recursive calls to investigate narrower questions. [1] We adapted that approach to Foam’s internal agent: child investigators examine saved tool results and return findings to the parent.
Before Recursive Language Models
The earlier version of our internal coding agent passed large tool results through its main model, Claude Sonnet 4.6. As the investigation continued, the model processed logs, traces, and command output while deciding what to inspect next. The investigation needed access to those records, but the parent model did not need to process them on every subsequent call.
Each call could carry material that ultimately had no bearing on the answer.
With Recursive Language Models
In Foam's implementation, the Claude Sonnet 4.6 parent plans the investigation and writes the final answer. It gives focused questions to Claude Haiku 4.5 child investigators. Each child can use query-foam to inspect logs and traces. Large telemetry results and repository command output are saved under /tmp/rlm-docs/. Children receive a preview and a document reference, read relevant portions, and return findings to the parent.
The parent and children exchange questions and findings directly. Children save large intermediate results to files, and both the parent and children read from those files as needed. This keeps full results available without repeatedly placing them in the parent model’s prompt.
Evaluation
We compared the earlier approach with this filesystem-based setup on 63 tasks across 25 evaluation runs. The evaluation checked whether the agent reached the correct result as well as its token spend. In these runs, accuracy was unchanged and token spend fell by roughly 90%.
The comparison changed three things together: filesystem storage, delegated investigation, and Claude Haiku 4.5 for child tasks. It cannot attribute the reduction to any one change. We have not published task-level scores or separate input and output token counts.
After rollout
Monthly AWS Bedrock spend fell from roughly $60,000 to $6,000 after the new setup was rolled out. The bill is a before-and-after observation, not a controlled estimate of the architecture’s effect. Workload, model mix, and other usage changes can also affect a monthly bill.
Conclusion
On the tasks we tested, our internal coding agent maintained its measured accuracy with roughly 90% lower token spend. Its larger logs, traces, and command outputs remained available for investigation without being repeatedly placed in the parent model's prompt. The rollout was also followed by a large decline in monthly Bedrock spend.
A follow-up should vary storage, delegation, and model choice separately, then test the result on more investigations.
References
- Zhang, A. L., Kraska, T., and Khattab, O. Recursive Language Models, arXiv:2512.24601.