If a language model understands your request, is it safe to let it run the simulation?
In our benchmark work (HEPToolBench, previous post) we found that models handle collider requests far more reliably when they return typed fields instead of raw tool syntax.
HEPLocalAgent is what happens when you take that lesson and build a working tool around it — one that runs entirely on your own machine, with a locally served model, no per-call charges, and nothing leaving your hardware.
The design rule is simple: *the model interprets; it never executes.*
You type “simulate p p > t t~ at 13 TeV with 10000 events, run Pythia8 and Delphes.” The model proposes a typed workflow. Then deterministic code takes over:
- re-reads your request and restores what you explicitly stated (process, energy, event count, model, stages) — so 13 TeV becomes 6500 GeV per beam whether or not the model got that right;
- checks the workflow against the physics model and the supported feature set;
- writes the MadGraph script itself and shows it to you;
- waits for your approval, then launches the external programs without a shell and verifies that the requested LHE/HepMC/ROOT outputs actually appeared.
How far can we answer now?
On a same-response comparison — one model proposal, replayed through two paths — the first structured proposal passes the external benchmark scorer in 11 of 47 cases; after the full grounding-and-validation pipeline it passes in 43. A three-way comparison makes the layering visible: direct native generation 2/61, structured proposal plus builder 8/61, full guarded agent 52/61. Structure alone buys a little. Grounding and validation carry most of the weight. Executed workflows reproduced cross sections to 0.3% between independent seeds.
And here is the part I think matters most for anyone building scientific agents.
We threw 96 deliberately problematic requests at it — contradictory instructions, impossible stage combinations, requests to run arbitrary code. No unsafe payload ever reached an executable artifact. But 57 of the 96 still reached the approval stage in modified form: a scan silently reduced to one run, a contradiction not noticed, a missing parameter filled with a default. In 17 cases the scientific intent had changed.
A workflow that fails loudly is easy to catch. A workflow that runs and answers a different question is not. That is the real unsolved boundary — and it’s why v1.0.1 requires expert approval before execution and should not be run unattended. Clarification turns and contradiction detection are the next release’s priority.
Although we built this for collider physics, the architecture is not physics-specific. Model interprets → deterministic code grounds, validates, and builds → human approves → tools run locally. That pattern is what you need wherever the data must stay on-premises for security or privacy reasons: medical imaging and patient records, genomics, proprietary industrial data, legal or financial documents. The code is MIT-licensed precisely so that people in those fields can take the skeleton and swap in their own tools.
Paper: https://arxiv.org/abs/2608.28244
Code (MIT): https://github.com/AadarshSingh0/HEPLocalAgent
This project was driven by Aadarsh Singh, who built the agent, the managed HEP software stack, and the evaluation — including the adversarial requests that exposed its limits. Centre for High Energy Physics, IISc Bengaluru.
