Bardiya Akhbari

We introduce a vocabulary for automated research systems built from one or more agents to make their design choices easier to describe and compare. The vocabulary specifies who the agents are, which operations the harness exposes, who may invoke them, how agents communicate, what state is visible within and across runs, how the next action is chosen, how a run begins, and how outputs are scored. A trajectory records one run from the input task to the returned artifact.

Set theory is foundational to mathematics and, when sets are finite, to reasoning about the world. An intelligent system should perform set operations consistently, regardless of superficial variations in the operands. Initially designed for semantically-oriented NLP tasks, large language models (LLMs) are now being evaluated on algorithmic tasks.

Published on Jul 13 · Submitted by Bardiya Akhbari on Jul 28 Abstract We introduce a vocabulary for automated research systems built from one or more agents to make their design choices easier to describe and compare.