Learning to Orchestrate Agents in Natural Language

Background

Multi-Agent Orchestration

The elicitation of “reasoning” in LLMs, broadly understood as the capability to understand and decompose highly complex tasks into methodically solvable steps, with appropriate exploration, self-refinement, backtracking, and aggregation of intermediate reasoning pathways, is increasingly accredited to reinforcement learning.

The RL Conductor

The Conductor objective is to solve tasks indirectly by designing agentic workflows to input questions.

Each workflow is made up of:

1. The participating agents
2. The subtasks assigned to each agent
3. The communication topology between agents

Conductor

Here's the plan:

1. Model 2 will develop an algorithm to efficiently count all possible complete subarrays for a given array.
2. Model 0 will implement the function to solve the problem.

subtasks = ["Develop an efficient algorithm to count the number of complete subarrays of an array", "Implement the algorithm described by the previous agent in Python"]
access_list = [[], ["all"]]

...over the format reward, which enforces the parseable Conductor structured response, and the final downstream task reward.

Training

m o d e l_id=(2,0J)
L i s t=I\{J/,,l^a{l}!l l^{n}J\}
"Verify the proposed strategy and find any gaps or errors in the understanding. Provide a step-by-step verification process in <idea>tags."

User

subtasks = ["Understand the problem and find a strategy to calculate the punishment number. Identify the integers whose squares can be partitioned into substrings that sum to the integer itself. Then, calculate the sum of the squares of these integers.",
"Refine the proposed strategy if necessary and then implement the punishmentNumber function in Python according to the given constraints. Provide a step-by-step solution in <idea>tags and return the final code in <answer>tags as specified."]

"Calculate the time it takes for the object to go from x=5.70 cm to x=-1.60 cm using the equation derived in the previous step and the given parameters."

subtasks = ["Analyze the given parameters of the simple harmonic motion (period and amplitude) and the initial position of the object. Identify the mathematical equation that describes the position of the object as a function of time."]

Empirical Results

...