# Benchmarking Verbal Reinforcement Learning for Robotics
By Cole Abbott, Liam Zhong
Frontier models like GPT-6 Astra and Gemini 4 Argon can control robots, but how much does the harness around them affect performance? Coding agents use text-based `SKILL.md` files to carry task-specific knowledge. We wanted to see whether the same idea could improve robotic manipulation.
We first tested Astra, Argon and Fable on 40 LIBERO tasks, with five attempts per task and model. With planning, rationale and memory enabled, but no task-specific skills, they achieved 174/600 native successes—29%. Some tasks were never completed.
## The Nomad harness
[Nomad](https://github.com/runtime-si/Nomad_Harness_Interface) connects model providers to simulated robots through a common control loop. The model receives camera images, robot state and feedback from its previous actions. It proposes one bounded action at a time, such as moving the end effector or closing the gripper.
Nomad checks the action against the robot’s limits, sends it to the simulator, then returns a fresh observation and execution feedback. The environment’s native task predicate determines success. Each run records the actions, outcomes, wall time and token usage, with video when supported.
A task skill is additional text passed to the model. It can describe a grasp, a sequence of steps or how to recover from a failure. In these experiments, skills changed the instructions; the robot controller and success criteria stayed fixed.
*Nomad keeps the action–observation contract consistent while the model, task instructions and simulator can change.*
## Developing skills on three tasks
We started with three tasks: two cans into a basket, a bowl into a drawer and close it, and a mug into a microwave and close it. In the initial 20-run no-skill controls, Astra never completed the drawer or microwave tasks.
A critic agent—GPT-6.1 Sol in Codex—reviewed failed traces and wrote a `SKILL.md` file for the robotics agent. We evaluated the skill, returned the new traces to the critic, and repeated the process. Each evaluated skill version was frozen before its comparison.
*Verbal reinforcement learning through skill refinement. Recorded outcomes guide a critic’s feedback, which becomes a new skill file. Each version is frozen before evaluation; model weights stay fixed. The feedback example is illustrative.*
The drawer skill reached **12/20 successes**, compared with **0/20 without a skill**, at the same 120-decision budget. The microwave needed more refinement: version 9 reached **16/20**, compared with **0/20 for version 4**, with both versions given 180 decisions. No model weights were changed.
*Twenty matched starts per task. Basket and drawer compare no skill with a skill at 120 decisions; microwave compares the earlier v4 skill with v9 at 180 decisions. These are separate comparisons.*
*Microwave, matched state 48: earlier v4 skill versus refined v9 skill. Playback is 4× simulation time; model waiting is omitted. This is an illustrative failure/success pair.*
## Expanding to the full suite
We then expanded skill development across all 40 LIBERO tasks and completed three confirmation rounds. For each task, we chose the best observed complete 15-attempt skill cohort, whether it used the original skill or a revision. One cohort was selected across all three models; every success and failure in it was retained. Ties favored the lower token budget, then a fixed batch order.
*Native success across refinement iterations, including non-improving revisions. Task sets and starts change between rounds; controls and revisions have matched budgets within each comparison. Interrupted v6 is excluded. Round-three artifact verification remains pending.*
[Per-model plot](skill-refinement/success-across-iterations-by-model.svg) · [Data](skill-refinement/success-across-iterations.csv)
Thirty-five tasks have complete skill cohorts. On those same tasks and starting states, no skill achieved 152/525 native successes (29.0%), compared with 283/525 (53.9%) for the best observed skill cohorts.
The full suite has 200 planned attempts per model. For the five tasks without a complete skill cohort, we retain the original-skill attempts, including 10 infrastructure interruptions. The table reports confirmed successes out of 200 planned; interrupted outcomes remain unknown, rather than task failures.
| Model | No skill /200 | Best skill: confirmed /200 |
| :--- | ---: | ---: |
| Astra | 66/200 · 33.0% | 123/200 · 61.5% |
| Argon | 51/200 · 25.5% | 92/200 · 46.0% |
| Fable | 57/200 · 28.5% | 88/200 · 44.0% |
*No skill versus the best observed original or revised skill cohort for each task. Percentages count confirmed native successes out of 200 planned attempts per model; 10 infrastructure-interrupted outcomes remain unknown. This retrospective comparison combines differing budgets and platforms; round-three artifact verification remains pending.*
*Alphabet soup, state 2: one of three models succeeds without a skill; all three succeed with the revised skill. We selected this example to show improvement. It is 1× simulation replay, with API waiting omitted. Across the full 15-case soup cohort, the revised skill scored 14/15 versus 3/15 for its same-budget original-skill control.*
## What this shows
Adding a skill was not enough. The first skill catalog reduced matched success from 165/562 without skills to 142/562 with skills. Failure-driven revision produced much stronger results on selected tasks.
The best-observed comparison selects cohorts after seeing their outcomes, sometimes across several evaluations of the same skill. It is a retrospective performance envelope, not a single evaluated benchmark. It also combines skill and budget effects: historical no-skill runs had 100 calls and 25 minutes; later skill runs had 180 calls and 45 minutes, or 240 calls and 60 minutes for selected tasks. Fable changed from macOS GLFW to Linux OSMesa. These adaptive, in-sample results do not establish held-out generalization or isolate the effect of skill text.
Text-based skills look useful for robotic manipulation, especially when they address failures visible in the traces. The next test is whether the frozen skills transfer to new models.
[All 40 task results](skill-refinement/no-skill-vs-best-skill-tasks.csv) · [Model statistics](skill-refinement/no-skill-vs-best-skill-models.csv) · [Detailed analysis](three-model-libero/latest.html)
*Working report: third-round success and wall time come from complete host snapshots; full artifact verification remains pending. API costs are usage-based estimates and exclude VM rental and unreported usage. Infrastructure interruptions are retained separately from task failures.*