All three old-model confirmation rounds are terminal. Skill tuning has stopped.
For each task, choose one complete 15-case candidate cohort that strictly beats its same-budget original-skill control. Rank by aggregate native success, then lower token budget, then deterministic batch name. The same chosen skill applies to all three models; no individual successful-case or per-model selection. Tasks without an eligible revision are excluded from this comparison and shown in the all-40 table.
| Model / condition | Native successes | Success rate | Mean wall min | Mean API USD / attempt | Cost subset n |
|---|---|---|---|---|---|
| Astra / No task skill | 39/140 | 27.9% | 16.8 | $3.63 | 110 |
| Astra / Revised task skill | 94/140 | 67.1% | 15.7 | $4.15 | 110 |
| Argon / No task skill | 42/140 | 30.0% | 15.3 | $0.92 | 110 |
| Argon / Revised task skill | 79/140 | 56.4% | 22.0 | $1.50 | 110 |
| Fable / No task skill | 41/140 | 29.3% | 18.3 | $6.88 | 110 |
| Fable / Revised task skill | 62/140 | 44.3% | 27.6 | $10.03 | 110 |
Historical no skill: planning + rationale + memory, 100 calls / 25 minutes / 500k tokens. Revised skills: 180 calls / 45 minutes / 1.2m tokens, with confirmed escalated tasks at 240 calls / 60 minutes / 1.8m tokens. Skill and budget effects are combined. Fable changed from macOS GLFW to Linux OSMesa. Adaptive in-sample selection makes this descriptive, not held-out generalization.
Same task and state 2: one of three models succeeds without a task skill; all three succeed with the revised skill. This example was selected for that contrast at the user’s request. Across all 15 soup trials, the revised skill achieved 14/15 versus 3/15 for its same-budget original-skill control. The video’s no-skill arm has the smaller historical budget.
1× simulation replay, API waiting omitted. Final frames held. Each panel shows final native outcome, actual episode wall time and estimated API cost. Download MP4 · Sources and selection provenance.
| Task | Original skill | Revised skill | Selection status |
|---|---|---|---|
| Open the middle layer of the drawer | 6/15 | 11/15 | Eligible improvement |
| Put the cream cheese on the bowl | 11/15 | 10/15 | Regression; excluded |
| Put the wine bottle on the rack | 3/15 | 9/15 | Eligible improvement |
| Pick the akita black bowl from table center and place it on the plate | 5/15 | 10/15 | Eligible improvement |
| Pick the akita black bowl next to the plate and place it on the plate | 4/15 | 6/15 | Eligible improvement |
Round-three aggregate: 29/75 original-skill successes versus 46/75 revised-skill successes. Equal standard budgets in both arms. No additional tasks reached the 12/15 quality target; four of 40 have done so across confirmation rounds.
Every task is listed. A dash means no eligible improving revised cohort. Individual model counts are out of five; aggregate counts are out of 15. A rejected candidate’s best observed score is included separately.
| Task | No skill /15 | Eligible revised /15 | Astra no→revised | Argon no→revised | Fable no→revised | Same-budget original skill /15 | Selected source | Best assessed candidate /15 | Revised cost |
|---|---|---|---|---|---|---|---|---|---|
| libero_spatial/0 Pick the akita black bowl between the plate and the ramekin and place it on the plate | 5 | 10 | 1 → 4 | 1 → 3 | 3 → 3 | 5 | confirm-standard | 10 | available |
| libero_spatial/1 Pick the akita black bowl next to the ramekin and place it on the plate | 3 | — | 1 → — | 0 → — | 2 → — | — | No eligible revision | Unassessed | not applicable |
| libero_spatial/2 Pick the akita black bowl from table center and place it on the plate | 7 | 10 | 1 → 3 | 3 → 3 | 3 → 4 | 5 | confirm-round3-standard | 10 | pending retrieval or unreported usage |
| libero_spatial/3 Pick the akita black bowl on the cookies box and place it on the plate | 5 | 6 | 1 → 4 | 2 → 2 | 2 → 0 | 5 | confirm-standard | 6 | pending retrieval or unreported usage |
| libero_spatial/4 Pick the akita black bowl in the top layer of the wooden cabinet and place it on the plate | 0 | 2 | 0 → 2 | 0 → 0 | 0 → 0 | 1 | confirm-standard | 2 | available |
| libero_spatial/5 Pick the akita black bowl on the ramekin and place it on the plate | 7 | 6 | 3 → 3 | 4 → 3 | 0 → 0 | 3 | confirm-standard | 6 | pending retrieval or unreported usage |
| libero_spatial/6 Pick the akita black bowl next to the cookies box and place it on the plate | 2 | 4 | 0 → 1 | 0 → 3 | 2 → 0 | 1 | confirm-round2-standard | 4 | available |
| libero_spatial/7 Pick the akita black bowl on the stove and place it on the plate | 5 | 9 | 1 → 5 | 1 → 2 | 3 → 2 | 4 | confirm-standard | 9 | available |
| libero_spatial/8 Pick the akita black bowl next to the plate and place it on the plate | 4 | 6 | 0 → 4 | 0 → 0 | 4 → 2 | 4 | confirm-round3-standard | 6 | pending retrieval or unreported usage |
| libero_spatial/9 Pick the akita black bowl on the wooden cabinet and place it on the plate | 4 | — | 2 → — | 0 → — | 2 → — | — | No eligible revision | 3 | not applicable |
| libero_object/0 Pick the alphabet soup and place it in the basket | 4 | 14 | 0 → 4 | 4 → 5 | 0 → 5 | 3 | confirm-round2-standard | 14 | available |
| libero_object/1 Pick the cream cheese and place it in the basket | 1 | 9 | 1 → 5 | 0 → 2 | 0 → 2 | 1 | confirm-standard | 9 | available |
| libero_object/2 Pick the salad dressing and place it in the basket | 8 | 10 | 4 → 2 | 2 → 5 | 2 → 3 | 9 | confirm-round2-standard | 10 | available |
| libero_object/3 Pick the bbq sauce and place it in the basket | 1 | 9 | 0 → 5 | 1 → 1 | 0 → 3 | 5 | confirm-round2-standard | 9 | available |
| libero_object/4 Pick the ketchup and place it in the basket | 4 | — | 1 → — | 3 → — | 0 → — | — | No eligible revision | Unassessed | not applicable |
| libero_object/5 Pick the tomato sauce and place it in the basket | 4 | — | 4 → — | 0 → — | 0 → — | — | No eligible revision | Unassessed | not applicable |
| libero_object/6 Pick the butter and place it in the basket | 12 | 14 | 5 → 5 | 3 → 4 | 4 → 5 | 12 | confirm-round2-standard | 14 | available |
| libero_object/7 Pick the milk and place it in the basket | 10 | 12 | 5 → 5 | 2 → 2 | 3 → 5 | 8 | confirm-standard | 12 | available |
| libero_object/8 Pick the chocolate pudding and place it in the basket | 14 | 15 | 5 → 5 | 5 → 5 | 4 → 5 | 14 | confirm-standard | 15 | available |
| libero_object/9 Pick the orange juice and place it in the basket | 8 | — | 5 → — | 0 → — | 3 → — | — | No eligible revision | 6 | not applicable |
| libero_goal/0 Open the middle layer of the drawer | 4 | 11 | 1 → 5 | 3 → 4 | 0 → 2 | 6 | confirm-round3-standard | 11 | pending retrieval or unreported usage |
| libero_goal/1 Put the bowl on the stove | 4 | 7 | 0 → 3 | 1 → 1 | 3 → 3 | 6 | confirm-standard | 7 | available |
| libero_goal/2 Put the wine bottle on the top of the drawer | 6 | 10 | 2 → 5 | 3 → 5 | 1 → 0 | 5 | confirm-round2-escalated | 10 | available |
| libero_goal/3 Open the top layer of the drawer and put the bowl inside | 0 | 2 | 0 → 0 | 0 → 2 | 0 → 0 | 0 | confirm-round2-standard | 2 | available |
| libero_goal/4 Put the bowl on the top of the drawer | 9 | — | 2 → — | 3 → — | 4 → — | — | No eligible revision | Unassessed | not applicable |
| libero_goal/5 Push the plate to the front of the stove | 0 | — | 0 → — | 0 → — | 0 → — | — | No eligible revision | Unassessed | not applicable |
| libero_goal/6 Put the cream cheese on the bowl | 6 | — | 4 → — | 1 → — | 1 → — | — | No eligible revision | 10 | not applicable |
| libero_goal/7 Turn on the stove | 12 | 11 | 4 → 2 | 4 → 5 | 4 → 4 | 8 | confirm-round2-standard | 11 | available |
| libero_goal/8 Put the bowl on the plate | 4 | — | 2 → — | 0 → — | 2 → — | — | No eligible revision | 8 | not applicable |
| libero_goal/9 Put the wine bottle on the rack | 2 | 9 | 0 → 5 | 2 → 4 | 0 → 0 | 3 | confirm-round3-standard | 9 | pending retrieval or unreported usage |
| libero_10/0 put both the alphabet soup and the tomato sauce in the basket | 7 | — | 3 → — | 2 → — | 2 → — | — | No eligible revision | Unassessed | not applicable |
| libero_10/1 put both the cream cheese box and the butter in the basket | 3 | — | 3 → — | 0 → — | 0 → — | — | No eligible revision | Unassessed | not applicable |
| libero_10/2 turn on the stove and put the moka pot on it | 3 | 8 | 2 → 3 | 0 → 4 | 1 → 1 | 4 | confirm-standard | 8 | available |
| libero_10/3 put the black bowl in the bottom drawer of the cabinet and close it | 0 | 11 | 0 → 3 | 0 → 5 | 0 → 3 | 0 | confirm-round2-standard | 11 | available |
| libero_10/4 put the white mug on the left plate and put the yellow and white mug on the right plate | 0 | 3 | 0 → 1 | 0 → 0 | 0 → 2 | 0 | confirm-standard | 3 | available |
| libero_10/5 pick up the book and place it in the back compartment of the caddy | 1 | 5 | 0 → 0 | 1 → 4 | 0 → 1 | 3 | confirm-standard | 5 | available |
| libero_10/6 put the white mug on the plate and put the chocolate pudding to the right of the plate | 1 | 9 | 1 → 4 | 0 → 2 | 0 → 3 | 4 | confirm-round2-standard | 9 | available |
| libero_10/7 put both the alphabet soup and the cream cheese box in the basket | 4 | 9 | 2 → 4 | 0 → 2 | 2 → 3 | 7 | confirm-round2-standard | 9 | available |
| libero_10/8 put both moka pots on the stove | 0 | — | 0 → — | 0 → — | 0 → — | — | No eligible revision | 0 | not applicable |
| libero_10/9 put the yellow and white mug in the microwave and close it | 0 | 4 | 0 → 2 | 0 → 1 | 0 → 1 | 0 | confirm-escalated | 4 | available |
Model stats CSV · All 40 tasks CSV · Stats/provenance JSON · Latest plot PDF · SVG · Report.
Costs estimate reported response token/cache usage at dated standard list prices; missing usage and VM rental excluded. The cost panel uses the same verified task subset in both arms. Original 38 infrastructure interruptions and development shutdown 19 remain preserved separately. No benchmark promotion or new-model evaluation is implied.
