PaperSMILE
SMILE: Smooth Motion for Improved Long-Horizon VLA Execution
Claim tested: SMILE's B-spline coefficient prediction improves real-robot task success over raw-action SmolVLA at matched prediction and execution horizons.
Tested
Result
Improvement over baseline
Not replicated
- Reported
- +11 pts
- Ours
- 0 pts
- Gain vs baseline
- 95% CI, −9 pts to +9 pts
- Paper's reported gain
- No difference
Success rate
- Reported
- 80%
- Ours
- 22%
- Measured on our stations
- 95% CI, 16%–29%
- Reported in the paper
Across iterations
- Iteration 1vs SmolVLA0 pts
- Iteration 2vs SmolVLA+6 pts
By task
| Task | BaselineSmolVLA | MethodSMILE | Improvement |
|---|---|---|---|
| e01Clamp electrical component | 3%1/30 | 3%1/30 | 0 pts−14 to +14 |
| e02Place loaded boxes in container (bimanual) | 10%3/30 | 7%2/30 | −3 pts−20 to +13 |
| e03Place spray bottles upright, nozzle forward (bimanual handover) | 47%14/30 | 50%15/30 | +3 pts−21 to +27 |
| e04Pour screws into the funnel | 50%15/30 | 50%15/30 | 0 pts−24 to +24 |
| e05Put tools in standing toolbox | 0%0/30 | 0%0/30 | 0 pts−11 to +11 |
| All tasks | 22%33/150 | 22%33/150 | 0 pts−9 to +9 |
| Reported in the paper | 69% | 80% | +11 pts |
Rollouts
One clip per environment from this entry's evaluation, beside where each evaluation episode placed the environment's objects, coloured by its rating. Use the button beside each environment to open its evaluation dataset in the dataset viewer.
e04Pour screws into the funnel BaselineSmolVLA15/3050%
Show in dataset viewer of SmolVLA on Pour screws into the funnelMethodSMILE15/3050%
Show in dataset viewer of SMILE on Pour screws into the funnel- Held-out benchmark environment
e01Clamp electrical component BaselineSmolVLA1/303%
Show in dataset viewer of SmolVLA on Clamp electrical componentMethodSMILE1/303%
Show in dataset viewer of SMILE on Clamp electrical component - Held-out benchmark environment
e02Place loaded boxes in container (bimanual) BaselineSmolVLA3/3010%
Show in dataset viewer of SmolVLA on Place loaded boxes in container (bimanual) - Held-out benchmark environment
e03Place spray bottles upright, nozzle forward (bimanual handover) - Held-out benchmark environment
e05Put tools in standing toolbox BaselineSmolVLA0/300%
Show in dataset viewer of SmolVLA on Put tools in standing toolboxMethodSMILE0/300%
Show in dataset viewer of SMILE on Put tools in standing toolbox
Implementation Details
What we changed
We compare SMILE with raw-action SmolVLA to test whether predicting B-spline coefficients improves real-robot task success at matched prediction and execution horizons. No official code is released, so we reimplemented SMILE’s action interface from the paper on SmolVLA’s pretrained backbone. The action expert denoises 8 cubic B-spline control points that are decoded into the action chunk. Both models predict relative translation and rotation-vector actions with absolute gripper widths and use matched training settings within each task.
- Prediction length. Both models predict 50 action steps, following the SmolVLA recipe. This prediction length is not confirmed in the SMILE paper.
- Synchronous execution. Both models execute 15 actions per plan, 0.5 seconds at 30 Hz, and the station waits for each plan to finish before requesting the next.
- Episode-end supervision. We fit splines only to valid samples and supervise only the coefficient directions they determine, filling unconstrained directions with the dataset mean. This retains partial windows in both arms; full windows use the paper’s coefficient-space loss.
- Fine-tuning budget. Both arms use batch 64, learning rate 1e-4 and at least 20k steps or three epochs, whichever is greater, without image augmentation or a validation split. Exact parity cannot be established because the paper does not report a complete training recipe.
Prompt summary
Test whether SMILE's B-spline coefficient prediction improves real-robot task success over raw-action SmolVLA at matched horizons on 5 tasks, replacing only SmolVLA's action interface and using synchronous fixed-horizon execution for both arms. Use delta translation and rotation-vector actions with an absolute gripper, F=50 and H=15 at 30 Hz, dataset-level coefficient normalisation, validity masking at episode boundaries, and batch 64 with lr 1e-4 for at least 20k steps. Finish the sync comparison by requesting the missing evals.
Response summary
Sync, matched horizons: SmolVLA vs SMILE pour 15/30 vs 15/30, boxes 3/30 vs 2/30, toolbox 0 vs 0, spray 14/30 vs 15/30, clamp 1/30 vs 1/30; 22.0% vs 22.0% success, +3.2 pp progress (p = 0.41).
Conclusion
Level on success under synchronous execution; floor tasks fail at grasp placement, which the action representation does not address. Moved to blended execution next.