PaperSMILE
SMILE: Smooth Motion for Improved Long-Horizon VLA Execution
Claim tested: SMILE's B-spline coefficient prediction improves real-robot task success over raw-action SmolVLA at matched prediction and execution horizons.
Tested
Result
Improvement over baseline
Inconclusive
- Reported
- +11 pts
- Ours
- +6 pts
- Gain vs baseline
- 95% CI, −4 pts to +16 pts
- Paper's reported gain
- No difference
Success rate
- Reported
- 80%
- Ours
- 21%
- Measured on our stations
- 95% CI, 15%–29%
- Reported in the paper
Across iterations
- Iteration 1vs SmolVLA0 pts
- Iteration 2vs SmolVLA+6 pts
By task
| Task | BaselineSmolVLA | MethodSMILE | Improvement |
|---|---|---|---|
| e01Clamp electrical component | 3%1/30 | 13%4/30 | +10 pts−6 to +27 |
| e02Place loaded boxes in container (bimanual) | 7%2/30 | 13%4/30 | +7 pts−10 to +24 |
| e03Pour screws into the funnel | 50%15/30 | 57%17/30 | +7 pts−18 to +30 |
| e04Put tools in standing toolbox | 0%0/30 | 0%0/30 | 0 pts−11 to +11 |
| All tasks | 15%18/120 | 21%25/120 | +6 pts−4 to +16 |
| Reported in the paper | 69% | 80% | +11 pts |
Rollouts
One clip per environment from this entry's evaluation, beside where each evaluation episode placed the environment's objects, coloured by its rating. Use the button beside each environment to open its evaluation dataset in the dataset viewer.
e03Pour screws into the funnel BaselineSmolVLA15/3050%
Show in dataset viewer of SmolVLA on Pour screws into the funnelMethodSMILE17/3057%
Show in dataset viewer of SMILE on Pour screws into the funnel- Held-out benchmark environment
e01Clamp electrical component BaselineSmolVLA1/303%
Show in dataset viewer of SmolVLA on Clamp electrical componentMethodSMILE4/3013%
Show in dataset viewer of SMILE on Clamp electrical component - Held-out benchmark environment
e02Place loaded boxes in container (bimanual) BaselineSmolVLA2/307%
Show in dataset viewer of SmolVLA on Place loaded boxes in container (bimanual) - Held-out benchmark environment
e04Put tools in standing toolbox BaselineSmolVLA0/300%
Show in dataset viewer of SmolVLA on Put tools in standing toolboxMethodSMILE0/300%
Show in dataset viewer of SMILE on Put tools in standing toolbox
Implementation Details
What we changed
We compare SMILE with raw-action SmolVLA to test whether predicting B-spline coefficients improves real-robot task success. No official code is released, so we reimplemented SMILE’s action interface from the paper on SmolVLA’s pretrained backbone. The action expert denoises 8 cubic B-spline control points that are decoded into the action chunk. Both models predict relative translation and rotation-vector actions with absolute gripper widths and use matched training settings within each task. This round re-evaluates the first round’s models under blended execution without retraining them.
- Prediction length. Both models predict 50 action steps, following the SmolVLA recipe. This prediction length is not confirmed in the SMILE paper.
- Episode-end supervision. We fit splines only to valid samples and supervise only the coefficient directions they determine, filling unconstrained directions with the dataset mean. This retains partial windows in both arms; full windows use the paper’s coefficient-space loss.
- Fine-tuning budget. Both arms use batch 64, learning rate 1e-4 and at least 20k steps or three epochs, whichever is greater, without image augmentation or a validation split. Exact parity cannot be established because the paper does not report a complete training recipe.
- Blended execution. Both models were trained to execute 15 of their 50 actions per plan, 0.5 seconds at 30 Hz, for synchronous fixed-horizon execution. Here the station requests a new plan once less than 800 ms of actions remain and crossfades plans over 600 ms. Because each plan is shorter than that window, the station replans about every 90 ms and executes about three actions of each plan. Both arms use the same settings, but this replaces the paper’s fixed execution horizon.
Prompt summary
Re-evaluate the 10 existing models on blend runtimes in a new project, with the same starting gripper width and everything else unchanged, and report one blend success row per model with its runtime config. (A retraining step with a matched LR schedule, augmentation and a validation split was started and then discarded.)
Response summary
SmolVLA vs SMILE: pour 15/30 vs 17/30, boxes 2/30 vs 4/30, clamp 1/30 vs 4/30, toolbox 0/30 vs 0/30, spray 22/30 vs 24/30; 40/150 vs 49/150 (+6.0 pp, 95% CI −0.7 to +12.7, p = 0.14).
Conclusion
SMILE ahead or tied on every task (+6.0 pp, p = 0.14): small, consistent and not significant. Concluded as partially reproduced.