PaperDSD
Direction-Scale Decomposition in Action Representation: Rethinking What to Tokenize for Vision-Language-Action Models
Claim tested: DSD's direction–scale action representation improves real-robot task success over standard action binning, without additional robotics pretraining.
Tested
Result
Improvement over baseline
Not replicated
- Reported
- +13 pts
- Ours
- −1 pts
- Gain vs baseline
- 95% CI, −12 pts to +10 pts
- Paper's reported gain
- No difference
Success rate
- Reported
- 73%
- Ours
- 45%
- Measured on our stations
- 95% CI, 38%–53%
- Reported in the paper
Across iterations
By task
| Task | BaselineUniform action binning | MethodDSD | Improvement |
|---|---|---|---|
| e01Clamp electrical component | 13%4/30 | 3%1/30 | −10 pts−27 to +6 |
| e02Open the toolbox with a screwdriver | 17%5/30 | 20%6/30 | +3 pts−17 to +23 |
| e03Place loaded boxes in container (bimanual) | 40%12/30 | 60%18/30 | +20 pts−5 to +42 |
| e04Pour screws into the funnel | 80%24/30 | 60%18/30 | −20 pts−41 to +3 |
| e05Datum-corner alignment | 80%24/30 | 83%25/30 | +3 pts−17 to +23 |
| All tasks | 46%69/150 | 45%68/150 | −1 pts−12 to +10 |
| Reported in the paper | 60% | 73% | +13 pts |
Rollouts
One clip per environment from this entry's evaluation, beside where each evaluation episode placed the environment's objects, coloured by its rating. Use the button beside each environment to open its evaluation dataset in the dataset viewer.
e02Open the toolbox with a screwdriver BaselineUniform action binning5/3017%
Show in dataset viewer of Uniform action binning on Open the toolbox with a screwdrivere04Pour screws into the funnel BaselineUniform action binning24/3080%
Show in dataset viewer of Uniform action binning on Pour screws into the funnelMethodDSD18/3060%
Show in dataset viewer of DSD on Pour screws into the funnel- Held-out benchmark environment
e01Clamp electrical component BaselineUniform action binning4/3013%
Show in dataset viewer of Uniform action binning on Clamp electrical componentMethodDSD1/303%
Show in dataset viewer of DSD on Clamp electrical component - Held-out benchmark environment
e03Place loaded boxes in container (bimanual) BaselineUniform action binning12/3040%
Show in dataset viewer of Uniform action binning on Place loaded boxes in container (bimanual) - Held-out benchmark environment
e05Datum-corner alignment BaselineUniform action binning24/3080%
Show in dataset viewer of Uniform action binning on Datum-corner alignmentMethodDSD25/3083%
Show in dataset viewer of DSD on Datum-corner alignment
Implementation Details
What we changed
We compare DSD with standard action binning to test whether separating motion direction from magnitude improves real-robot task success without additional robotics pretraining. We reimplemented both representations on the same Florence-2 policy, starting from its public weights and following the paper’s parallel action decoding. Both arms train for 40,000 steps at batch 32 on one GPU, and the two arms differ only in the action representation.
- Longer chunks. We predict 30 steps instead of 20 because controls run at 30 Hz, giving a one-second action chunk.
- Absolute-command execution. Predicted relative increments are accumulated into absolute commands. The first increment is measured from the observed pose and has separate normalization statistics; later increments are differences between successive commands. Both arms use this construction.
- Wider normalization bounds. Both arms use the 0.1st/99.9th percentiles instead of the paper’s 1st/99th, for BIN coordinates and DSD magnitudes alike. With 1st/99th bounds, BIN alone clipped fast motions in our data, so even perfect tokens ended its one-second chunks about 15 to 25 mm off at the 95th percentile against about 1.5 mm for DSD; with the wider bounds both are near 1 mm.
- Exact-zero decoding. Motion bins containing zero decode to exactly zero instead of their bin centers, preventing idle drift when increments are accumulated. This replaces reliance on a controller dead zone and applies to both arms; grippers are unchanged.
- Direction-loss weighting. Our DSD weights the direction loss with a soft ramp,
w = clip(s/τ, 0.1, 1), with τ = 0.3% of the full-scale range. The authors’ implementation uses a step: weight 0.1 below the threshold and 1.0 above it. In the authors’ LIBERO experiments, switching to the ramp alone reduced success by about 3 points, bringing DSD close to their binning baseline. The authors also recommend a lower threshold for our action definition (consecutive pose differences): 0.05–0.1% of the full-scale range, compared with the 0.3% we used. Both differences may have reduced DSD’s performance in our evaluation; their effect on our tasks was not measured.
Prompt summary
Test whether DSD's direction-scale action representation improves real-robot task success over standard action binning without additional robotics pretraining, on 5 tasks of your choice. (Florence-2-Base policy with parallel action-token decoding; both arms used 0.1/99.9 percentile bounds after a review.)
Response summary
No benefit: DSD 45.3% vs BIN 46.0% SR (-0.7 pp, 95% CI -8.0 to +6.7, p = 1.0; AP 59.6% vs 61.4%), with per-task effects from -20 pp (pour) to +20 pp (boxes) cancelling out.
Conclusion
No benefit, but the BIN baseline used wider bounds than the paper. Next: use the paper's 1/99 bounds with clipping for BIN.