Poke &Wiggle
Papers
Paper

DSD

Direction-Scale Decomposition in Action Representation: Rethinking What to Tokenize for Vision-Language-Action Models

Claim tested: DSD's direction–scale action representation improves real-robot task success over standard action binning, without additional robotics pretraining.

Tested

Paper
arXiv⁠ (opens in a new tab)
Code
—
Venue
—
Authors
Yufei Duan et al.
Farm tasks
5
Evaluation
600 episodes (300 for baseline, 300 for method)

Result

Improvement over baseline

Not replicated

Reported
+13 pts
Ours
−1 pts
−200+20+40+60
  • Gain vs baseline
  • 95% CI, −12 pts to +10 pts
  • Paper's reported gain
  • No difference

Success rate

Reported
73%
Ours
45%
0%20%40%60%80%100%
  • Measured on our stations
  • 95% CI, 38%–53%
  • Reported in the paper

Across iterations

  1. Iteration 1vs Uniform action binning−1 pts
  2. Iteration 2vs Uniform action binning+13 pts
−200+20+40+60

By task

TaskBaselineUniform action binningMethodDSDImprovement
e01Clamp electrical component13%4/303%1/30−10 pts−27 to +6
e02Open the toolbox with a screwdriver17%5/3020%6/30+3 pts−17 to +23
e03Place loaded boxes in container (bimanual)40%12/3060%18/30+20 pts−5 to +42
e04Pour screws into the funnel80%24/3060%18/30−20 pts−41 to +3
e05Datum-corner alignment80%24/3083%25/30+3 pts−17 to +23
All tasks46%69/15045%68/150−1 pts−12 to +10
Reported in the paper60%73%+13 pts

Rollouts

One clip per environment from this entry's evaluation, beside where each evaluation episode placed the environment's objects, coloured by its rating. Use the button beside each environment to open its evaluation dataset in the dataset viewer.

  1. e02Open the toolbox with a screwdriver
  2. e04Pour screws into the funnel
  3. Held-out benchmark environment
    e01Clamp electrical component
  4. Held-out benchmark environment
    e03Place loaded boxes in container (bimanual)
  5. Held-out benchmark environment
    e05Datum-corner alignment

Implementation Details

What we changed

We compare DSD with standard action binning to test whether separating motion direction from magnitude improves real-robot task success without additional robotics pretraining. We reimplemented both representations on the same Florence-2 policy, starting from its public weights and following the paper’s parallel action decoding. Both arms train for 40,000 steps at batch 32 on one GPU, and the two arms differ only in the action representation.

  • Longer chunks. We predict 30 steps instead of 20 because controls run at 30 Hz, giving a one-second action chunk.
  • Absolute-command execution. Predicted relative increments are accumulated into absolute commands. The first increment is measured from the observed pose and has separate normalization statistics; later increments are differences between successive commands. Both arms use this construction.
  • Wider normalization bounds. Both arms use the 0.1st/99.9th percentiles instead of the paper’s 1st/99th, for BIN coordinates and DSD magnitudes alike. With 1st/99th bounds, BIN alone clipped fast motions in our data, so even perfect tokens ended its one-second chunks about 15 to 25 mm off at the 95th percentile against about 1.5 mm for DSD; with the wider bounds both are near 1 mm.
  • Exact-zero decoding. Motion bins containing zero decode to exactly zero instead of their bin centers, preventing idle drift when increments are accumulated. This replaces reliance on a controller dead zone and applies to both arms; grippers are unchanged.
  • Direction-loss weighting. Our DSD weights the direction loss with a soft ramp, w = clip(s/τ, 0.1, 1), with τ = 0.3% of the full-scale range. The authors’ implementation uses a step: weight 0.1 below the threshold and 1.0 above it. In the authors’ LIBERO experiments, switching to the ramp alone reduced success by about 3 points, bringing DSD close to their binning baseline. The authors also recommend a lower threshold for our action definition (consecutive pose differences): 0.05–0.1% of the full-scale range, compared with the 0.3% we used. Both differences may have reduced DSD’s performance in our evaluation; their effect on our tasks was not measured.

See the code changes on GitHub⁠ (opens in a new tab)

Prompt summary

Test whether DSD's direction-scale action representation improves real-robot task success over standard action binning without additional robotics pretraining, on 5 tasks of your choice. (Florence-2-Base policy with parallel action-token decoding; both arms used 0.1/99.9 percentile bounds after a review.)

Response summary

No benefit: DSD 45.3% vs BIN 46.0% SR (-0.7 pp, 95% CI -8.0 to +6.7, p = 1.0; AP 59.6% vs 61.4%), with per-task effects from -20 pp (pour) to +20 pp (boxes) cancelling out.

Conclusion

No benefit, but the BIN baseline used wider bounds than the paper. Next: use the paper's 1/99 bounds with clipping for BIN.