PaperCSP
Hierarchical Policy Learning via Spectral Decomposition
Claim tested: CSP improves real-robot task success over Diffusion Policy on precision-sensitive manipulation tasks.
Tested
Result
Improvement over baseline
Not replicated
- Reported
- +62 pts
- Ours
- +2 pts
- Gain vs baseline
- 95% CI, −5 pts to +9 pts
- Paper's reported gain
- No difference
Success rate
- Reported
- 86%
- Ours
- 6%
- Measured on our stations
- 95% CI, 2%–12%
- Reported in the paper
By task
| Task | BaselineDiffusion Policy | MethodCSP | Improvement |
|---|---|---|---|
| e01Clamp electrical component | 10%3/30 | 3%1/30 | −7 pts−23 to +8 |
| e02Connect DC jack | 0%0/30 | 0%0/30 | 0 pts−11 to +11 |
| e03Datum-corner alignment | 0%0/30 | 13%4/30 | +13 pts−1 to +30 |
| All tasks | 3%3/90 | 6%5/90 | +2 pts−5 to +9 |
| Reported in the paper | 24% | 86% | +62 pts |
Rollouts
One clip per environment from this entry's evaluation, beside where each evaluation episode placed the environment's objects, coloured by its rating. Use the button beside each environment to open its evaluation dataset in the dataset viewer.
e02Connect DC jack BaselineDiffusion Policy0/300%
Show in dataset viewer of Diffusion Policy on Connect DC jackMethodCSP0/300%
Show in dataset viewer of CSP on Connect DC jack- Held-out benchmark environment
e01Clamp electrical component BaselineDiffusion Policy3/3010%
Show in dataset viewer of Diffusion Policy on Clamp electrical componentMethodCSP1/303%
Show in dataset viewer of CSP on Clamp electrical component - Held-out benchmark environment
e03Datum-corner alignment BaselineDiffusion Policy0/300%
Show in dataset viewer of Diffusion Policy on Datum-corner alignmentMethodCSP4/3013%
Show in dataset viewer of CSP on Datum-corner alignment
Implementation Details
What we changed
We reimplemented CSP from the paper; the authors have not released code, so where the paper is silent we followed BAKU, whose transformer settings match the paper’s. This round compares CSP with Diffusion Policy on three precision-sensitive tasks: connecting the DC jack, clamping the electrical component and pushing the box into the datum corner. Both arms train on the same data with the same cameras, chunk length, batch size and number of training steps. Differences from the paper:
- Control. The control is Diffusion Policy, one of the paper’s baselines, trained on the same data as CSP with the platform’s standard settings.
- Two arms and three cameras. Our platform is bimanual with three cameras, instead of the paper’s single arm with a fixed and a wrist camera.
- Absolute pose actions. CSP decomposes chunks of 32 absolute end-effector poses, the same actions the control predicts, instead of the paper’s velocity-style actions.
- Fixed cutoff. The coarse stage predicts the 8 lowest of 32 frequency components, inside the range the paper reports as working well, because the paper’s automatic cutoff rule gives no usable answer on absolute poses.
- Shared image encoder. CSP uses Diffusion Policy’s image encoder, so the comparison isolates the action model.
- Gripper in the coarse stage. Gripper width stays in the time domain and is predicted by the coarse stage, which the paper does not specify.
- No language input. Neither arm receives the task instruction, because each task has a single fixed instruction.
- Execution. Both arms run on the station’s blend runtime, which requests the next plan 800 ms ahead and crossfades into it over 600 ms.
Prompt summary
Read the CSP paper (spectral coarse-to-fine action decomposition), implement CSP and a Diffusion Policy control, choose three precision-sensitive tasks, train in the cloud, evaluate on stations and compare. (Chosen tasks: connect_dc_jack, clamp_electrical_component and push_box_into_datum_corner; default blend runtime.)
Response summary
CSP vs DP success: datum 4/30 vs 0/30 (AP 13 vs 0), clamp 1/30 vs 3/30 (AP 36 vs 40), dc jack 0/30 vs 0/30 (AP 34 vs 0); no clear SR difference overall.
Conclusion
No clear difference; CSP ahead on datum, behind on clamp. Note: we could not find the paper's hypothesis in our data, that high-precision behaviour lives in the high-frequency components of the actions.