PaperManiFlow
ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training
Claim tested: Full RGB ManiFlow's task-success advantage over RGB Diffusion Policy, as the paper reports it, transfers to real-robot tasks.
Tested
Result
Improvement over baseline
Inconclusive
- Reported
- +17 pts
- Ours
- +8 pts
- Gain vs baseline
- 95% CI, −7 pts to +22 pts
- Paper's reported gain
- No difference
Success rate
- Reported
- 57%
- Ours
- 51%
- Measured on our stations
- 95% CI, 41%–61%
- Reported in the paper
Across iterations
By task
| Task | BaselineDiffusion Policy | MethodManiFlow | Improvement |
|---|---|---|---|
| e01Place loaded boxes in container (bimanual) | 47%14/30 | 57%17/30 | +10 pts−15 to +33 |
| e02Pour screws into the funnel | 83%25/30 | 80%24/30 | −3 pts−23 to +17 |
| e03Put tools in standing toolbox | 0%0/30 | 17%5/30 | +17 pts+2 to +34 |
| All tasks | 43%39/90 | 51%46/90 | +8 pts−7 to +22 |
| Reported in the paper | 39% | 57% | +17 pts |
Rollouts
One clip per environment from this entry's evaluation, beside where each evaluation episode placed the environment's objects, coloured by its rating. Use the button beside each environment to open its evaluation dataset in the dataset viewer.
e02Pour screws into the funnel BaselineDiffusion Policy25/3083%
Show in dataset viewer of Diffusion Policy on Pour screws into the funnelMethodManiFlow24/3080%
Show in dataset viewer of ManiFlow on Pour screws into the funnel- Held-out benchmark environment
e01Place loaded boxes in container (bimanual) BaselineDiffusion Policy14/3047%
Show in dataset viewer of Diffusion Policy on Place loaded boxes in container (bimanual)MethodManiFlow17/3057%
Show in dataset viewer of ManiFlow on Place loaded boxes in container (bimanual) - Held-out benchmark environment
e03Put tools in standing toolbox BaselineDiffusion Policy0/300%
Show in dataset viewer of Diffusion Policy on Put tools in standing toolboxMethodManiFlow5/3017%
Show in dataset viewer of ManiFlow on Put tools in standing toolbox
Implementation Details
What we changed
We ported ManiFlow’s DiT-X policy from the official code and compare it with the platform’s Diffusion Policy. ManiFlow conditions on 64 feature-map tokens from each of three cameras plus a state token, and samples actions in 2 steps. Both arms share the ImageNet ResNet-18 backbone, data, image augmentation, a 64-step horizon, the training budget and EMA weights. Differences from the paper:
- Ran the paper’s RGB comparison against Diffusion Policy, which it reports only in simulation, on real robots instead of its point-cloud setup against 3D Diffusion Policy.
- Used an ImageNet-pretrained ResNet-18 in both arms instead of training one from scratch.
- Conditioned on one observation frame instead of two, because the station sends one observation per request.
- Executed 32 steps of each 64-step prediction and blended successive plans on the station instead of temporal ensembling, assuming 30 Hz because the paper doesn’t state its control rate.
- Controlled absolute end-effector poses with a parallel gripper instead of joints and dexterous hands.
- Kept the platform’s Diffusion Policy preset for the control, with a wider U-Net and 16 DDIM steps, rather than the original real-robot configuration.
- Trained one seed per arm and evaluated its final checkpoint instead of averaging three seeds.
Prompt summary
Retrain both arms fresh on all three tasks with six fixes: multi-token conditioning following the official observation encoder, 2 inference steps for ManiFlow, horizon 64 for both with blending, EMA weights served in both arms, replace the spray handover task with put_tools_in_standing_toolbox, and commit a parameter checklist. Run fresh reviews and a 2-step vs 10-step offline check, and wait for approval before launch.
Response summary
ManiFlow 51.1% vs DP 43.3% SR overall (+7.8 pp, 95% CI -4.2 to +19.7, p = 0.27; AP 70.7% vs 64.7%, +5.9 pp, p = 0.058), driven mainly by toolbox (5/30 vs 0/30); pour 24/30 vs 25/30 and boxes 17/30 vs 14/30.
Conclusion
ManiFlow still ahead (+7.8 pp, not significant), with the gain concentrated on toolbox; also about 3× faster to plan. Concluded.