PaperManiFlow
ManiFlow: A General Robot Manipulation Policy via Consistency Flow Training
Claim tested: Full RGB ManiFlow's task-success advantage over RGB Diffusion Policy, as the paper reports it, transfers to real-robot tasks.
Tested
Result
Improvement over baseline
Inconclusive
- Reported
- +17 pts
- Ours
- +11 pts
- Gain vs baseline
- 95% CI, −2 pts to +24 pts
- Paper's reported gain
- No difference
Success rate
- Reported
- 57%
- Ours
- 77%
- Measured on our stations
- 95% CI, 67%–84%
- Reported in the paper
Across iterations
By task
| Task | BaselineDiffusion Policy | MethodManiFlow | Improvement |
|---|---|---|---|
| e01Place loaded boxes in container (bimanual) | 27%8/30 | 63%19/30 | +37 pts+11 to +56 |
| e02Place spray bottles upright, nozzle forward (bimanual handover) | 97%29/30 | 90%27/30 | −7 pts−23 to +8 |
| e03Pour screws into the funnel | 73%22/30 | 77%23/30 | +3 pts−18 to +25 |
| All tasks | 66%59/90 | 77%69/90 | +11 pts−2 to +24 |
| Reported in the paper | 39% | 57% | +17 pts |
Rollouts
One clip per environment from this entry's evaluation, beside where each evaluation episode placed the environment's objects, coloured by its rating. Use the button beside each environment to open its evaluation dataset in the dataset viewer.
e03Pour screws into the funnel BaselineDiffusion Policy22/3073%
Show in dataset viewer of Diffusion Policy on Pour screws into the funnelMethodManiFlow23/3077%
Show in dataset viewer of ManiFlow on Pour screws into the funnel- Held-out benchmark environment
e01Place loaded boxes in container (bimanual) BaselineDiffusion Policy8/3027%
Show in dataset viewer of Diffusion Policy on Place loaded boxes in container (bimanual)MethodManiFlow19/3063%
Show in dataset viewer of ManiFlow on Place loaded boxes in container (bimanual) - Held-out benchmark environment
e02Place spray bottles upright, nozzle forward (bimanual handover) BaselineDiffusion Policy29/3097%
Show in dataset viewer of Diffusion Policy on Place spray bottles upright, nozzle forward (bimanual handover)
Implementation Details
What we changed
We ported ManiFlow’s DiT-X policy and its consistency flow training from the official code and compare it with the platform’s Diffusion Policy. ManiFlow samples actions in 10 steps and conditions on a single token that concatenates the robot state with every camera’s features. Both arms share the ImageNet ResNet-18 backbone, data, the paper’s image augmentation, a 32-step horizon and the training budget. Differences from the paper:
- Ran the paper’s RGB comparison against Diffusion Policy, which it reports only in simulation, on real robots instead of its point-cloud setup against 3D Diffusion Policy.
- Fed DiT-X’s cross-attention one conditioning token instead of multiple visual tokens.
- Sampled in 10 steps instead of the 1 to 2 steps behind the paper’s headline result.
- Used an ImageNet-pretrained ResNet-18 in both arms instead of training one from scratch.
- Conditioned on one observation frame instead of two, because the station sends one observation per request.
- Predicted and executed 32-step action chunks, blended on the station, instead of 64-step chunks with temporal ensembling.
- Served the online weights in both arms instead of the EMA weights the official code evaluates.
- Controlled absolute end-effector poses with a parallel gripper instead of joints and dexterous hands.
- Kept the platform’s Diffusion Policy preset for the control, with a wider U-Net and 16 DDIM steps, rather than the original real-robot configuration.
Prompt summary
Reproduce the paper's comparison of full RGB ManiFlow against RGB Diffusion Policy on 3 real-robot tasks of your choice, training in the cloud and evaluating on stations. (Ported ManiFlow with 10 inference steps and a single conditioning token; the control is the platform Diffusion Policy preset with the paper's image augmentation.)
Response summary
ManiFlow 76.7% vs DP 65.8% SR overall (+11.1 pp, 95% CI -1.2 to +23.4, p = 0.12; AP 89.7% vs 86.3%), with the gain coming almost entirely from boxes (19/30 vs 8/30); pour tied (23/30 vs 23/31) and spray slightly favoured DP (27/30 vs 29/30).
Conclusion
ManiFlow ahead (+11 pp) but not significant, and the port deviated from the official code. Next: fix conditioning, inference steps, horizon and EMA, and swap spray (no headroom) for toolbox.