Poke &Wiggle

Reproducing Robotics Research with AI Agents

We operate a large-scale robot farm with real-world benchmarking stations, simulation stations and dedicated on-site teams. Our agent-native platform gives AI agents API access to datasets, model training and experiments in both simulation and the physical world, while our team handles station preparation, resets and quality control.

Research replication puts this infrastructure to work: agents implement methods from papers and test selected claims on real robots, with human review and steering where needed.

See the workflow below, then explore how the improvements measured in our setup compare with those reported in each paper. Open any experiment for detailed results, adaptations and the agent’s trajectory.

How it works

AI agents run the workflow through our platform, with human review and steering where needed.

  1. 1. Identify the claimThe agent or a human identifies the claim to test and the paper’s experimental setup.
  2. 2. Select tasksThe agent selects suitable tasks from our farm and documents differences from the original setup.
  3. 3. Implement the methodThe agent uses the authors’ code where available or implements the method, documenting adaptations.
  4. 4. Train the modelsThe agent trains the baseline and proposed method, recording settings and departures from the paper.
  5. 5. Test on real robotsThe agent requests evaluations on matched object placements, then analyzes the results and uncertainty.

Papers

−200+20+40+60Open
+8 pts95% CI −9 to +25Paper: +36 ptsInconclusive
−16 pts95% CI −28 to −2Paper: +33 ptsNot replicated
+6 pts95% CI −7 to +18Paper: +10 ptsInconclusive
+6 pts95% CI −4 to +16Paper: +11 ptsInconclusive
+2 pts95% CI −7 to +12Paper: +33 ptsNot replicated
+6 pts95% CI −9 to +20Paper: +43 ptsNot replicated
+13 pts95% CI +2 to +24Paper: +13 ptsReplicated
+8 pts95% CI −7 to +22Paper: +17 ptsInconclusive
+22 pts95% CI +12 to +32Paper: +17 ptsReplicated
+2 pts95% CI −5 to +9Paper: +62 ptsNot replicated
+4 pts95% CI −9 to +17Paper: +17 ptsInconclusive
  • Gain vs baseline
  • 95% CI
  • Paper's reported gain
  • No difference