OAT

We implemented OAT and QueST on the paper’s shared autoregressive backbone, ported from the authors’ code. The two arms differ only in the action tokenizer, and both decode all 8 tokens over the paper’s 1,000-entry codebook. This round retrains both arms after the first round, in which both rarely succeeded. Differences from the paper:

• Pretrained visual encoder. Both arms use an ImageNet-pretrained ResNet-18 with BatchNorm and ImageNet input normalization, instead of the paper’s randomly initialised ResNet with GroupNorm.
• One observation frame. Both arms see one frame instead of two, because station serving sends one observation per request.
• Two arms and three cameras. Actions are 20-D end-effector poses for two arms at 30 Hz with three cameras, instead of a 7-D single arm with one webcam.
• Chunk-relative actions. Positions and rotations in each chunk are expressed relative to the current end-effector pose, while gripper widths stay absolute.
• Larger crop. Random crops keep 116 of 128 pixels instead of 76.
• Checkpoint selection. Both arms hold out every tenth demonstration, train for at most 8,000 steps and serve the checkpoint with the lowest held-out token loss, instead of selecting checkpoints by simulated success.
• Blend execution. Both arms run on the station’s blend runtime, which crossfades overlapping chunks, instead of executing 16 of 32 actions and replanning synchronously.
• QueST loss. QueST’s reconstruction loss is computed on normalized actions.