OAT

We implemented OAT and QueST on the paper’s shared autoregressive backbone, ported from the authors’ code. The two arms differ only in the action tokenizer, and both decode all 16 tokens. Differences from the paper:

• Token budget. Both tokenizers compress each chunk into 16 tokens over the paper’s 1,000-entry codebook instead of 8, doubling the token budget because our policy controls two arms instead of the paper’s one. QueST halves its temporal downsampling to produce the 16 tokens.
• Pretrained visual encoder. Both arms use an ImageNet-pretrained ResNet-18 with BatchNorm and ImageNet input normalization, instead of the paper’s randomly initialised ResNet with GroupNorm. This follows review feedback after the first round, in which both arms rarely succeeded.
• One observation frame. Both arms see one frame instead of two, because station serving sends one observation per request.
• Two arms and three cameras. Actions are 20-D end-effector poses for two arms at 30 Hz with three cameras, instead of a 7-D single arm with one webcam.
• Chunk-relative actions. Positions and rotations in each chunk are expressed relative to the current end-effector pose, while gripper widths stay absolute.
• Larger crop. Random crops keep 116 of 128 pixels instead of 76.
• Checkpoint selection. Both arms hold out every tenth demonstration, train for at most 8,000 steps and serve the checkpoint with the lowest held-out position error of sampled actions, instead of selecting checkpoints by simulated success.
• Blend execution. Both arms run on the station’s blend runtime, which crossfades overlapping chunks, instead of executing 16 of 32 actions and replanning synchronously.
• QueST loss. QueST’s reconstruction loss is computed on normalized actions.