Nvidia B300 GPUs Run New NVFP4 Fix for Robot Video Model Quantization

World action models, the systems that let a robot watch video and decide what to do next, typically reuse a pretrained video VAE whose compressed latents feed a frozen action policy. The paper argues that squeezing that VAE down to Nvidia’s NVFP4 format is not a simple accuracy tradeoff, because the policy depends on the exact shape of the latent space, not just on how good the reconstructed video looks. Researchers found that running the VAE’s Wan2.1 model straight through NVFP4 left its video quality score on VBench-7 almost untouched against the full precision version, yet the downstream LIBERO robot benchmark fell apart.

The authors trace the failure to how quantizing and dequantizing activations change the decoder’s gradient signal. That altered signal gets sent back to the encoder during joint training, nudging the latent space away from where the frozen policy expects it to sit, a drift that keeps compounding rather than settling down. Their fix, called LatentQuant, splits training into two stages: it first lines up the quantized encoder with its full precision counterpart, then locks the encoder and lets only the decoder adapt around it. Tested across both Wan2.1 and Wan2.2, the method holds task success close to the unquantized baseline on two separate robot benchmarks, while still running the lower precision format on Nvidia B300 hardware.

The numbers

• Direct NVFP4 quantization dropped LIBERO task success from a 95.5 percent baseline to 10.5 percent.
• LatentQuant recovered LIBERO success to 95.75 percent across the two tested Wan models.
• On the separate RoboTwin benchmark, LatentQuant reached 68.8 percent success.
• NVFP4 execution on Nvidia B300 GPUs ran 1.17x to 1.26x faster end to end than BF16 cuDNN.

The result is a caution for anyone reading quantization papers purely through video or image fidelity metrics. A model can keep looking almost unchanged to a human or to a reconstruction score while the information a downstream policy actually reads has shifted enough to break the system it feeds. For teams weighing Nvidia’s NVFP4 path for robotics inference specifically to capture the B300 speedup, the paper’s own comparison suggests that a reconstruction-only sign-off is not sufficient evidence that a quantized checkpoint is safe to deploy in a closed control loop.