We ran controlled experiments in a simplified simulated setting to test how distractors affect training of latent action models (now popular in generalist robot models pre-training pipelines). Even small models trained on filtered vs. unfiltered data show meaningful differences — suggesting a fundamental issue that could scale poorly for generalist robot models trained on web-scale datasets.
