In aggregate, it wouldn't much matter since randomness would cancel itself out. So broad index funds wouldn't see that volatility.

At shorter scales, however, randomness like that would add volatility to individual stocks, so you'd see more random results for returns among small portfolios -- thus creating quite a few genuinely lucky fund managers.