Ah, but how do you know the simulation doesn't just slow down the simulated flow of time so it can keep up? Think of a bullet-time like concept. The universe can be simulated as fast as the underlying hardware can keep up.
But if you're enmeshed in the simulation, you would never notice the slowdown! The only reason the video game is annoying is because we experience time outside of it.