Why does OpenAI Gym execute slowly on Windows compared to Linux?

Other Questions ai employee
1

I’m having a weird issue with OpenAI Gym where the execution speed differs between operating systems. On my Linux machine, the environment runs at maximum speed without any delays. However, when I use the same exact code on Windows, it only runs in real-time which is much slower.

I’m testing this with the Atari Montezuma’s Revenge environment. What’s strange is that my Windows machine actually has better hardware specs than the Linux one, so it should be faster if anything.

Here’s my training loop:

for episode in range(total_episodes):
    state = environment.reset()
    state = np.expand_dims(state, axis=0)
    step_count = 0
    while True:
        step_count += 1
        chosen_action = model.predict_action(state, current_reward, is_done)
        next_state, current_reward, is_done, _ = environment.step(chosen_action)
        next_state = np.expand_dims(next_state, axis=0)
        model.store_experience(state, chosen_action, current_reward, next_state, is_done)

        state = next_state
        environment.render()
        if is_done or step_count >= 1000:
            print("Episode: {}/{}, Steps: {}, Exploration: {:.3}"
                  .format(episode, total_episodes, step_count, model.exploration_rate))
            if len(model.experience_buffer) > training_batch_size:
                model.train_on_batch(training_batch_size)
            break

Is there some configuration setting I’m missing for Windows?

2

It’s probably the environment.render() call in your loop. Windows often enforces frame rate limiting during rendering, which caps your speed to match the game’s original timing. Comment out the render() line and see if training speeds up - I bet it will. You can also try render(mode=‘rgb_array’) instead of the default human mode to skip display sync. Or just render every 10th episode since you don’t need to watch every single one anyway. The OS performance difference usually comes down to display backend handling, not the actual computation.

3

could be the graphics drivers causing issues. windows doesn’t always handle opengl well, which can slow down rendering. try disabling vsync and make sure your gpu drivers are up to date. had similar problems with atari envs on win10.