AttributeError when Training with TensorFlow and OpenAI Gym

[Alex_Thunder]

I’m encountering an error while trying to run a policy gradient reinforcement learning script using TensorFlow and OpenAI Gym. The error arises during the training process when the environment attempts to render.

Here’s the initial setup code I have:

import tensorflow as tf
import gym
import numpy as np
import matplotlib.pyplot as plt
import os

env = gym.make('CartPole-v1')

This is my neural network implementation:

class PolicyNetwork:
    def __init__(self, action_count, observation_size):
        init = tf.contrib.layers.xavier_initializer()
        
        self.state_input = tf.placeholder(dtype=tf.float32, shape=[None, observation_size])
        
        layer1 = tf.layers.dense(self.state_input, 16, activation=tf.nn.relu, kernel_initializer=init)
        layer2 = tf.layers.dense(layer1, 16, activation=tf.nn.relu, kernel_initializer=init)
        
        logits = tf.layers.dense(layer2, action_count, activation=None)
        
        self.action_probs = tf.nn.softmax(logits)
        self.predicted_action = tf.argmax(self.action_probs, axis=1)
        
        self.reward_input = tf.placeholder(shape=[None, ], dtype=tf.float32)
        self.action_input = tf.placeholder(shape=[None, ], dtype=tf.int32)
        
        action_onehot = tf.one_hot(self.action_input, action_count)
        
        cross_ent = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=action_onehot)
        
        self.policy_loss = tf.reduce_mean(cross_ent * self.reward_input)
        
        self.grads = tf.gradients(self.policy_loss, tf.trainable_variables())
        
        self.grad_placeholders = []
        for idx, var in enumerate(tf.trainable_variables()):
            grad_ph = tf.placeholder(tf.float32)
            self.grad_placeholders.append(grad_ph)
        
        opt = tf.train.AdamOptimizer(learning_rate=1e-2)
        self.apply_grads = opt.apply_gradients(zip(self.grad_placeholders, tf.trainable_variables()))

I also created a reward processing function:

gamma = 0.95

def process_rewards(reward_list):
    processed = np.zeros_like(reward_list)
    running_total = 0
    
    for idx in reversed(range(len(reward_list))):
        running_total = running_total * gamma + reward_list[idx]
        processed[idx] = running_total
    
    processed -= np.mean(processed)
    processed /= np.std(processed)
    
    return processed

The issue arises in my training loop when I execute env.render(). The complete error message is AttributeError: 'NoneType' object has no attribute 'set_current', which suggests a problem in the pyglet display system.

I’m using Windows with TensorFlow 1.4.0, OpenAI Gym 0.10.5, and Python 3.6.5. Has anyone else faced this rendering issue? How might I resolve the context error, or would it be better to skip rendering calls during training?

[SwimmingShark]

This AttributeError happens because pyglet’s graphics backend won’t initialize properly on Windows. I’ve hit the same issue running training scripts through certain dev environments or remote desktop connections. Usually it’s Windows blocking direct OpenGL access or your display drivers messing with pyglet’s context creation. I fixed mine by installing the latest Visual C++ Redistributables and updating graphics drivers completely. You can also force pyglet to use software rendering with os.environ['PYGLET_WINDOW'] = 'xlib' before importing gym. But honestly, since you’re doing policy gradient training, rendering just kills performance anyway. Set up a training flag that disables all render calls during learning and only shows visualization when evaluating your final policy. Your neural network architecture looks good though - don’t let this display bug derail your RL work.

[Sky24]

This rendering error happens all the time on Windows, especially with headless setups or virtual environments. It’s a pyglet context issue from OpenGL initialization problems - nothing wrong with your TensorFlow code. I’ve hit this same problem training agents on Windows servers. Easiest fix? Wrap your render calls in try-except and skip rendering when it fails. The important part: this doesn’t break training at all. The environment still works fine for observations and rewards. What worked for me was switching render mode to ‘rgb_array’ instead of human mode, then saving frames occasionally rather than displaying real-time. You can also try updating pyglet or downgrading to version 1.3.2 - some versions play nicer with Windows. For actual training runs, I just disable rendering completely since it slows everything down anyway. Your policy gradient code looks good, so don’t let this display bug mess with your head.

[JumpingRabbit]

Had this exact AttributeError with gym environments on Windows. It’s usually OpenGL contexts clashing when pyglet tries to create the rendering window. Happens a lot in IDEs or remote desktop sessions where graphics get confused. Fixed it by setting an environment variable before importing gym: os.environ['SDL_VIDEODRIVER'] = 'windib'. You can also try setting the render_mode parameter explicitly or check for conflicting graphics drivers. Running from command line instead of PyCharm/Jupyter worked too - those IDEs mess with OpenGL context sometimes. If you’re just training and don’t need visuals, skip render calls during training. The environment trains fine without rendering, and you can test with visuals once your agent’s ready.

[mikechen]

I hit this same AttributeError when building policy gradient agents on Windows. It’s usually because pyglet can’t set up the OpenGL context properly - happens a lot with certain IDEs or integrated graphics that don’t play nice with OpenGL. I fixed it by installing Mesa OpenGL drivers as backup. Just download the Mesa3D libraries and drop them in your Python environment’s DLL path. You can also try setting render_mode parameters when you create the gym environment instead of calling render() later. If you need to see what’s happening, use matplotlib to plot episode rewards and action distributions instead of gym’s rendering. Honestly, training runs way faster without the rendering overhead anyway, so this might actually help your workflow.

[Zack_45Gaming]

classic windows pyglet issue! add import os; os.environ['DISPLAY'] = ':0' before importing gym, or just comment out the render() calls. your rl training doesn’t need visuals anyway and runs way faster without rendering. the error’s pyglet trying to grab a missing OpenGL context - nothing wrong with your tensorflow setup.