I’m facing an error when executing a reinforcement learning code with TensorFlow and OpenAI Gym. The issue arises when the environment attempts to render the output visually.
Setup Code:
Creating the Environment:
import tensorflow as tf
import gym
import numpy as np
import matplotlib.pyplot as plt
import os
game_environment = gym.make('CartPole-v1')
Basic Agent Class:
class SimpleAgent:
def __init__(self, action_size, state_dimension):
weight_initializer = tf.contrib.layers.xavier_initializer()
self.input_state = tf.placeholder(dtype=tf.float32, shape=[None, state_dimension])
# Layers of the neural network
hidden_layer1 = tf.layers.dense(self.input_state, 16, activation=tf.nn.relu, kernel_initializer=weight_initializer)
hidden_layer2 = tf.layers.dense(hidden_layer1, 16, activation=tf.nn.relu, kernel_initializer=weight_initializer)
# Output layer setup
action_logits = tf.layers.dense(hidden_layer2, action_size, activation=None)
self.action_probabilities = tf.nn.softmax(action_logits)
self.selected_action = tf.argmax(self.action_probabilities, axis=1)
# Establishing the training input
self.reward_placeholder = tf.placeholder(shape=[None, ], dtype=tf.float32)
self.action_placeholder = tf.placeholder(shape=[None, ], dtype=tf.int32)
action_one_hot = tf.one_hot(self.action_placeholder, action_size)
loss = tf.nn.softmax_cross_entropy_with_logits(logits=action_logits, labels=action_one_hot)
self.loss_value = tf.reduce_mean(loss * self.reward_placeholder)
Training Process:
num_episodes = 500
max_steps_per_episode = 5000
with tf.Session() as session:
session.run(tf.global_variables_initializer())
for episode in range(num_episodes):
state = game_environment.reset()
for step in range(max_steps_per_episode):
if episode % 10 == 0:
game_environment.render() # ERROR OCCURRING HERE
# Calculate action probabilities
action_probs = session.run(simple_agent.action_probabilities,
feed_dict={simple_agent.input_state: [state]})
action_selected = np.random.choice(range(action_size), p=action_probs[0])
next_state, reward, done, _ = game_environment.step(action_selected)
state = next_state
if done:
break
Error Description:
AttributeError: 'NoneType' object has no attribute 'set_current'
This problem arises in the rendering process when game_environment.render() is executed. The traceback indicates the error occurs within the pyglet window rendering functions.
System Information:
- HP Pavilion laptop
- Intel i7 CPU
- 8GB RAM
- AMD Radeon graphics
- TensorFlow version 1.4.0
- OpenAI Gym version 0.10.5
- Python version 3.6.5
The training algorithm runs smoothly if I exclude the render calls, but I would like to visualize the output. Has anyone else experienced this rendering problem?