Cannot find '_policies' module when running OpenAI gym example

Other Questions OpenAI
1

I’m trying to execute a Cross-Entropy Method example from OpenAI gym but getting an import error. Here’s my simplified code:

import gym
import numpy as np
from _policies import BinaryActionLinearPolicy

def cross_entropy_method(eval_func, initial_params, batch_count, iterations, top_fraction):
    selected_count = int(np.round(batch_count * top_fraction))
    param_std = np.ones_like(initial_params)
    
    for step in range(iterations):
        samples = np.array([initial_params + variation for variation in param_std[None,:] * np.random.randn(batch_count, initial_params.size)])
        scores = np.array([eval_func(sample) for sample in samples])
        best_indices = scores.argsort()[::-1][:selected_count]
        top_samples = samples[best_indices]
        initial_params = top_samples.mean(axis=0)
        param_std = top_samples.std(axis=0)
        yield {'scores': scores, 'mean_params': initial_params, 'avg_score': scores.mean()}

def run_episode(policy, environment, max_steps):
    total_reward = 0
    state = environment.reset()
    for step in range(max_steps):
        action = policy.act(state)
        state, reward, finished, info = environment.step(action)
        total_reward += reward
        if finished:
            break
    return total_reward

env = gym.make('CartPole-v0')

def evaluate_policy(parameters):
    policy = BinaryActionLinearPolicy(parameters)
    reward = run_episode(policy, env, 200)
    return reward

When I run this, I get: ModuleNotFoundError: No module named '_policies'. I tried installing the policies package but that didn’t work. I also tried importing without the underscore but got another error about BinaryActionLinearPolicy not being found. What’s the correct way to handle this import?

2

the _policies mod isn’t something gym gives u, it’s somethin custom. u gotta either make the BinaryActionLinearPolicy urself or find where it came from. simple fix: just make a class with an act method that returns binary actions.

3

This error happens because _policies isn’t part of OpenAI gym - it’s not a standard module. I’ve run into this before with older tutorials where someone made their own custom policy. The BinaryActionLinearPolicy class is probably just a simple wrapper that takes parameters and implements a linear policy for binary actions. You’ll need to write this class yourself. It needs an __init__ method for the parameters and an act method that computes actions from the current state. For CartPole, you’d typically do a dot product between the state vector and your parameters, then use a threshold or sign function to get binary actions. Check if the original source has the policy implementation in another file.