I’m working with OpenAI gym environments and noticed that many games have two versions available. Take Pong as an example - there’s Pong-v0 which gives you image observations and Pong-ram-v0 which provides RAM state data.
When I use Pong-ram-v0, I get observations as arrays with 128 elements. But with the regular Pong-v0, the observations are images with dimensions 160x210 pixels.
My question is: Is there a way to convert the visual observation from Pong-v0 (the 160x210 image) into the same format as Pong-ram-v0 observations (the 128-element array)?
What I want to do is train an agent using the RAM version of the environment, but then show it playing using the visual version so I can actually see what’s happening on screen. Has anyone figured out how to do this conversion between the two observation formats?
No, you can’t directly convert between RAM and visual observations. The 128-element RAM array captures the entire Atari system’s memory state, while the 160x210 image is just what you see on screen - they’re completely different data types. Instead of converting observations, just run your trained RAM agent on the visual environment. Most gym environments allow you to load the same model across different observation formats for the same game since the core mechanics don’t change. You’ll probably need a simple wrapper to handle the format differences, but the action space should work fine.
this question pops up a lot, but there’s really no simple solution. ram observations hold the internal game state data that can’t be easily extracted from pixels. best bet is to train on ram and then render those actions visually. they’ll stay in sync since it’s the same game.
yea, for sure! converting visuals to ram data is super complex. u’d hav to analyze pixels to extract relevant game state, which is hard. not a lot of straightforward methods for this stuff. gl with it!
Skip the format conversion entirely. The real problem is workflow automation.
You need a system that switches between environments automatically during training and visualization. I’ve hit this exact issue building RL pipelines.
Automate everything. Train your agent on RAM data, then auto-deploy it to the visual environment for demos. Build a pipeline that handles environment switching, model loading, and visualization recording without you touching anything.
I built something like this with Latenode. It trains the agent on RAM data, saves checkpoints, then switches to visual mode for performance videos. Runs on schedule and sends notifications when it hits training milestones.
Bonus: you can automate comparison runs between different training approaches, auto-generate performance reports, and trigger retraining when performance tanks.
Why mess with complex data conversions when you can just automate the workflow right?
I encountered a similar issue during a project I worked on. The key takeaway is that converting visual data back to RAM format isn’t feasible without delving deep into the emulator’s workings, which is quite complex. What I found effective was to shift focus away from conversion. Instead, train the agent using the RAM version, then create a simple wrapper to showcase the visual output during evaluation. Both versions operate on the same game mechanics, allowing you to execute the agent’s actions in RAM while displaying the visuals in real-time. This way, you avoid the challenges of converting and maintain synchronization effortlessly.