The Reinforcement Learning Toybox

A set of games designed for testing deep RL agents.

This is the public release repository. It contains:

1. The rust code for building Breakout.
2. The Python API to the Rust game.
3. The Python OpenAI Environment for training RL agents.

The current release does not contain the testing framework, nor the tests themselves. We will release the testing framework, tests, and a larger suite of games upon acceptance as an archival, peer-reviewed venue (i.e., conference proceedings or journal publication).

If you use this code, or otherwise are inspired by our white-box testing approach, please cite our NeurIPS workshop paper:

@inproceedings{foley2018toybox,
  title={{Toybox: Better Atari Environments for Testing Reinforcement Learning Agents}},
  author={Foley, John and Tosch, Emma and Clary, Kaleigh and Jensen, David},
  booktitle={{NeurIPS 2018 Workshop on Systems for ML}},
  year={2018}
}

Setup and Dependencies

Target Rust Version (1.28)

For this project, we are currently target rustc 1.28, here's how to make sure you're using that version (important for contributing) any newer version should also build the code just fine.

rustup override set 1.28.0

Mac Dev Setup Instructions

• brew install rustup
• source $HOME/.cargo/env
• rustup-init with the default install
• clone this repo

Lints and Formatting in Rust

The best rust tools require the nightly compiler (because they don't want to stabilize the compiler internals yet). Follow the readme instructions to get rustfmt and clippy.

Then you can check automatically format your files with cargo +nightly fmt and peruse the best lints with cargo +nightly clippy.

A pre-commit hook will ensure that your code is always properly formatted. To do this, run

git config core.hooksPath .githooks

from the top-level directory. This will ensure that your files are formatted properly pior to committing.

Python

Tensorflow, OpenAI Gym, OpenCV, and other libraries may or may not break with various Python versions. We have confirmed that the code in this repository will work with the following Python versions:

• 3.5
• 3.6

Build

Compile code and dependencies (takes a while the first time) in release mode.

cargo build --release

Developing New Games

Please contact us if you would like to see new games! We also have a slack team. Since Slack doesn't allow people to request access to teams at this time, we ask that you sign up using a Google form.