CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents

[Arxiv Paper] [PDF] [Docs] [LLM Agents] [Tensor Agent]

CivRealm is an interactive environment for the open-source strategy game Freeciv-web based on Freeciv, a Civilization-inspired game. Within CivRealm, we provide interfaces for two typical agent types: tensor-based reinforcement learning agents (see Tensor-agent Repo) based on the Gymnasium API, and language-based agents (see LLM-agent Repo) powered by language models.

We also provide a set of tools for training and evaluating agents, as well as a set of baselines for both agent type. We hope that CivRealm can serve as a testbed for developing and evaluating agents that can learn and reason in complex environments. For detailed usages on the CivRealm API, please refer to Documentation.

About

CivRealm is developed based on freeciv-bot, depending on freeciv-web and FCIV-NET. Moving forward, CivRealm will be maintained by BIGAI in the long term.

Prerequisites

In order to test the civrealm on http://localhost, kindly follow the docker installation instructions on https://bigai-ai.github.io/civrealm/getting_started/requirements.html.

⚠️ Please make sure you have installed the latest docker engine and docker-compose. Using older versions of docker may result in unexpected errors.

Installation

Installation for CivRealm developers

cd civrealm
pip install -e .

Testing the installation

To test if the installation is successful, run

test_civrealm

To test with multiple players, run

test_civrealm --minp=2 --username=myagent

Then in another terminal, run

test_civrealm --username=myagent1

Trouble shooting

The following are some common issues that you may encounter when running the code. If you encounter any other issues, please feel free to open an issue.

If firefox keeps loading the page, please try to add the following line to /etc/hosts:
```
127.0.0.1 maxcdn.bootstrapcdn.com
127.0.0.1 cdn.webglstats.com
```

If you encounter the following error when running sudo civ_prep_selenium.sh, please try this solution.

...
The following packages have unmet dependencies:
containerd.io : Conflicts: containerd
                Conflicts: runc
...

If you see the following error when running test_civrealm, please see this solution. If this does not solve the problem, please check geckodriver.log for more information.
```
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 1
```
One potential solution on Ubuntu 22.04 is:
```
sudo apt install firefox-geckodriver
ln -s /snap/bin/firefox.geckodriver geckodriver
```
If you see the following error when setting take_screenshot: True, it is caused by snap version of Firefox. Please try System Firefox installation.
```
Your Firefox profile cannot be loaded. 
It may be missing or inaccessible.
```
If the screenshot is not centered on the location of your first unit, it is because you are using multiple displays. Please ensure the Firefox browser for screenshot pops up on your primary display.

Check out our paper

Our paper is available on Arxiv. If you find our code or databases useful, please consider citing us!

@inproceedings{qi2024civrealm,
  title     = {CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents},
  author    = {Siyuan Qi and Shuo Chen and Yexin Li and Xiangyu Kong and Junqi Wang and Bangcheng Yang and Pring Wong and Yifan Zhong and Xiaoyuan Zhang and Zhaowei Zhang and Nian Liu and Wei Wang and Yaodong Yang and Song-Chun Zhu},
  booktitle = {International Conference on Learning Representations},
  year      = {2024},
  url       = {https://openreview.net/forum?id=UBVNwD3hPN}
}