https://github.com/agarcl/agarcl

Agar.io for Continual Reinforcement Learning
https://github.com/agarcl/agarcl

continual-reinforcement-learning cplusplus deep-reinforcement-learning games python reinforcement-learning representation-learning

Last synced: 2 months ago
JSON representation

Agar.io for Continual Reinforcement Learning

Host: GitHub
URL: https://github.com/agarcl/agarcl
Owner: AgarCL
License: mit
Created: 2025-05-15T03:44:56.000Z (about 1 year ago)
Default Branch: main
Last Pushed: 2025-05-16T02:43:23.000Z (about 1 year ago)
Last Synced: 2025-06-15T03:13:56.664Z (about 1 year ago)
Topics: continual-reinforcement-learning, cplusplus, deep-reinforcement-learning, games, python, reinforcement-learning, representation-learning
Language: C++
Homepage: https://agarcl.github.io/
Size: 7.88 MB
Stars: 1
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# AgarCL

A research platform for continual RL that allows for a progression of increasingly sophisticated behaviour.

**AgarCL is based on the game Agar.io.** It's a non-episodic, high-dimensional problem featuring stochastic, ever-evolving dynamics, continuous actions, and partial observability.

## Installation instructions

The recommended way to use AgarCL is within a Docker container running a Linux OS. This ensures there are no conflicts with other installed packages or platforms. This installation script will allow you to interact with AgarCL in a headless mode.

### Setting up the container
Follow these steps to set up the container:

1. **Download the Dockerfile**
- Download the [Dockerfile.txt](https://github.com/AgarCL/AgarCL/blob/master/Dockerfile.txt).
- You can jump to step 4 for instructions to use a pre-built [image](https://hub.docker.com/repository/docker/agarcl/agarclimage/general) that we share.

2. **Navigate to the Directory Containing the Dockerfile**
- Open your terminal and navigate to the folder where the `Dockerfile.txt` is located:
```bash
cd /path/to/Dockerfile/directory
```

3. **Build the Docker Image**
- Build the Docker image by specifying the custom Dockerfile using the `-f` flag:
```bash
docker build -f Dockerfile.txt -t agarclimage .
```
- Skip to step 5, now that your image is built.
4. **Directly use the pre-built image**
- ```bash
docker pull agarcl/agarclimage
```

5. **Run the Docker Container**
- Once the image has been built, run the container:
```bash
docker run --gpus all -it --name agarclcontainer agarclimage
```
- This command will start the container with the name `agarclcontainer`. The `--gpus all` flag tells Docker to use all available GPUs on your host system for the container.

### Installing the AgarCL Platform

Now, let's install the platform on your system (`agarclcontainer` container):

1. **Clone the AgarCL Repository**
- Clone the repository with the `--recursive` flag to ensure all submodules are included:
```bash
git clone --recursive git@github.com:AgarCL/AgarCL.git
```

2. **Install the Platform**
- Change into the `AgarCL` directory:
```bash
cd AgarCL
```

- Run the installation command to set up the platform:
```bash
python setup.py install --user
```

- This will install the platform in your local user environment.

#### Done!

### Installing the AgarCL Platform and benchmarking tools

1. **Clone the AgarCL-benchmark Repository**
- Clone the repository:
```bash
git clone git@github.com:AgarCL/AgarCL-benchmark.git
```

2. **Navigate to the AgarCL-Benchmark Directory**
- Change into the `AgarCL-benchmark` directory:
```bash
cd AgarCL-benchmark
```

4. **Clone the AgarCLgit Repository**
- Clone the `AgarCL` repository with the `--recursive` flag to ensure all submodules are included:
```bash
git clone --recursive git@github.com:AgarCL/AgarCL.git
```

5. **Navigate to the AgarCL Directory**
- Change into the `AgarCL` directory:
```bash
cd AgarCL
```

6. **Install the Platform**
- Run the installation command to set up the platform:
```bash
python setup.py install --user
```

#### Done!

## macOS and Linux Installation Guide

### Installation notes:

Ensure the project is compiled with clang++, not g++

### macOS Installation Guide

> 💡 Before starting:
>
> Follow the instructions [here](https://brew.sh/) and make sure you have **homebrew** correctly installed and updated.
>
> Make sure **Command Line Tools** are installed properly, follow the [documentation](https://developer.apple.com/xcode/resources/).
>
> Note: The installer will automatically install **CMake 3.22** (a compatible version for this project)

Then follow these steps to set up the AgarCL environment on macOS:

1. **Clone the repository:**
```bash
git clone --recursive git@github.com:AgarCL/AgarCL.git
```
2. **Change into the project directory:**
```bash
cd AgarCL
```
3. **Create a Python virtual environment:**
```bash
python -m venv agarclenv
```
4. **Activate the virtual environment:**
```bash
source agarclenv/bin/activate
```
5. **Run the installer script:**
```bash
./install.sh
```
6. **Install Python dependencies:**
```bash
pip install -r requirements.txt
```
7. **Build & install the Python package:**
```bash
python setup.py install
```

#### Done!

### Linux Installation Guide

1. **Clone the repository:**
```bash
git clone --recursive git@github.com:AgarCL/AgarCL.git
```
2. **Change into the project directory:**
```bash
cd AgarCL
```
3. **Create a Python virtual environment:**
```bash
python -m venv agarclenv
```
4. **Activate the virtual environment:**
```bash
source agarclenv/bin/activate
```
5. **Make the install script executable:**
```bash
chmod +x install.sh
```

6. **Run the installer script (may require sudo):**
```bash
sudo ./install.sh
```
7. **Install Python dependencies:**
```bash
pip install -r requirements.txt
```
8. **Build and install the Python package:**
```bash
python setup.py install
```

## Running the code
To run the Go Bigger example, execute the following line:

```python
python project_path/bench/go_bigger_example.py
```

To run the Screen Observations example, execute the following line:

```python
python project_path/bench/screen_obs_example.py
```

## Using the environment

```python
import gymnasium as gym

# Initialise the environment
env = gym.make("agario-screen-v0", render_mode="human")

# Reset the environment to generate the first observation
observation = env.reset()
for _ in range(1000):
# this is where you would insert your policy
c_target_space = gym.spaces.Box(low=-1, high=1, shape=(2,))
d_target_space = gym.spaces.Discrete(3)
action = [(c_target_space.sample(), d_target_space.sample())]

# step (transition) through the environment with the action
# receiving the next observation, reward and if the episode has terminated or truncated
observation, reward, terminated, truncated, info = env.step(action)

# If the episode has ended then we can reset to start a new episode
if terminated or truncated:
observation = env.reset()

env.close()
```

### Self-Play setup

In order to play the game yourself or enable rendering in the gym environment, you will need to build the game
client yourself on a system where OpenGL has been installed. Issue the following commands:

```shell
git submodule update --init --recursive
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j 2 client agario
```

This will output an executable named client in the directory agario

```shell
agario/client
```

Use your cursor to control the agent.

### Loading and Saving Environment Snapshots

AgarCL allows you to save and load snapshots of the environment's state. This feature is useful for debugging, benchmarking, or resuming training from a specific point.

#### Saving a Snapshot

To save the current state of the environment, use the `save_env_state` method:

```python
env.save_env_state('path_to_save_snapshot.json')
```

This will save the environment's state to a JSON file at the specified path.

#### Loading a Snapshot

To load a previously saved snapshot, use the `load_env_state` method:

```python
env.load_env_state('path_to_snapshot.json')
```
Before loading a snapshot, ensure that the `load_env_state` option is enabled in the environment configuration. This will allow the environment to restore its state from the specified JSON file.

#### Example Usage

Here is an example of how to use these methods in a script:

```python
import gymnasium as gym

# Initialize the environment
env = gym.make("agario-screen-v0", render_mode="human")

# Load a snapshot if available
env.load_env_state('snapshot.json')

# Reset the environment
env.reset()

# Perform some steps
for _ in range(100):
action = [(env.action_space.sample(), env.action_space.sample())]
observation, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
break

# Save the environment's state
env.save_env_state('snapshot.json')

env.close()
```

This functionality ensures reproducibility and allows for efficient experimentation with different configurations.

### Recording and Saving Videos

AgarCL provides functionality to record and save videos of the environment's execution. This is useful for visualizing agent behavior or debugging.

#### Enabling Video Recording

To enable video recording, set the `record_video` parameter to `True` in the environment configuration. You can also enable video recording programmatically:

```python
env.enable_video_recorder()
```

#### Saving the Video

To save the recorded video, use the `generate_video` method:

```python
env.generate_video('path_to_save_video', 'video_name.avi')
```

This will save the video to the specified path with the given file name.

#### Disabling Video Recording

To stop recording, use the `disable_video_recorder` method:

```python
env.disable_video_recorder()
```

#### Example Usage

Here is an example of how to record and save a video:

```python
import gymnasium as gym

# Initialize the environment
env = gym.make("agario-screen-v0", render_mode="rgb_array")

# Enable video recording
env.enable_video_recorder()

# Reset the environment
env.reset()

# Save the video
env.generate_video('videos', 'example_run.avi')

# Disable video recording
env.disable_video_recorder()

env.close()
```

### Real-Time Render View

Display the environment in a live GUI window for debugging, demos, and visually tracking your agent’s decisions as they happen.

An example of how to invoke the window:

```python
import gymnasium as gym

# Initialize the environment
env = gym.make("agario-screen-v0", render_mode="human")

# Reset the environment
env.reset()

# Perform some steps
for _ in range(100):
action = [(env.action_space.sample(), env.action_space.sample())]
observation, reward, terminated, truncated, info = env.step(action)

# Update the on-screen display
env.render()
if terminated or truncated:
break

env.close()
```

This functionality allows you to capture and analyze the agent's performance visually.

## Acknowledgment
This implementation is built upon the [AgarLE repository](https://github.com/jondeaton/AgarLE).

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/agarcl/agarcl

Awesome Lists containing this project

README