https://github.com/gdonald/reversi-py
Reversi with reinforcement learning
https://github.com/gdonald/reversi-py
ai machine-learning othello reversi
Last synced: 15 days ago
JSON representation
Reversi with reinforcement learning
- Host: GitHub
- URL: https://github.com/gdonald/reversi-py
- Owner: gdonald
- License: mit
- Created: 2025-09-06T20:09:58.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-09-23T15:51:40.000Z (10 months ago)
- Last Synced: 2025-10-07T13:53:00.128Z (10 months ago)
- Topics: ai, machine-learning, othello, reversi
- Language: Python
- Homepage:
- Size: 597 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# reversi-py
Experiments with Reversi in a deep learning environment.
## Install
```
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
Python 3.10+ recommended. Default device is CPU; override with `--device mps|cuda|auto` if you want GPU. MPS is often slower than CPU for this setup.
## Scripts
- Play the classic console game (no RL): `python reversi.py`
- Play the console game vs the SB3 agent: `python reversi.py --sb3-model checkpoints/sb3/best_model.zip` (add `--device cpu|mps|cuda` to force)
- Train agent (defaults below): `python train_sb3.py`
- Evaluate a checkpoint vs bots (get winrate): `python eval_agent_sb3.py --model checkpoints/sb3/best_model.zip --games 50 --opponents heuristic random`
- Head-to-head in a simple prompt (coords like `d3`/`pass`): `python play_vs_agent.py --model checkpoints/sb3/best_model.zip`
- Plot eval winrate history from training logs (uses `logs/sb3/eval_history.csv`): `python plot_eval.py --logdir logs/sb3`
- Baseline bots vs each other: `python eval_bots.py --games 50`
## Training defaults (tweak in `train_sb3.py`)
```
total_timesteps = 10_000_000
n_envs = 32
bot_mix_prob = 0.8 # fraction of episodes vs heuristic opponent
eval_freq = 40_000 # timesteps between evals
eval_games = 50
learning_rate = 3e-4
ent_coef = 0.03
clip_range = 0.3
n_steps = 2048
batch_size = 2048
gamma = 0.99
gae_lambda = 0.95
n_epochs = 4
target_kl = 0.02
checkpoints = checkpoints/sb3
logdir = logs/sb3
```
Overrides (examples):
- Auto-resume is on by default (uses `best_model.zip` > `final_model.zip` > newest step checkpoint)
- Continue from a specific checkpoint: `--load-model checkpoints/sb3/best_model.zip`
- Change sparring mix: `--bot-mix-prob 0.5`
- Force single-process vec env (if SubprocVecEnv blocked): `--no-subproc`
- Adjust device: `--device mps|cuda|cpu|auto` (default is `cpu`)
Outputs:
- Checkpoints saved every `--save-freq` to `checkpoints/sb3/` (`best_model.zip` from eval; `final_model.zip` at end).
- Logs and `eval_history.csv` in `logs/sb3/`
- Winrate tracking: use `python eval_agent_sb3.py ...` for a one-off, or run `python plot_eval.py --logdir logs/sb3` to graph the eval callback history.
- TensorBoard: `tensorboard --logdir logs/sb3` (opens http://localhost:6006) to see PPO losses/entropy/etc.
## Preview
