{"id":49233587,"url":"https://github.com/kuds/reinforce-tactics","last_synced_at":"2026-04-24T13:31:33.894Z","repository":{"id":320631103,"uuid":"1082834937","full_name":"kuds/reinforce-tactics","owner":"kuds","description":"Turned based strategy game with the goal of developing reinforcement learning algorithms","archived":false,"fork":false,"pushed_at":"2026-04-16T18:27:15.000Z","size":5241,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-16T20:26:23.938Z","etag":null,"topics":["gymnasium","heirarchical-reinforcement-learning","pygame","reinforcement-learning","turn-based-strategy"],"latest_commit_sha":null,"homepage":"https://reinforcetactics.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kuds.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"docs/ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-24T21:02:24.000Z","updated_at":"2026-04-16T18:27:22.000Z","dependencies_parsed_at":"2026-01-09T06:08:32.111Z","dependency_job_id":null,"html_url":"https://github.com/kuds/reinforce-tactics","commit_stats":null,"previous_names":["kuds/reinforce-tactics"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/kuds/reinforce-tactics","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuds%2Freinforce-tactics","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuds%2Freinforce-tactics/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuds%2Freinforce-tactics/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuds%2Freinforce-tactics/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kuds","download_url":"https://codeload.github.com/kuds/reinforce-tactics/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kuds%2Freinforce-tactics/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32225732,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T13:21:15.438Z","status":"ssl_error","status_checked_at":"2026-04-24T13:21:15.005Z","response_time":64,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gymnasium","heirarchical-reinforcement-learning","pygame","reinforcement-learning","turn-based-strategy"],"created_at":"2026-04-24T13:31:33.096Z","updated_at":"2026-04-24T13:31:33.877Z","avatar_url":"https://github.com/kuds.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Reinforce Tactics\n\n[![GitHub Stars](https://img.shields.io/github/stars/kuds/reinforce-tactics)](https://github.com/kuds/reinforce-tactics/stargazers)\n[![GitHub License](https://img.shields.io/github/license/kuds/reinforce-tactics)](https://github.com/kuds/reinforce-tactics/blob/main/LICENSE)\n[![Tests](https://img.shields.io/github/actions/workflow/status/kuds/reinforce-tactics/python-package.yml?label=tests)](https://github.com/kuds/reinforce-tactics/actions/workflows/python-package.yml)\n[![Lint](https://img.shields.io/github/actions/workflow/status/kuds/reinforce-tactics/lint.yml?label=lint)](https://github.com/kuds/reinforce-tactics/actions/workflows/lint.yml)\n[![Docs](https://img.shields.io/github/actions/workflow/status/kuds/reinforce-tactics/deploy-docusaurus.yml?label=docs)](https://github.com/kuds/reinforce-tactics/actions/workflows/deploy-docusaurus.yml)\n[![Documentation](https://img.shields.io/badge/docs-reinforcetactics.com-blue)](https://reinforcetactics.com)\n\n![](images/rt_demo.gif)\n\u003c!-- ![](images/reinforce_tactics_logo.svg) --\u003e\n\nA turn-based strategy game built with Pygame and Gymnasium for reinforcement learning research. Train RL agents, play against AI opponents (rule-based or LLM-powered), and experiment with tactical decision-making.\n\n\u003e **Requires Python 3.10+**\n\n## Features\n\n- **Tactical Gameplay**: 8 unit types (Warrior, Mage, Cleric, Archer, Knight, Rogue, Sorcerer, Barbarian) with unique abilities\n- **Gymnasium Integration**: Standard RL environment with observation/action spaces and reward shaping\n- **Multiple AI Opponents**: Rule-based bots (Easy/Medium/Hard) and LLM bots (GPT, Claude, Gemini)\n- **Training Algorithms**: PPO/A2C/DQN via Stable-Baselines3, AlphaZero with MCTS, and Feudal RL (hierarchical manager-worker)\n- **Action Masking**: MaskablePPO and legal-action masking across all bot types\n- **Self-Play**: Train agents against copies of themselves with safe weight swapping\n- **Tournament System**: Round-robin tournaments with ELO ratings, Docker support, and result tracking\n- **Fog of War**: Line-of-sight visibility with terrain bonuses\n- **Map Editor**: In-game editor for creating and modifying maps\n- **Multi-Player Modes**: 1v1, 1v1v1 (free-for-all), and 2v2 (team) maps\n- **Replay System**: Record games, replay them, and export to video\n- **Sprite Animations**: Per-team palette swapping and movement path transitions\n- **Save/Load**: Persist and resume in-progress games\n- **Multi-Language**: English, Korean, Spanish, French, Chinese\n\n\u003c!-- ![](images/rt_demo.gif) --\u003e\n\n## Installation\n\n```bash\n# Clone the repository\ngit clone https://github.com/kuds/reinforce-tactics.git\ncd reinforce-tactics\n\n# Install base package (RL training, headless mode)\npip install -e .\n\n# Install with GUI support\npip install -e \".[gui]\"\n\n# Install with LLM bot support\npip install -e \".[llm]\"\n\n# Install everything (GUI + LLM + dev tools)\npip install -e \".[all]\"\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eWhat each extra includes\u003c/summary\u003e\n\n| Extra | Packages |\n|-------|----------|\n| *(base)* | gymnasium, pettingzoo, stable-baselines3, sb3-contrib, numpy, torch, tensorboard, pandas |\n| `[gui]` | pygame-ce, opencv-python, matplotlib, Pillow |\n| `[llm]` | openai, anthropic, google-genai |\n| `[dev]` | pytest, pytest-cov, pre-commit, ruff, mypy |\n| `[all]` | All of the above |\n\n\u003c/details\u003e\n\n## Quick Start\n\n### Play the Game\n\n```bash\npython main.py\n```\n\n### Train an RL Agent\n\n```bash\n# Train with PPO against bot\npython main.py --mode train --algorithm ppo --timesteps 1000000 --opponent bot\n\n# Train with self-play\npython main.py --mode train --algorithm ppo --timesteps 1000000 --opponent self\n\n# Train with reward shaping\npython main.py --mode train --algorithm ppo --timesteps 500000 \\\n    --reward-income 0.1 --reward-units 0.05 --reward-structures 0.1\n\n# Evaluate trained model\npython main.py --mode evaluate --model models/ppo_model.zip --episodes 10\n\n# View training stats\npython main.py --mode stats\n```\n\n### Advanced Training\n\n```bash\n# AlphaZero with MCTS\npython train/train_alphazero.py\n\n# Feudal RL (hierarchical manager-worker)\npython train/train_feudal_rl.py\n\n# Self-play training\npython train/train_self_play.py\n```\n\n### Use as Gymnasium Environment\n\n```python\nfrom reinforcetactics.rl.gym_env import StrategyGameEnv\n\nenv = StrategyGameEnv(\n    map_file='maps/1v1/beginner.csv',\n    opponent='bot',\n    render_mode=None  # None for headless, 'human' for GUI\n)\n\nobs, info = env.reset()\naction = env.action_space.sample()\nobs, reward, terminated, truncated, info = env.step(action)\n```\n\n### Play Against LLM Bots\n\n```python\nfrom reinforcetactics.core.game_state import GameState\nfrom reinforcetactics.game.llm_bot import OpenAIBot, ClaudeBot, GeminiBot\nfrom reinforcetactics.utils.file_io import FileIO\n\nmap_data = FileIO.load_map('maps/1v1/test_map.csv')\ngame = GameState(map_data, num_players=2)\n\n# Requires API key in environment (OPENAI_API_KEY, ANTHROPIC_API_KEY, or GOOGLE_API_KEY)\nbot = ClaudeBot(game, player=2, model='claude-sonnet-4-5-20250929')\n```\n\nSee the `examples/` directory for more, including an action-masking training demo.\n\n## Game Rules\n\n| Unit | Cost | Move | HP | Special |\n|------|------|------|-----|---------|\n| Warrior | 200 | 3 | 15 | High HP melee |\n| Mage | 300 | 2 | 10 | Ranged 1-2, paralyze (3 turns, 2-turn cooldown) |\n| Cleric | 200 | 2 | 8 | Heal/cure allies (range 1-2) |\n| Archer | 250 | 3 | 15 | Ranged 2-3 tiles (+1 on mountains) |\n| Knight | 350 | 4 | 18 | Charge (+50% dmg if moved 3+ tiles) |\n| Rogue | 350 | 4 | 12 | Flank (+50% dmg), Evade (15% dodge, 30% in forest) |\n| Sorcerer | 400 | 2 | 10 | Haste, Attack/Defence Buff (+35%) |\n| Barbarian | 400 | 5 | 20 | Fast, high-damage melee |\n\n**Win Conditions**: Capture enemy HQ or eliminate all enemy units\n\n**Economy**: Starting gold $250. Income from structures each turn (HQ: $150, Building: $100, Tower: $50)\n\n**Terrain**: Grass, forest (stealth bonus), mountains (vision/range bonus), roads (fast movement), water/ocean (impassable)\n\n**Fog of War**: Enemy HQ is always visible; buildings and towers are hidden until scouted\n\n## Project Structure\n\n```\nreinforce-tactics/\n├── main.py                    # CLI entry point (train/evaluate/play/stats)\n├── pyproject.toml             # Package config and dependencies\n├── reinforcetactics/          # Main package\n│   ├── core/                  # Game state, units, grid, visibility\n│   ├── game/                  # Mechanics, bots (rule-based, LLM, model, AlphaZero)\n│   ├── rl/                    # Gymnasium env, AlphaZero, Feudal RL, MCTS, self-play\n│   ├── tournament/            # Tournament runner, ELO ratings, scheduling\n│   ├── ui/                    # Pygame renderer, menus, map editor, sprites\n│   └── utils/                 # File I/O, replay, settings, language, fonts\n├── cli/                       # CLI command implementations\n├── game/                      # Game loop, input handler, action executor\n├── maps/                      # CSV map files (1v1, 1v1v1, 2v2)\n├── train/                     # Training scripts (self-play, AlphaZero, Feudal RL)\n├── eval/                      # Evaluation scripts\n├── tests/                     # Test suite\n├── examples/                  # Example scripts and demos\n├── notebooks/                 # Jupyter notebooks (PPO training, tournaments)\n├── docker/                    # Docker configs for tournaments\n├── assets/                    # Sprite sheets\n├── docs-site/                 # Docusaurus documentation site\n└── benchmarks/                # Performance benchmarks\n```\n\n## Testing\n\n```bash\n# Run all tests\npytest tests/\n\n# Run a specific test file\npytest tests/test_mechanics.py -v\n```\n\n## Docker (Tournaments)\n\nRun bot tournaments in a containerized environment:\n\n```bash\ncd docker/tournament\ndocker-compose up --build\n```\n\nConfigure bots, maps, and settings in `docker/tournament/config.json`. See `docker/tournament/README.md` for details.\n\n## Documentation\n\nDocs are split by audience:\n\n- **Users** — [reinforcetactics.com](https://reinforcetactics.com) (sourced from\n  [`docs-site/`](docs-site/)): game rules, RL environment API, LLM bot\n  configuration, tournament results, map creation guide.\n- **Contributors** — [`docs/`](docs/) (see [`docs/README.md`](docs/README.md) for\n  an index): roadmap, internal code reviews, and developer-facing guides.\n\n## Contributing\n\nContributions welcome! Install dev dependencies and set up pre-commit hooks:\n\n```bash\npip install -e \".[dev]\"\npre-commit install\n```\n\nSee the [documentation](https://reinforcetactics.com) for development guidelines.\n\n## License\n\nApache License 2.0\n\n## Citation\n\n```bibtex\n@software{reinforce_tactics,\n  author = {Michael Kudlaty},\n  title = {Reinforce Tactics: A Turn-Based Strategy Game for Reinforcement Learning},\n  year = {2025},\n  url = {https://github.com/kuds/reinforce-tactics}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuds%2Freinforce-tactics","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkuds%2Freinforce-tactics","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkuds%2Freinforce-tactics/lists"}