{"id":29547012,"url":"https://github.com/valyriantech/valyriangamesleaderboard","last_synced_at":"2026-02-08T12:32:34.150Z","repository":{"id":304489875,"uuid":"1018891078","full_name":"ValyrianTech/ValyrianGamesLeaderboard","owner":"ValyrianTech","description":"Leaderboard for the Valyrian LLM Games","archived":false,"fork":false,"pushed_at":"2025-07-13T11:42:14.000Z","size":94,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-13T13:21:52.460Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ValyrianTech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-13T09:24:15.000Z","updated_at":"2025-07-13T11:42:00.000Z","dependencies_parsed_at":"2025-07-13T13:21:54.539Z","dependency_job_id":"61f2284c-975c-485b-8449-de9375d70896","html_url":"https://github.com/ValyrianTech/ValyrianGamesLeaderboard","commit_stats":null,"previous_names":["valyriantech/valyriangamesleaderboard"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/ValyrianTech/ValyrianGamesLeaderboard","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ValyrianTech%2FValyrianGamesLeaderboard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ValyrianTech%2FValyrianGamesLeaderboard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ValyrianTech%2FValyrianGamesLeaderboard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ValyrianTech%2FValyrianGamesLeaderboard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ValyrianTech","download_url":"https://codeload.github.com/ValyrianTech/ValyrianGamesLeaderboard/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ValyrianTech%2FValyrianGamesLeaderboard/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265649110,"owners_count":23804821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-17T19:08:55.215Z","updated_at":"2026-02-08T12:32:34.144Z","avatar_url":"https://github.com/ValyrianTech.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ValyrianGamesLeaderboard\nLeaderboard for the Valyrian LLM Games\n\n🌐 **[View Live Leaderboard](https://valyriantech.github.io/ValyrianGamesLeaderboard/)** \n\n## Overview\n\nThe Valyrian Games Leaderboard is a web-based system for displaying the results and rankings of LLM competitions. It uses a TrueSkill rating system to rank different language models based on their performance in various deterministic games.\n\nThis project focuses on the front-end display of results, with the actual game execution handled by a separate system. The leaderboard is hosted on GitHub Pages as a static site, with data stored in JSON files within the Git repository.\n\n## Features\n\n- **Interactive Leaderboard**: Display LLM rankings with TrueSkill ratings\n- **Game History**: Browse all past games with filtering options\n- **Detailed Game Information**: View detailed results for each game\n- **Data Visualization**: Charts showing model performance and statistics\n- **Static Site Generation**: Python-based static site for GitHub Pages hosting\n- **Git-Based Data Storage**: Game results stored as JSON files in the repository\n\n## Technical Stack\n\n- **Python 3.12** for data processing and static site generation\n- **Flask** for development server\n- **Frozen-Flask** for converting Flask app to static files\n- **Jinja2** templates for HTML generation\n- **Bootstrap** for styling (via CDN)\n- **Chart.js** for data visualizations\n- **TrueSkill** for rating calculations\n- **GitHub Actions** for CI/CD pipeline\n\n## Setup Instructions\n\n### Prerequisites\n\n- Python 3.10+\n- Git\n\n### Installation\n\n1. Clone the repository:\n   ```\n   git clone https://github.com/ValyrianTech/ValyrianGamesLeaderboard.git\n   cd ValyrianGamesLeaderboard\n   ```\n\n2. Create a virtual environment:\n   ```\n   python -m venv venv\n   source venv/bin/activate  # On Windows: venv\\Scripts\\activate\n   ```\n\n3. Install dependencies:\n   ```\n   pip install -r requirements.txt\n   ```\n\n### Development\n\nTo run the development server:\n\n```\npython run.py\n```\n\nThe site will be available at http://localhost:5000\n\n## Usage\n\n### Adding Game Results\n\nTo add a new game result:\n\n1. Create a JSON file with the game data (see format below)\n2. Run the update script:\n   ```\n   python scripts/update_leaderboard.py path/to/game_result.json --commit\n   ```\n\nThis will:\n- Add the game result to the data/games directory\n- Update the leaderboard.json file with new ratings\n- Commit the changes to the Git repository\n\n### Game Result Format\n\n```json\n{\n  \"game_id\": \"unique-game-id\",\n  \"date\": \"2025-07-13T10:30:00Z\",\n  \"game_type\": \"CodeGolf\",\n  \"participants\": [\"TestModel-Alpha\", \"TestModel-Beta\", \"TestModel-Gamma\"],\n  \"ranks\": [0, 1, 2],\n  \"scores\": [10, 8, 5],\n  \"description\": \"A code golf challenge to implement quicksort in the fewest characters.\"\n}\n```\n\n### Generating the Static Site\n\nTo generate the static site for GitHub Pages:\n\n```\npython scripts/freeze.py\n```\n\nThe static site will be generated in the `build` directory.\n\n## Day-to-Day Workflow\n\n### 1. Local Development\n\nTo work on the project locally:\n\n```bash\n# Start the development server\n./run_dev.sh\n```\n\nThis script:\n- Activates the Python virtual environment\n- Installs dependencies if needed\n- Runs the Flask development server at http://localhost:5000\n\nYou can then view and test the site in your browser while making changes to the code.\n\n### 2. Adding New Game Results\n\nWhen new LLM competitions are completed:\n\n1. Create a JSON file with the game results in the format shown in `data/sample_new_game.json`\n2. Run the update script:\n\n```bash\npython scripts/update_leaderboard.py --game-file path/to/new_game.json --commit\n```\n\nThis script:\n- Processes the new game results\n- Updates the leaderboard ratings using TrueSkill\n- Saves the game JSON to the data/games directory\n- Updates the leaderboard.json file\n- Optionally commits changes to Git (with the `--commit` flag)\n\n### 3. Generating the Static Site\n\nTo build the static site for deployment:\n\n```bash\npython scripts/freeze.py\n```\n\nThis generates all static files in the `build` directory, ready for hosting.\n\n### 4. Deployment\n\nThe site is automatically deployed to GitHub Pages when changes are pushed to the main branch, thanks to the GitHub Actions workflow in `.github/workflows/deploy.yml`.\n\nManual deployment steps:\n1. Commit your changes: `git add . \u0026\u0026 git commit -m \"Your message\"`\n2. Push to GitHub: `git push origin main`\n3. The GitHub Actions workflow will build and deploy the site\n\n### 5. Common Tasks\n\n#### Adding a New Game Result\n\n1. Create a JSON file with the game result data (see format below)\n2. Run the update script: `python scripts/update_leaderboard.py --game-file path/to/game_result.json`\n3. The script will:\n   - Validate the game data\n   - Save the game result to the `data/games` directory\n   - Automatically update the leaderboard with new ratings\n   - Add any new models that haven't been seen before\n4. Optionally commit the changes: `python scripts/update_leaderboard.py --game-file path/to/game_result.json --commit`\n\n#### Recalculating the Leaderboard\n\nIf game files have been deleted, modified, or you need to rebuild the leaderboard from scratch:\n\n1. Run the update script with the recalculate flag:\n```bash\npython scripts/update_leaderboard.py --recalculate\n```\n\n2. This will:\n   - Delete the existing leaderboard.json file\n   - Load all game files from the data/games directory\n   - Process each game in chronological order\n   - Rebuild the leaderboard from scratch\n   - Skip any invalid game files\n\n3. Optionally commit the changes:\n```bash\npython scripts/update_leaderboard.py --recalculate --commit\n```\n\n#### Adding a New Model\n\nNew models are automatically added to the leaderboard when they first appear in a game result. You don't need to manually edit the leaderboard.json file.\n\nWhen a new model appears in a game result:\n1. It's automatically added to the leaderboard with default TrueSkill ratings\n2. Its ratings are updated based on its performance in the game\n3. The leaderboard is sorted by the conservative rating\n\nIf you need to add metadata for a model (like a description or link), you can edit the model entry in `data/leaderboard.json` after it has been automatically added.\n\n#### Generating the Static Site\n\nTo generate the static site for GitHub Pages:\n\n```\npython scripts/freeze.py\n```\n\nThe static site will be generated in the `build` directory.\n\n## Project Structure\n\n```\nvalyrian-games-leaderboard/\n├── app/                      # Flask application\n│   ├── __init__.py\n│   ├── routes.py\n│   ├── models.py\n│   ├── utils/\n│   ├── templates/\n│   └── static/\n├── data/                     # Data storage\n│   ├── leaderboard.json\n│   └── games/\n├── scripts/                  # Utility scripts\n│   ├── update_leaderboard.py\n│   └── freeze.py\n├── .github/workflows/        # GitHub Actions workflows\n│   └── deploy.yml\n├── venv/                     # Python virtual environment\n├── requirements.txt          # Python dependencies\n├── config.py                 # Configuration settings\n├── run.py                    # Development server entry point\n└── README.md                 # This file\n```\n\n## Deployment\n\nThe site is automatically deployed to GitHub Pages when changes are pushed to the `main` branch that affect the data or application code. The GitHub Actions workflow:\n\n1. Sets up a Python environment\n2. Installs dependencies\n3. Generates the static site using Frozen-Flask\n4. Deploys the static files to the `gh-pages` branch\n\n## License\n\nThis project is licensed under the MIT License - see the LICENSE file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalyriantech%2Fvalyriangamesleaderboard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvalyriantech%2Fvalyriangamesleaderboard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalyriantech%2Fvalyriangamesleaderboard/lists"}