{"id":28262958,"url":"https://github.com/ronenh24/farkle_simulation","last_synced_at":"2025-06-16T14:31:19.086Z","repository":{"id":213229153,"uuid":"733381276","full_name":"ronenh24/farkle_simulation","owner":"ronenh24","description":"Simulating two-player Farkle with different strategies.","archived":false,"fork":false,"pushed_at":"2025-04-17T16:34:48.000Z","size":16001,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-20T07:13:20.076Z","etag":null,"topics":["deep-q-learning","farkle","game","simulation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ronenh24.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-19T07:47:05.000Z","updated_at":"2025-04-17T16:34:51.000Z","dependencies_parsed_at":"2025-04-16T21:22:43.478Z","dependency_job_id":"a86ed852-a43b-4304-bb0b-0749687e47c9","html_url":"https://github.com/ronenh24/farkle_simulation","commit_stats":null,"previous_names":["ronenh24/farkle_simulation"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ronenh24/farkle_simulation","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ronenh24%2Ffarkle_simulation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ronenh24%2Ffarkle_simulation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ronenh24%2Ffarkle_simulation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ronenh24%2Ffarkle_simulation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ronenh24","download_url":"https://codeload.github.com/ronenh24/farkle_simulation/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ronenh24%2Ffarkle_simulation/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260175737,"owners_count":22970039,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-q-learning","farkle","game","simulation"],"created_at":"2025-05-20T07:13:19.585Z","updated_at":"2025-06-16T14:31:19.060Z","avatar_url":"https://github.com/ronenh24.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Farkle Simulation\n\n## Author\nRonen Huang\n\n## Time Frame\nNovember 2021 to December 2021 (modified for Python usage), January 2025 to Present (add deep Q-Learning).  \n\n## Install Farkle Simulation\nThe Farkle Simulation can be downloaded by pip.\n```commandline\npip install farkle-simulation\n```\n\n## Farkle\nThe rules of Farkle can be seen in the *In a Nutshell* section of [https://farkle.games/official-rules/](https://farkle.games/official-rules/). In this case, there are two players with **Player 1** as the first player and **Player 2** as the second player.\n\nThe scoring system can be seen in the table below\n\u003ctable style=\"margin-left: auto; margin-right: auto;\"\u003e\n  \u003ctr\u003e\u003cth\u003eDice to Keep\u003c/th\u003e \u003cth\u003eScore\u003c/th\u003e\n  \u003ctr\u003e\u003ctd\u003eThree 1s or Straight\u003c/td\u003e \u003ctd\u003e1000\u003c/td\u003e\n  \u003ctr\u003e\u003ctd\u003eThree Different Pairs\u003c/td\u003e \u003ctd\u003e750\u003c/td\u003e\n  \u003ctr\u003e\u003ctd\u003eThree 6s\u003c/td\u003e \u003ctd\u003e600\u003c/td\u003e\n  \u003ctr\u003e\u003ctd\u003eThree 5s\u003c/td\u003e \u003ctd\u003e500\u003c/td\u003e\n  \u003ctr\u003e\u003ctd\u003eThree 4s\u003c/td\u003e \u003ctd\u003e400\u003c/td\u003e\n  \u003ctr\u003e\u003ctd\u003eThree 3s\u003c/td\u003e \u003ctd\u003e300\u003c/td\u003e\n  \u003ctr\u003e\u003ctd\u003eThree 2s\u003c/td\u003e \u003ctd\u003e200\u003c/td\u003e\n  \u003ctr\u003e\u003ctd\u003eOne 1\u003c/td\u003e \u003ctd\u003e100\u003c/td\u003e\n  \u003ctr\u003e\u003ctd\u003eOne 5\u003c/td\u003e \u003ctd\u003e50\u003c/td\u003e\n\u003c/table\u003e\n\nA player begins their turn with all six dice.\n- If there are scoring combinations, a player can choose any and\n  - Either keep rolling or stop\n  - When number of dice is 0, it resets to 6\n- Otherwise, a player has **\"farkled\"** and the turn score is **0**\n\nOnce a player reaches 10,000 points, they have won the game.\n\n## Strategies\nThe implementation of the strategies is in [`strategy.py`](src/farkle_simulation/components/strategy.py).\n\nThe strategies that Player 1 and Player 2 can use are naive strategy, simple RL strategy, and custom strategy (manual). The `turn` function goes through one turn.\n```python\nfrom farkle_simulation.components.strategy import\\\n  NaiveStrategy, SimpleRLStrategy, CustomStrategy\n\nnaive_strategy = NaiveStrategy()\nsimple_rl_strategy = SimpleRLStrategy()\ncustom_strategy = CustomStrategy()\n\nnaive_turn_score = naive_strategy.turn(current_score, advantage)\nsimple_rl_turn_score = simple_rl_strategy.turn(current_score, advantage)\ncustom_turn_score = custom_strategy.turn(current_score, advantage)\n```\n\n### Custom Strategy\nThe player chooses the action each roll via typing 1 to 7 on the command line.\n\n### Naive Strategy\nFor each roll\n- Choose action that maximizes roll score\n- Stop if number of dice is less than or equal to 2 and disadvantage of    less than 1,000.\n\n### Simple Reinforcment Learning Strategy\nFor each roll\n- Choose action that maximizes reward based on\n  - Number of Rolls\n    ```python\n    # The gamma is reward factor decrease by roll.\n    actual_reward = gamma ** np.log2(num_rolls + 1) * roll_score\n    ```\n  - Distance to 10,000 and Current Score\n    ```python\n    # Increases as close to 10,000.\n    distance_factor = distance_scale ** (current_score / 10000)\n    ```\n  - Advantage\n    ```python\n    # Increase as more behind.\n    advantage_factor = 1\n    if advantage \u003c 0:\n        advantage_factor *= max(\n          np.emath.logn(advantage_scale, -advantage), 1\n        )\n    ```\nThe architecture of the deep Q-learning networks takes 11 dimensional input state\n- Distance\n- Advantage\n- Current Turn Score\n- Roll Maxes\n\nand predicts reward for each of 7 possible actions\n- Keep 1 to 6 dice\n- Stop\n\nThis can be seen in [`architecture.py`](src/farkle_simulation/components/architecture.py).\n\nThe training process can be seen at [`simple_farkle_rl.py`](src/farkle_simulation/components/simple_farkle_rl.py). The best model state dictionary is saved as [simple_action_reward_state_dict.pt](src/farkle_simulation/components/simple_action_reward_state_dict.pt).\n```python\nfrom farkle_simulation.components.simple_farkle_rl import train\n\ntrain()\n```\n\nThe plot of turn score by current score can be seen in **[training_simple_rl.jpg](src/plots/training_simple_rl.jpg)** and the table in **[training_simple_rl.csv](src/tables/training_simple_rl.csv)**.\n![Training Simple RL](src/plots/training_simple_rl.jpg)\n\n## Simulation\nThe implementation of the simulation is in [`simulation.py`](src/farkle_simulation/components/simulation.py).  \n\nThe `convergence_plot` function plots the expected probability Player 1 wins using some strategy with Player 2 using some other strategy.\n```python\nfrom farkle_simulation.components.simulation import convergence_plot\n\nconvergence_plot(naive_strategy, simple_rl_strategy)\nconvergence_plot(simple_rl_strategy, naive_strategy)\n```\n\nThe convergence plots can be seen at **[convergence_naive_simple_rl.jpg](src/plots/convergence_naive_simple_rl.jpg)** ![Convergence Plot Naive vs Simple RL](src/plots/convergence_naive_simple_rl.jpg) and **[convergence_simple_rl_naive.jpg](src/plots/convergence_simple_rl_naive.jpg)** ![Convergence Plot Simple RL vs Naive](src/plots/convergence_simple_rl_naive.jpg)\n\nThe `histogram` function plots the average probability Player 1 wins using some strategy with Player 2 using some other strategy.\n```python\nfrom farkle_simulation.components.simulation import histogram\n\nhistogram(naive_strategy, simple_rl_strategy)\nhistogram(simple_rl_strategy, naive_strategy)\n```\n\nThe histograms can be seen at **[histogram_naive_simple_rl.jpg](src/plots/histogram_naive_simple_rl.jpg)** ![Histogram Naive vs Simple RL](src/plots/histogram_naive_simple_rl.jpg) and **[histogram_simple_rl_naive.jpg](src/plots/histogram_simple_rl_naive.jpg)** ![Histogram Simple RL vs Naive](src/plots/histogram_simple_rl_naive.jpg)\n\n## Pipeline\nTo train the simple RL agent and compare the strategies through convergence plot and histogram.\n```python\nfrom farkle_simulation.pipeline import train_simulate\n\ntrain_simulate()\n```\n\nTo play a game against the simple RL agent.\n```python\nfrom farkle_simulation.pipeline import play_game\n\nplay_game()\n```\nInput 1 to play as first player and 2 to play as second player.\n```console\nPlayer 1 (1) or Player (2)? 1\n```\nInput action to chose for dice combination.\n```console\nDice Combination - (1, 1, 1, 1, 1, 4)\nLegal Moves - (1) Keep 1 - max 100 remaining dice 5, (2) Keep 2 - max 200 remaining dice 4, (3) Keep 3 - max 1000 remaining dice 3, (4) Keep 4 - max 1100 remaining dice 2, (5) Keep 5 - max 1200 remaining dice 1, (7) No Roll - max 1200 remaining dice 0\nAction - 7\n```\nThe output of action is.\n```console\nCurrent Turn Score - 1200\nCurrent Score - 1200\nTurn Score - 1200\nPlayer 1 Score - 1200 Player 2 Score - 0\n```\nThe output of a \"farkle\" is.\n```console\nDice Combination - (2, 6)\nFarkled\nTurn Score - 0\n```\n\n## References\nFarkle Official Rules. (2025). Retrieved from https://farkle.games/official-rules/\n\nLe Cam, L. (1986). The Central Limit Theorem Around 1935. *Statistical Science*, *1*(1), 78–91. Retrieved from http://www.jstor.org/stable/2245503\n\nMnih, V., Kavukcuoglu, K., Silver, D. *et al.* (2015). Human-level control through deep reinforcement learning. *Nature* **518**, 529–533. Retrieved from https://doi.org/10.1038/nature14236\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fronenh24%2Ffarkle_simulation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fronenh24%2Ffarkle_simulation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fronenh24%2Ffarkle_simulation/lists"}