{"id":20056065,"url":"https://github.com/zackakil/deep-tic-tac-toe","last_synced_at":"2025-04-13T05:13:31.837Z","repository":{"id":165194945,"uuid":"237809673","full_name":"ZackAkil/deep-tic-tac-toe","owner":"ZackAkil","description":"Used deep reinforcement learning to train a deep neural network to play tic-tac-toe and deployed using tensorflow.js.","archived":false,"fork":false,"pushed_at":"2024-10-30T10:39:16.000Z","size":898,"stargazers_count":60,"open_issues_count":2,"forks_count":15,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-13T05:13:20.655Z","etag":null,"topics":["convolutional-neural-networks","keras","machine-learning","neural-network","reinforcement-learning","tensorflow-js"],"latest_commit_sha":null,"homepage":"https://zackakil.github.io/deep-tic-tac-toe/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ZackAkil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-02T17:39:20.000Z","updated_at":"2025-04-03T12:23:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"5e85dab6-aa5d-4a16-894e-e407a13a533f","html_url":"https://github.com/ZackAkil/deep-tic-tac-toe","commit_stats":{"total_commits":34,"total_committers":1,"mean_commits":34.0,"dds":0.0,"last_synced_commit":"bfa7a47e24e2bebd2621cfbe72d93f3b75e58068"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZackAkil%2Fdeep-tic-tac-toe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZackAkil%2Fdeep-tic-tac-toe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZackAkil%2Fdeep-tic-tac-toe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ZackAkil%2Fdeep-tic-tac-toe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ZackAkil","download_url":"https://codeload.github.com/ZackAkil/deep-tic-tac-toe/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248665746,"owners_count":21142123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-neural-networks","keras","machine-learning","neural-network","reinforcement-learning","tensorflow-js"],"created_at":"2024-11-13T12:51:20.760Z","updated_at":"2025-04-13T05:13:31.814Z","avatar_url":"https://github.com/ZackAkil.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deep Tic-Tac-Toe [Play game](https://zackakil.github.io/deep-tic-tac-toe)\n\nThis project uses deep reinforcement learning to train a neural network to play Tic-Tac-Toe. The trained model is deployed in a web browser using TensorFlow.js.\n\n\n![screenshot](screen_shot.png)\n\n## How it Works\n\nThe project consists of two main components:\n\n1. **Model Training (Python):**  A Jupyter Notebook (`deep_learning_tic_tac-toe_model_training.ipynb` and `[player_goes_first]_deep_learning_tic_tac_toe_model_training.ipynb`) handles training the neural network.  It uses a convolutional neural network (CNN) built with Keras. The training process involves:\n\n    * **Game Environment:** A custom `XandOs` class simulates the Tic-Tac-Toe environment, allowing the agent to interact with it.\n    * **Reinforcement Learning:** The agent learns through experience by playing against a random agent.  Rewards are assigned for wins, losses, ties, and invalid moves.\n    * **Experience Replay:**  Game states, actions, and rewards are stored in a memory buffer (`memory`). The agent learns from a batch of randomly sampled experiences from this buffer, improving stability and convergence.\n    * **CNN Architecture:** The CNN takes the current game board (represented as a 3x3x2 tensor, where the two channels indicate player 1 and player 2's marks) as input and outputs a probability distribution over the 9 possible moves.\n    * **Training Loop:** The agent repeatedly plays games, stores experiences in memory, and updates the CNN's weights based on the rewards received.\n   \n\n2. **Web Deployment (TensorFlow.js):**  The trained model is converted to a TensorFlow.js Layers format and loaded in a web browser using `index.html`.  The webpage provides a user interface to play against the AI.  The `predict` function takes the current game grid as input and uses the loaded model to select the AI's next move.  A small delay is added before the AI's move to simulate \"thinking\" time.\n\n\n## Dependencies\n\n* **Python:**  NumPy, Matplotlib, Keras, TensorFlow (or TensorFlow 1.x in Colab)\n* **Web:**  Vue.js, TensorFlow.js\n\n\n##  Key Files\n\n* **`deep_learning_tic_tac_toe_model_training.ipynb`:** Jupyter Notebook for training the AI model.  \n* **`[player_goes_first]_deep_learning_tic_tac_toe_model_training.ipynb`:** Jupyter Notebook for training the AI model where the player goes first\n* **`index.html`:**  HTML file for the web-based game.\n* **`model/model.json`:** TensorFlow.js Layers model file.\n* **`python model weights/winer_weights.keras`:** Keras model weights (for the version of the model that has been trained where the agent goes second)\n\n## Potential Improvements\n\n* **Training against a stronger opponent:** The current random agent is a relatively weak opponent. Training against a minimax algorithm or another deep learning agent could potentially lead to a stronger AI.\n* **Exploring different network architectures:**  Experimenting with different CNN architectures or other types of neural networks (e.g., recurrent neural networks) might improve performance.\n* **Hyperparameter tuning:** Fine-tuning the hyperparameters (e.g., learning rate, batch size, decay rate) used during training could lead to better results.\n* **Adding difficulty levels:** Implement different difficulty levels by adjusting the epsilon-greedy exploration strategy or by using different trained models.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzackakil%2Fdeep-tic-tac-toe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzackakil%2Fdeep-tic-tac-toe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzackakil%2Fdeep-tic-tac-toe/lists"}