{"id":13538886,"url":"https://github.com/tensorflow/minigo","last_synced_at":"2025-09-30T08:31:46.152Z","repository":{"id":37458188,"uuid":"119455332","full_name":"tensorflow/minigo","owner":"tensorflow","description":"An open-source implementation of the AlphaGoZero algorithm","archived":true,"fork":false,"pushed_at":"2021-03-04T17:42:02.000Z","size":9958,"stargazers_count":3464,"open_issues_count":77,"forks_count":558,"subscribers_count":150,"default_branch":"master","last_synced_at":"2024-09-27T21:01:20.088Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tensorflow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-01-29T23:29:07.000Z","updated_at":"2024-09-26T12:17:42.000Z","dependencies_parsed_at":"2022-07-15T21:30:48.228Z","dependency_job_id":null,"html_url":"https://github.com/tensorflow/minigo","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Fminigo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Fminigo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Fminigo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorflow%2Fminigo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tensorflow","download_url":"https://codeload.github.com/tensorflow/minigo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234722054,"owners_count":18876896,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T09:01:17.345Z","updated_at":"2025-09-30T08:31:45.565Z","avatar_url":"https://github.com/tensorflow.png","language":"C++","readme":"Minigo: A minimalist Go engine modeled after AlphaGo Zero, built on MuGo\n==================================================\n\nThis is an implementation of a neural-network based Go AI, using TensorFlow.\nWhile inspired by DeepMind's AlphaGo algorithm, this project is not\na DeepMind project nor is it affiliated with the official AlphaGo project.\n\n### This is NOT an official version of AlphaGo ###\n\nRepeat, *this is not the official AlphaGo program by DeepMind*.  This is an\nindependent effort by Go enthusiasts to replicate the results of the AlphaGo\nZero paper (\"Mastering the Game of Go without Human Knowledge,\" *Nature*), with\nsome resources generously made available by Google.\n\nMinigo is based off of Brian Lee's \"[MuGo](https://github.com/brilee/MuGo)\"\n-- a pure Python implementation of the first AlphaGo paper\n[\"Mastering the Game of Go with Deep Neural Networks and\nTree Search\"](https://www.nature.com/articles/nature16961) published in\n*Nature*. This implementation adds features and architecture changes present in\nthe more recent AlphaGo Zero paper, [\"Mastering the Game of Go without Human\nKnowledge\"](https://www.nature.com/articles/nature24270). More recently, this\narchitecture was extended for Chess and Shogi in [\"Mastering Chess and Shogi by\nSelf-Play with a General Reinforcement Learning\nAlgorithm\"](https://arxiv.org/abs/1712.01815).  These papers will often be\nabridged in Minigo documentation as *AG* (for AlphaGo), *AGZ* (for AlphaGo\nZero), and *AZ* (for AlphaZero) respectively.\n\n\nGoals of the Project\n==================================================\n\n1. Provide a clear set of learning examples using Tensorflow, Kubernetes, and\n   Google Cloud Platform for establishing Reinforcement Learning pipelines on\n   various hardware accelerators.\n\n2. Reproduce the methods of the original DeepMind AlphaGo papers as faithfully\n   as possible, through an open-source implementation and open-source pipeline\n   tools.\n\n3. Provide our data, results, and discoveries in the open to benefit the Go,\n   machine learning, and Kubernetes communities.\n\nAn explicit non-goal of the project is to produce a competitive Go program that\nestablishes itself as the top Go AI. Instead, we strive for a readable,\nunderstandable implementation that can benefit the community, even if that\nmeans our implementation is not as fast or efficient as possible.\n\nWhile this product might produce such a strong model, we hope to focus on the\nprocess.  Remember, getting there is half the fun. :)\n\nWe hope this project is an accessible way for interested developers to have\naccess to a strong Go model with an easy-to-understand platform of python code\navailable for extension, adaptation, etc.\n\nIf you'd like to read about our experiences training models, see [RESULTS.md](RESULTS.md).\n\nTo see our guidelines for contributing, see [CONTRIBUTING.md](CONTRIBUTING.md).\n\nGetting Started\n===============\n\nThis project assumes you have the following:\n\n- virtualenv / virtualenvwrapper\n- Python 3.5+\n- [Docker](https://docs.docker.com/install/)\n- [Cloud SDK](https://cloud.google.com/sdk/downloads)\n\nThe [Hitchhiker's guide to\npython](http://docs.python-guide.org/en/latest/dev/virtualenvs/) has a good\nintro to python development and virtualenv usage. The instructions after this\npoint haven't been tested in environments that are not using virtualenv.\n\n```shell\npip3 install virtualenv\npip3 install virtualenvwrapper\n```\n\nInstall Bazel\n------------------\n\n```shell\nBAZEL_VERSION=0.24.1\nwget https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh\nchmod 755 bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh\nsudo ./bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh\n```\n\nInstall TensorFlow\n------------------\nFirst set up and enter your virtualenv and then the shared requirements:\n\n```\npip3 install -r requirements.txt\n```\n\nThen, you'll need to choose to install the GPU or CPU tensorflow requirements:\n\n- GPU: `pip3 install \"tensorflow-gpu==1.15.0\"`.\n  - *Note*: You must install [CUDA 10.0](https://developer.nvidia.com/cuda-10.0-download-archive). for Tensorflow\n    1.13.0+.\n- CPU: `pip3 install \"tensorflow==1.15.0\"`.\n\nSetting up the Environment\n--------------------------\n\nYou may want to use a cloud project for resources. If so set:\n\n```shell\nPROJECT=foo-project\n```\n\nThen, running\n\n```shell\nsource cluster/common.sh\n```\n\nwill set up other environment variables defaults.\n\nRunning unit tests\n------------------\n```\n./test.sh\n```\n\nTo run individual modules\n\n```\nBOARD_SIZE=9 python3 tests/run_tests.py test_go\nBOARD_SIZE=19 python3 tests/run_tests.py test_mcts\n```\n\nAutomated Tests\n----------------\n\n[Test Dashboard](https://k8s-testgrid.appspot.com/sig-big-data#tf-minigo-presubmit)\n\nTo automatically test PRs, Minigo uses\n[Prow](https://github.com/kubernetes/test-infra/tree/master/prow), which is a\ntest framework created by the Kubernetes team for testing changes in a hermetic\nenvironment. We use prow for running unit tests, linting our code, and\nlaunching our test Minigo Kubernetes clusters.\n\nYou can see the status of our automated tests by looking at the Prow and\nTestgrid UIs:\n\n- Testgrid (Test Results Dashboard): https://k8s-testgrid.appspot.com/sig-big-data\n- Prow (Test-runner dashboard): https://prow.k8s.io/?repo=tensorflow%2Fminigo\n\nBasics\n======\n\nAll commands are compatible with either Google Cloud Storage as a remote file\nsystem, or your local file system. The examples here use GCS, but local file\npaths will work just as well.\n\nTo use GCS, set the `BUCKET_NAME` variable and authenticate via `gcloud login`.\nOtherwise, all commands fetching files from GCS will hang.\n\nFor instance, this would set a bucket, authenticate, and then look for the most\nrecent model.\n\n```shell\n# When you first start we recommend using our minigo-pub bucket.\n# Later you can setup your own bucket and store data there.\nexport BUCKET_NAME=minigo-pub/v9-19x19\ngcloud auth application-default login\ngsutil ls gs://$BUCKET_NAME/models | tail -4\n```\n\nWhich might look like:\n\n```\ngs://$BUCKET_NAME/models/000737-fury.data-00000-of-00001\ngs://$BUCKET_NAME/models/000737-fury.index\ngs://$BUCKET_NAME/models/000737-fury.meta\ngs://$BUCKET_NAME/models/000737-fury.pb\n```\n\nThese four files comprise the model. Commands that take a model as an\nargument usually need the path to the model basename, e.g.\n`gs://$BUCKET_NAME/models/000737-fury`\n\nYou'll need to copy them to your local disk.  This fragment copies the files\nassociated with `$MODEL_NAME` to the directory specified by `MINIGO_MODELS`:\n\n```shell\nMODEL_NAME=000737-fury\nMINIGO_MODELS=$HOME/minigo-models\nmkdir -p $MINIGO_MODELS/models\ngsutil ls gs://$BUCKET_NAME/models/$MODEL_NAME.* | \\\n       gsutil cp -I $MINIGO_MODELS/models\n```\n\nSelfplay\n--------\nTo watch Minigo play a game, you need to specify a model. Here's an example\nto play using the latest model in your bucket\n\n```shell\npython3 selfplay.py \\\n  --verbose=2 \\\n  --num_readouts=400 \\\n  --load_file=$MINIGO_MODELS/models/$MODEL_NAME\n```\n\nwhere `READOUTS` is how many searches to make per move.  Timing information and\nstatistics will be printed at each move.  Setting verbosity to 3 or\nhigher will print a board at each move.\n\nPlaying Against Minigo\n----------------------\n\nMinigo uses the\n[GTP Protocol](http://www.lysator.liu.se/~gunnar/gtp/gtp2-spec-draft2/gtp2-spec.html),\nand you can use any gtp-compliant program with it.\n\n```shell\n# Latest model should look like: /path/to/models/000123-something\nLATEST_MODEL=$(ls -d $MINIGO_MODELS/* | tail -1 | cut -f 1 -d '.')\npython3 gtp.py --load_file=$LATEST_MODEL --num_readouts=$READOUTS --verbose=3\n```\n\nAfter some loading messages, it will display `GTP engine ready`, at which point\nit can receive commands.  GTP cheatsheet:\n\n```\ngenmove [color]             # Asks the engine to generate a move for a side\nplay [color] [coordinate]   # Tells the engine that a move should be played for `color` at `coordinate`\nshowboard                   # Asks the engine to print the board.\n```\n\nOne way to play via GTP is to use gogui-display (which implements a UI that\nspeaks GTP.) You can download the gogui set of tools at\n[http://gogui.sourceforge.net/](http://gogui.sourceforge.net/). See also\n[documentation on interesting ways to use\nGTP](http://gogui.sourceforge.net/doc/reference-twogtp.html).\n\n```shell\ngogui-twogtp -black 'python3 gtp.py --load_file=$LATEST_MODEL' -white 'gogui-display' -size 19 -komi 7.5 -verbose -auto\n```\n\nAnother way to play via GTP is to watch it play against GnuGo, while\nspectating the games:\n\n```shell\nBLACK=\"gnugo --mode gtp\"\nWHITE=\"python3 gtp.py --load_file=$LATEST_MODEL\"\nTWOGTP=\"gogui-twogtp -black \\\"$BLACK\\\" -white \\\"$WHITE\\\" -games 10 \\\n  -size 19 -alternate -sgffile gnugo\"\ngogui -size 19 -program \"$TWOGTP\" -computer-both -auto\n```\n\nTraining Minigo\n======================\n\nOverview\n--------\n\nThe following sequence of commands will allow you to do one iteration of\nreinforcement learning on 9x9. These are the basic commands used to produce the\nmodels and games referenced above.\n\nThe commands are\n\n - bootstrap: initializes a random model\n - selfplay: plays games with the latest model, producing data used for training\n - train: trains a new model with the selfplay results from the most recent N\n   generations.\n\nTraining works via tf.Estimator; a working directory manages checkpoints and\ntraining logs, and the latest checkpoint is periodically exported to GCS, where\nit gets picked up by selfplay workers.\n\nConfiguration for things like \"where do debug SGFs get written\", \"where does\ntraining data get written\", \"where do the latest models get published\" are\nmanaged by the helper scripts in the rl\\_loop directory. Those helper scripts\nexecute the same commands as demonstrated below. Configuration for things like\n\"what size network is being used?\" or \"how many readouts during selfplay\" can\nbe passed in as flags. The mask\\_flags.py utility helps ensure all parts of the\npipeline are using the same network configuration.\n\nAll local paths in the examples can be replaced with `gs://` GCS paths, and the\nKubernetes-orchestrated version of the reinforcement learning loop uses GCS.\n\nBootstrap\n---------\n\nThis command initializes your working directory for the trainer and a random\nmodel. This random model is also exported to `--model-save-path` so that\nselfplay can immediately start playing with this random model.\n\nIf these directories don't exist, bootstrap will create them for you.\n\n```shell\nexport MODEL_NAME=000000-bootstrap\npython3 bootstrap.py \\\n  --work_dir=estimator_working_dir \\\n  --export_path=outputs/models/$MODEL_NAME\n```\n\nSelf-play\n---------\n\nThis command starts self-playing, outputting its raw game data as tf.Examples\nas well as in SGF form in the directories.\n\n\n```shell\npython3 selfplay.py \\\n  --load_file=outputs/models/$MODEL_NAME \\\n  --num_readouts 10 \\\n  --verbose 3 \\\n  --selfplay_dir=outputs/data/selfplay \\\n  --holdout_dir=outputs/data/holdout \\\n  --sgf_dir=outputs/sgf\n```\n\nTraining\n--------\n\nThis command takes a directory of tf.Example files from selfplay and trains a\nnew model, starting from the latest model weights in the `estimator_working_dir`\nparameter.\n\nRun the training job:\n\n```shell\npython3 train.py \\\n  outputs/data/selfplay/* \\\n  --work_dir=estimator_working_dir \\\n  --export_path=outputs/models/000001-first_generation\n```\n\nAt the end of training, the latest checkpoint will be exported to.\nAdditionally, you can follow along with the training progress with TensorBoard.\nIf you point TensorBoard at the estimator working directory, it will find the\ntraining log files and display them.\n\n```shell\ntensorboard --logdir=estimator_working_dir\n```\n\nValidation\n----------\n\nIt can be useful to set aside some games to use as a 'validation set' for\ntracking the model overfitting.  One way to do this is with the `validate`\ncommand.\n\n### Validating on holdout data\n\nBy default, Minigo will hold out 5% of selfplay games for validation. This can\nbe changed by adjusting the `holdout_pct` flag on the `selfplay` command.\n\nWith this setup, `rl_loop/train_and_validate.py` will validate on the same\nwindow of games that were used to train, writing TensorBoard logs to the\nestimator working directory.\n\n### Validating on a different set of data\n\nThis might be useful if you have some known set of 'good data' to test your\nnetwork against, e.g., a set of pro games.\nAssuming you've got a set of .sgfs with the proper komi \u0026 boardsizes, you'll\nwant to preprocess them into the .tfrecord files, by running something similar\nto\n\n```python\nimport preprocessing\nfilenames = [generate a list of filenames here]\nfor f in filenames:\n    try:\n        preprocessing.make_dataset_from_sgf(f, f.replace(\".sgf\", \".tfrecord.zz\"))\n    except:\n        print(f)\n```\n\nOnce you've collected all the files in a directory, producing validation is as\neasy as\n\n```shell\npython3 validate.py \\\n  validation_files/ \\\n  --work_dir=estimator_working_dir \\\n  --validation_name=pro_dataset\n```\n\nThe validate.py will glob all the .tfrecord.zz files under the\ndirectories given as positional arguments and compute the validation error\nfor the positions from those files.\n\n\nRetraining a model\n======================\n\nThe training data for most of Minigo's models up to v13 is publicly available in\nthe `minigo-pub` Cloud storage bucket, e.g.:\n\n```shell\ngsutil ls gs://minigo-pub/v13-19x19/data/golden_chunks/\n```\n\nFor models v14 and onwards, we started using Cloud BigTable and are still\nworking on making that data public.\n\nHere's how to retrain your own model from this source data using a Cloud TPU:\n\n```shell\n# I wrote these notes using our existing TPU-enabled project, so they're missing\n# a few preliminary steps, like setting up a Cloud account, creating a project,\n# etc. New users will also need to enable Cloud TPU on their project using the\n# TPUs panel.\n\n###############################################################################\n\n# Note that you will be billed for any storage you use and also while you have\n# VMs running. Remember to shut down your VMs when you're not using them!\n\n# To use a Cloud TPU on GCE, you need to create a special TPU-enabled VM using\n# the `ctpu` tool. First, set up some environment variables:\n#   GCE_PROJECT=\u003cyour project name\u003e\n#   GCE_VM_NAME=\u003cyour VM's name\u003e\n#   GCE_ZONE\u003cthe zone in which you want to bring uo your VM, e.g. us-central1-f\u003e\n\n# In this example, we will use the following values:\nGCE_PROJECT=example-project\nGCE_VM_NAME=minigo-etpu-test\nGCE_ZONE=us-central1-f\n\n# Create the Cloud TPU enabled VM.\nctpu up \\\n  --project=\"${GCE_PROJECT}\" \\\n  --zone=\"${GCE_ZONE}\" \\\n  --name=\"${GCE_VM_NAME}\" \\\n  --tf-version=1.13\n\n# This will take a few minutes and you should see output similar to the\n# following:\n#   ctpu will use the following configuration values:\n#         Name:                 minigo-etpu-test\n#         Zone:                 us-central1-f\n#         GCP Project:          example-project\n#         TensorFlow Version:   1.13\n#  OK to create your Cloud TPU resources with the above configuration? [Yn]: y\n#  2019/04/09 10:50:04 Creating GCE VM minigo-etpu-test (this may take a minute)...\n#  2019/04/09 10:50:04 Creating TPU minigo-etpu-test (this may take a few minutes)...\n#  2019/04/09 10:50:11 GCE operation still running...\n#  2019/04/09 10:50:12 TPU operation still running...\n\n# Once the Cloud TPU is created, `ctpu` will have SSHed you into the machine.\n\n# Remember to set the same environment variables on your VM.\nGCE_PROJECT=example-project\nGCE_VM_NAME=minigo-etpu-test\nGCE_ZONE=us-central1-f\n\n# Clone the Minigo Github repository:\ngit clone --depth 1 https://github.com/tensorflow/minigo\ncd minigo\n\n# Install virtualenv.\npip3 install virtualenv virtualenvwrapper\n\n# Create a virtual environment\nvirtualenv -p /usr/bin/python3 --system-site-packages \"${HOME}/.venvs/minigo\"\n\n# Activate the virtual environment.\nsource \"${HOME}/.venvs/minigo/bin/activate\"\n\n# Install Minigo dependencies (TensorFlow for Cloud TPU is already installed as\n# part of the VM image).\npip install -r requirements.txt\n\n# When training on a Cloud TPU, the training work directory must be on Google Cloud Storage.\n# You'll need to choose your own globally unique bucket name.\n# The bucket location should be close to your VM.\nGCS_BUCKET_NAME=minigo_test_bucket\nGCE_BUCKET_LOCATION=us-central1\ngsutil mb -p \"${GCE_PROJECT}\" -l \"${GCE_BUCKET_LOCATION}\" \"gs://${GCS_BUCKET_NAME}\"\n\n# Run the training script and note the location of the training work_dir\n# it reports, e.g.\n#    Writing to gs://minigo_test_bucket/train/2019-04-25-18\n./oneoffs/train.sh \"${GCS_BUCKET_NAME}\"\n\n# Launch tensorboard, pointing it at the work_dir reported by the train.sh script.\ntensorboard --logdir=gs://minigo_test_bucket/train/2019-04-25-18\n\n# After a few minutes, TensorBoard should start updating.\n# Interesting graphs to look at are value_cost_normalized, policy_cost and policy_entropy.\n```\n\nRunning Minigo on a Kubernetes Cluster\n==============================\n\nSee more at [cluster/README.md](https://github.com/tensorflow/minigo/tree/master/cluster/README.md)\n","funding_links":[],"categories":["Python","C++","C++ (70)","Uncategorized"],"sub_categories":["Uncategorized"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftensorflow%2Fminigo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftensorflow%2Fminigo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftensorflow%2Fminigo/lists"}