{"id":28753451,"url":"https://github.com/google-deepmind/spiral","last_synced_at":"2025-06-17T00:40:10.457Z","repository":{"id":50445374,"uuid":"135987199","full_name":"google-deepmind/spiral","owner":"google-deepmind","description":"We provide a pre-trained model for unconditional 19-step generation of CelebA-HQ images","archived":false,"fork":false,"pushed_at":"2019-10-29T14:04:09.000Z","size":4508,"stargazers_count":326,"open_issues_count":5,"forks_count":45,"subscribers_count":16,"default_branch":"master","last_synced_at":"2024-12-06T20:31:30.480Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-deepmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-04T07:28:02.000Z","updated_at":"2024-09-11T07:36:00.000Z","dependencies_parsed_at":"2022-09-17T00:23:07.403Z","dependency_job_id":null,"html_url":"https://github.com/google-deepmind/spiral","commit_stats":null,"previous_names":["google-deepmind/spiral","deepmind/spiral"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/google-deepmind/spiral","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fspiral","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fspiral/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fspiral/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fspiral/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-deepmind","download_url":"https://codeload.github.com/google-deepmind/spiral/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fspiral/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260268635,"owners_count":22983601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-17T00:40:08.885Z","updated_at":"2025-06-17T00:40:10.379Z","avatar_url":"https://github.com/google-deepmind.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SPIRAL\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"75%\" src=\"media/default_wgangp_celebahq64_gen_19steps.gif\"\u003e\n\u003c/p\u003e\n\n## Overview\n\nThis repository contains agents and environments described in the ICML'18\npaper [\"Synthesizing Programs for Images using Reinforced Adversarial Learning\"](http://proceedings.mlr.press/v80/ganin18a.html).\nFor the time being, we are providing two simulators:\none based on [`libmypaint`](https://github.com/mypaint/libmypaint) and one\nbased on [`Fluid Paint`](https://github.com/dli/paint) (**NOTE:** our\nimplementation is written in `C++` whereas the original is in `javascript`).\nAdditionally, we supply a [Sonnet](https://github.com/deepmind/sonnet) module\nfor the unconditional agent as well as\n[pre-trained model snapshots](https://tfhub.dev/s?q=spiral)\n(9 agents from a single population for `libmypaint` and 1 agent for\n`Fluid Paint`) available from [TF-Hub](https://www.tensorflow.org/hub).\n\nIf you feel an immediate urge to dive into the code the most relevant files are:\n\n| Path | Description |\n| :--- | :--- |\n| [`spiral/agents/default.py`](spiral/agents/default.py) | The architecture of the agent |\n| [`spiral/environments/libmypaint.py`](spiral/environments/libmypaint.py) | The `libmypaint`-based environment |\n| [`spiral/environments/fluid.py`](spiral/environments/fluid.py) | The `Fluid Paint`-based environment |\n\n## Reference\n\nIf this repository is helpful for your research please cite the following\npublication:\n\n```\n@inproceedings{ganin2018synthesizing,\n  title={Synthesizing Programs for Images using Reinforced Adversarial Learning},\n  author={Ganin, Yaroslav and Kulkarni, Tejas and Babuschkin, Igor and Eslami, SM Ali and Vinyals, Oriol},\n  booktitle={ICML},\n  year={2018}\n}\n```\n\n## Quickstart with Docker\n\nThe easiest way to get SPIRAL up and running on your machine is to use a\npre-built [Docker image](https://github.com/ddtm/spiral-docker).\n\n## Installation\n\nThis section describes how to build and install the package on Ubuntu (16.04 or\nnewer). The following instructions (with slight modifications) might also work\nfor other Linux distributions.\n\nClone this repository and fetch the external submodules:\n\n```shell\ngit clone https://github.com/deepmind/spiral.git\ncd spiral\ngit submodule update --init --recursive\n```\n\nInstall required packages:\n\n```shell\napt-get install cmake pkg-config protobuf-compiler libjson-c-dev intltool libpython3-dev python3-pip\npip3 install six setuptools numpy scipy tensorflow==1.14 tensorflow-hub dm-sonnet==1.35\n```\n\n**WARNING:** Make sure that you have `cmake` **3.14** or later since we rely\non its capability to find `numpy` libraries. If your package manager doesn't\nprovide it follow the installation instructions from\n[here](https://cmake.org/install/). You can check the version by\nrunning `cmake --version `.\n\nFinally, run the following command to install the SPIRAL package itself:\n\n```shell\npython3 setup.py develop --user\n```\n\nYou will also need to obtain the brush files for the `libmypaint` environment\nto work properly. These can be found\n[here](https://github.com/mypaint/mypaint-brushes). For example, you can\nplace them in `third_party` folder like this:\n\n```shell\nwget -c https://github.com/mypaint/mypaint-brushes/archive/v1.3.0.tar.gz -O - | tar -xz -C third_party\n```\n\nFinally, the `Fluid Paint` environment depends on the shaders from the original\n`javascript` [implementation](https://github.com/dli/paint). You can obtain\nthem by running the following commands:\n\n```shell\ngit clone https://github.com/dli/paint third_party/paint\npatch third_party/paint/shaders/setbristles.frag third_party/paint-setbristles.patch\n```\n\nOptionally, in order to be able to try out the package in the provided\n`jupyter` [notebook](notebooks/spiral-demo.ipynb), you’ll need to install\nthe following packages:\n\n```shell\npip3 install matplotlib jupyter\n```\n\n## Usage\n\nFor a basic example of how to use the package please follow\n[this notebook](notebooks/spiral-demo.ipynb).\n\n### Sampling from a pre-trained model\n\nWe provide [pre-trained models](https://tfhub.dev/s?q=spiral) for unconditional\n19-step generation of [CelebA-HQ](https://github.com/tkarras/progressive_growing_of_gans)\nimages. Here is an example of how you can sample from an agent interacting\nwith the `libmypaint` environment:\n\n```python\nimport matplotlib.pyplot as plt\n\nimport spiral.agents.default as default_agent\nimport spiral.agents.utils as agent_utils\nimport spiral.environments.libmypaint as libmypaint\n\n\n# The path to a TF-Hub module.\nMODULE_PATH = \"https://tfhub.dev/deepmind/spiral/default-wgangp-celebahq64-gen-19steps/agent4/1\"\n# The folder containing `libmypaint` brushes.\nBRUSHES_PATH = \"the/path/to/libmypaint-brushes\"\n\n# Here, we create an environment.\nenv = libmypaint.LibMyPaint(episode_length=20,\n                            canvas_width=64,\n                            grid_width=32,\n                            brush_type=\"classic/dry_brush\",\n                            brush_sizes=[1, 2, 4, 8, 12, 24],\n                            use_color=True,\n                            use_pressure=True,\n                            use_alpha=False,\n                            background=\"white\",\n                            brushes_basedir=BRUSHES_PATH)\n\n\n# Now we load the agent from a snapshot.\ninitial_state, step = agent_utils.get_module_wrappers(MODULE_PATH)\n\n# Everything is ready for sampling.\nstate = initial_state()\nnoise_sample = np.random.normal(size=(10,)).astype(np.float32)\n\ntime_step = env.reset()\nfor t in range(19):\n    time_step.observation[\"noise_sample\"] = noise_sample\n    action, state = step(time_step.step_type, time_step.observation, state)\n    time_step = env.step(action)\n\n# Show the sample.\nplt.close(\"all\")\nplt.imshow(time_step.observation[\"canvas\"], interpolation=\"nearest\")\n```\n\n### Converting a trained agent into a TF-Hub module\n\n```python\nimport spiral.agents.default as default_agent\nimport spiral.agents.utils as agent_utils\nimport spiral.environments.libmypaint as libmypaint\n\n\n# This where we're going to put our TF-Hub module.\nTARGET_PATH = ...\n# A path to a checkpoint of the trained model.\nCHECKPOINT_PATH = ...\n\n# We will need to create an environment in order to obtain the specifications\n# for the agent's action and the observation.\nenv = libmypaint.LibMyPaint(...)\n\n# Here, we wrap a Sonnet module constructor for our agent in a function.\n# This is to avoid contaminating the default tensorflow graph.\ndef agent_ctor():\n  return default_agent.Agent(action_spec=env.action_spec(),\n                             input_shape=(64, 64),\n                             grid_shape=(32, 32),\n                             action_order=\"libmypaint\")\n\n# Finally, export a TF-Hub module. We need to specify which checkpoint to use\n# to extract the weights for the agent. Since the variable names in the\n# checkpoint may differ from the names in the Sonnet module produced by\n# `agent_ctor`, we may also want to provide an appropriate name mapping\n# function.\nagent_utils.export_hub_module(agent_ctor=agent_ctor,\n                              observation_spec=env.observation_spec(),\n                              noise_dim=10,\n                              module_path=TARGET_PATH,\n                              checkpoint_path=CHECKPOINT_PATH,\n                              name_transform_fn=lambda name: ...)\n```\n\n## Disclaimer\n\nThis is not an official Google product.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fspiral","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-deepmind%2Fspiral","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fspiral/lists"}