{"id":13416274,"url":"https://github.com/google-deepmind/sonnet","last_synced_at":"2026-03-09T08:02:03.309Z","repository":{"id":37431158,"uuid":"87067212","full_name":"google-deepmind/sonnet","owner":"google-deepmind","description":"TensorFlow-based neural network library","archived":false,"fork":false,"pushed_at":"2026-02-10T16:29:11.000Z","size":16843,"stargazers_count":9907,"open_issues_count":39,"forks_count":1304,"subscribers_count":411,"default_branch":"v2","last_synced_at":"2026-03-09T05:51:04.414Z","etag":null,"topics":["artificial-intelligence","deep-learning","machine-learning","neural-networks","tensorflow"],"latest_commit_sha":null,"homepage":"https://sonnet.dev/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google-deepmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2017-04-03T11:34:35.000Z","updated_at":"2026-03-09T04:08:27.000Z","dependencies_parsed_at":"2025-04-22T18:54:12.646Z","dependency_job_id":null,"html_url":"https://github.com/google-deepmind/sonnet","commit_stats":{"total_commits":850,"total_committers":60,"mean_commits":"14.166666666666666","dds":0.7070588235294117,"last_synced_commit":"6d5972518ab0996d15f9d18bb42f2d7169aba950"},"previous_names":["google-deepmind/sonnet","deepmind/sonnet"],"tags_count":38,"template":false,"template_full_name":null,"purl":"pkg:github/google-deepmind/sonnet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsonnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsonnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsonnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsonnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google-deepmind","download_url":"https://codeload.github.com/google-deepmind/sonnet/tar.gz/refs/heads/v2","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google-deepmind%2Fsonnet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30287446,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-09T02:57:19.223Z","status":"ssl_error","status_checked_at":"2026-03-09T02:56:26.373Z","response_time":61,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","deep-learning","machine-learning","neural-networks","tensorflow"],"created_at":"2024-07-30T21:00:56.300Z","updated_at":"2026-03-09T08:02:03.279Z","avatar_url":"https://github.com/google-deepmind.png","language":"Python","readme":"![Sonnet](https://sonnet.dev/images/sonnet_logo.png)\n\n# Sonnet\n\n[**Documentation**](https://sonnet.readthedocs.io/) | [**Examples**](#examples)\n\nSonnet is a library built on top of [TensorFlow 2](https://www.tensorflow.org/)\ndesigned to provide simple, composable abstractions for machine learning\nresearch.\n\n# Introduction\n\nSonnet has been designed and built by researchers at DeepMind. It can be used to\nconstruct neural networks for many different purposes (un/supervised learning,\nreinforcement learning, ...). We find it is a successful abstraction for our\norganization, you might too!\n\nMore specifically, Sonnet provides a simple but powerful programming model\ncentered around a single concept: `snt.Module`. Modules can hold references to\nparameters, other modules and methods that apply some function on the user\ninput. Sonnet ships with many predefined modules (e.g. `snt.Linear`,\n`snt.Conv2D`, `snt.BatchNorm`) and some predefined networks of modules (e.g.\n`snt.nets.MLP`) but users are also encouraged to build their own modules.\n\nUnlike many frameworks Sonnet is extremely unopinionated about **how** you will\nuse your modules. Modules are designed to be self contained and entirely\ndecoupled from one another. Sonnet does not ship with a training framework and\nusers are encouraged to build their own or adopt those built by others.\n\nSonnet is also designed to be simple to understand, our code is (hopefully!)\nclear and focussed. Where we have picked defaults (e.g. defaults for initial\nparameter values) we try to point out why.\n\n# Getting Started\n\n## Examples\n\nThe easiest way to try Sonnet is to use Google Colab which offers a free Python\nnotebook attached to a GPU or TPU.\n\n- [Predicting MNIST with an MLP](https://colab.research.google.com/github/deepmind/sonnet/blob/v2/examples/mlp_on_mnist.ipynb)\n- [Training a Little GAN on MNIST](https://colab.research.google.com/github/deepmind/sonnet/blob/v2/examples/little_gan_on_mnist.ipynb)\n- [Distributed training with `snt.distribute`](https://colab.research.google.com/github/deepmind/sonnet/blob/v2/examples/distributed_cifar10.ipynb)\n\n## Installation\n\nTo get started install TensorFlow 2.0 and Sonnet 2:\n\n```shell\n$ pip install tensorflow tensorflow-probability\n$ pip install dm-sonnet\n```\n\nYou can run the following to verify things installed correctly:\n\n```python\nimport tensorflow as tf\nimport sonnet as snt\n\nprint(\"TensorFlow version {}\".format(tf.__version__))\nprint(\"Sonnet version {}\".format(snt.__version__))\n```\n\n### Using existing modules\n\nSonnet ships with a number of built in modules that you can trivially use. For\nexample to define an MLP we can use the `snt.Sequential` module to call a\nsequence of modules, passing the output of a given module as the input for the\nnext module. We can use `snt.Linear` and `tf.nn.relu` to actually define our\ncomputation:\n\n```python\nmlp = snt.Sequential([\n    snt.Linear(1024),\n    tf.nn.relu,\n    snt.Linear(10),\n])\n```\n\nTo use our module we need to \"call\" it. The `Sequential` module (and most\nmodules) define a `__call__` method that means you can call them by name:\n\n```python\nlogits = mlp(tf.random.normal([batch_size, input_size]))\n```\n\nIt is also very common to request all the parameters for your module. Most\nmodules in Sonnet create their parameters the first time they are called with\nsome input (since in most cases the shape of the parameters is a function of\nthe input). Sonnet modules provide two properties for accessing parameters.\n\nThe `variables` property returns **all** `tf.Variable`s that are referenced by\nthe given module:\n\n```python\nall_variables = mlp.variables\n```\n\nIt is worth noting that `tf.Variable`s are not just used for parameters of your\nmodel. For example they are used to hold state in metrics used in\n`snt.BatchNorm`. In most cases users retrieve the module variables to pass them\nto an optimizer to be updated. In this case non-trainable variables should\ntypically not be in that list as they are updated via a different mechanism.\nTensorFlow has a built in mechanism to mark variables as \"trainable\" (parameters\nof your model) vs. non-trainable (other variables). Sonnet provides a mechanism\nto gather all trainable variables from your module which is probably what you\nwant to pass to an optimizer:\n\n```python\nmodel_parameters = mlp.trainable_variables\n```\n\n### Building your own module\n\nSonnet strongly encourages users to subclass `snt.Module` to define their own\nmodules. Let's start by creating a simple `Linear` layer called `MyLinear`:\n\n```python\nclass MyLinear(snt.Module):\n\n  def __init__(self, output_size, name=None):\n    super(MyLinear, self).__init__(name=name)\n    self.output_size = output_size\n\n  @snt.once\n  def _initialize(self, x):\n    initial_w = tf.random.normal([x.shape[1], self.output_size])\n    self.w = tf.Variable(initial_w, name=\"w\")\n    self.b = tf.Variable(tf.zeros([self.output_size]), name=\"b\")\n\n  def __call__(self, x):\n    self._initialize(x)\n    return tf.matmul(x, self.w) + self.b\n```\n\nUsing this module is trivial:\n\n```python\nmod = MyLinear(32)\nmod(tf.ones([batch_size, input_size]))\n```\n\nBy subclassing `snt.Module` you get many nice properties for free. For example\na default implementation of `__repr__` which shows constructor arguments (very\nuseful for debugging and introspection):\n\n```python\n\u003e\u003e\u003e print(repr(mod))\nMyLinear(output_size=10)\n```\n\nYou also get the `variables` and `trainable_variables` properties:\n\n```python\n\u003e\u003e\u003e mod.variables\n(\u003ctf.Variable 'my_linear/b:0' shape=(10,) ...)\u003e,\n \u003ctf.Variable 'my_linear/w:0' shape=(1, 10) ...)\u003e)\n```\n\nYou may notice the `my_linear` prefix on the variables above. This is because\nSonnet modules also enter the modules name scope whenever methods are called.\nBy entering the module name scope we provide a much more useful graph for tools\nlike TensorBoard to consume (e.g. all operations that occur inside my_linear\nwill be in a group called my_linear).\n\nAdditionally your module will now support TensorFlow checkpointing and saved\nmodel which are advanced features covered later.\n\n# Serialization\n\nSonnet supports multiple serialization formats. The simplest format we support\nis Python's `pickle`, and all built in modules are tested to make sure they can\nbe saved/loaded via pickle in the same Python process. In general we discourage\nthe use of pickle, it is not well supported by many parts of TensorFlow and in\nour experience can be quite brittle.\n\n## TensorFlow Checkpointing\n\n**Reference:** https://www.tensorflow.org/alpha/guide/checkpoints\n\nTensorFlow checkpointing can be used to save the value of parameters\nperiodically during training. This can be useful to save the progress of\ntraining in case your program crashes or is stopped. Sonnet is designed to work\ncleanly with TensorFlow checkpointing:\n\n```python\ncheckpoint_root = \"/tmp/checkpoints\"\ncheckpoint_name = \"example\"\nsave_prefix = os.path.join(checkpoint_root, checkpoint_name)\n\nmy_module = create_my_sonnet_module()  # Can be anything extending snt.Module.\n\n# A `Checkpoint` object manages checkpointing of the TensorFlow state associated\n# with the objects passed to it's constructor. Note that Checkpoint supports\n# restore on create, meaning that the variables of `my_module` do **not** need\n# to be created before you restore from a checkpoint (their value will be\n# restored when they are created).\ncheckpoint = tf.train.Checkpoint(module=my_module)\n\n# Most training scripts will want to restore from a checkpoint if one exists. This\n# would be the case if you interrupted your training (e.g. to use your GPU for\n# something else, or in a cloud environment if your instance is preempted).\nlatest = tf.train.latest_checkpoint(checkpoint_root)\nif latest is not None:\n  checkpoint.restore(latest)\n\nfor step_num in range(num_steps):\n  train(my_module)\n\n  # During training we will occasionally save the values of weights. Note that\n  # this is a blocking call and can be slow (typically we are writing to the\n  # slowest storage on the machine). If you have a more reliable setup it might be\n  # appropriate to save less frequently.\n  if step_num and not step_num % 1000:\n    checkpoint.save(save_prefix)\n\n# Make sure to save your final values!!\ncheckpoint.save(save_prefix)\n```\n\n## TensorFlow Saved Model\n\n**Reference:** https://www.tensorflow.org/alpha/guide/saved_model\n\nTensorFlow saved models can be used to save a copy of your network that is\ndecoupled from the Python source for it. This is enabled by saving a TensorFlow\ngraph describing the computation and a checkpoint containing the value of\nweights.\n\nThe first thing to do in order to create a saved model is to create a\n`snt.Module` that you want to save:\n\n```python\nmy_module = snt.nets.MLP([1024, 1024, 10])\nmy_module(tf.ones([1, input_size]))\n```\n\nNext, we need to create another module describing the specific parts of our\nmodel that we want to export. We advise doing this (rather than modifying the\noriginal model in-place) so you have fine grained control over what is actually\nexported. This is typically important to avoid creating very large saved models,\nand such that you only share the parts of your model you want to (e.g. you only\nwant to share the generator for a GAN but keep the discriminator private).\n\n```python\n@tf.function(input_signature=[tf.TensorSpec([None, input_size])])\ndef inference(x):\n  return my_module(x)\n\nto_save = snt.Module()\nto_save.inference = inference\nto_save.all_variables = list(my_module.variables)\ntf.saved_model.save(to_save, \"/tmp/example_saved_model\")\n```\n\nWe now have a saved model in the `/tmp/example_saved_model` folder:\n\n```shell\n$ ls -lh /tmp/example_saved_model\ntotal 24K\ndrwxrwsr-t 2 tomhennigan 154432098 4.0K Apr 28 00:14 assets\n-rw-rw-r-- 1 tomhennigan 154432098  14K Apr 28 00:15 saved_model.pb\ndrwxrwsr-t 2 tomhennigan 154432098 4.0K Apr 28 00:15 variables\n```\n\nLoading this model is simple and can be done on a different machine without any\nof the Python code that built the saved model:\n\n```python\nloaded = tf.saved_model.load(\"/tmp/example_saved_model\")\n\n# Use the inference method. Note this doesn't run the Python code from `to_save`\n# but instead uses the TensorFlow Graph that is part of the saved model.\nloaded.inference(tf.ones([1, input_size]))\n\n# The all_variables property can be used to retrieve the restored variables.\nassert len(loaded.all_variables) \u003e 0\n```\n\nNote that the loaded object is not a Sonnet module, it is a container object\nthat has the specific methods (e.g. `inference`) and properties (e.g.\n`all_variables`) that we added in the previous block.\n\n## Distributed training\n\n**Example:** https://github.com/deepmind/sonnet/blob/v2/examples/distributed_cifar10.ipynb\n\nSonnet has support for distributed training using\n[custom TensorFlow distribution strategies](https://sonnet.readthedocs.io/en/latest/api.html#module-sonnet.distribute).\n\nA key difference between Sonnet and distributed training using `tf.keras` is\nthat Sonnet modules and optimizers do not behave differently when run under\ndistribution strategies (e.g. we do not average your gradients or sync your\nbatch norm stats). We believe that users should be in full control of these\naspects of their training and they should not be baked into the library. The\ntrade off here is that you need to implement these features in your training\nscript (typically this is just 2 lines of code to all reduce your gradients\nbefore applying your optimizer) or swap in modules that are explicitly\ndistribution aware (e.g. `snt.distribute.CrossReplicaBatchNorm`).\n\nOur [distributed Cifar-10](https://github.com/deepmind/sonnet/blob/v2/examples/distributed_cifar10.ipynb)\nexample walks through doing multi-GPU training with Sonnet.\n","funding_links":[],"categories":["Python","By Industry","其他_机器学习与深度学习","🤖 Machine Learning \u0026 AI","By Language","Computation and Communication Optimisation"],"sub_categories":["Data Science","Tools","Python"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fsonnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle-deepmind%2Fsonnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle-deepmind%2Fsonnet/lists"}