{"id":13653340,"url":"https://github.com/ejmichaud/torch-foresight","last_synced_at":"2025-04-23T06:31:37.507Z","repository":{"id":113822732,"uuid":"202230007","full_name":"ejmichaud/torch-foresight","owner":"ejmichaud","description":"🧠:mag: Tools for characterizing and predicting the dynamics of neural nets built with PyTorch","archived":false,"fork":false,"pushed_at":"2024-07-25T10:16:46.000Z","size":731,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-10T04:36:24.994Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://torch-foresight.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ejmichaud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-13T22:03:34.000Z","updated_at":"2023-05-09T16:07:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"6d0f1c49-5fc3-4fe6-9935-58fea85f0544","html_url":"https://github.com/ejmichaud/torch-foresight","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ejmichaud%2Ftorch-foresight","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ejmichaud%2Ftorch-foresight/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ejmichaud%2Ftorch-foresight/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ejmichaud%2Ftorch-foresight/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ejmichaud","download_url":"https://codeload.github.com/ejmichaud/torch-foresight/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250385041,"owners_count":21421836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T02:01:08.917Z","updated_at":"2025-04-23T06:31:32.492Z","avatar_url":"https://github.com/ejmichaud.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"```\n _______             _\n|__   __|           | |                               _---~~(~~-_.\n   | | ___  _ __ ___| |__ ______                    _{        )   )\n   | |/ _ \\| '__/ __| '_ \\______|                 ,   ) -~~- ( ,-' )_\n   | | (_) | | | (__| | | |                      (  `-,_..`., )-- '_,)\n __|_|\\___/|_|  \\___|_| |_|      _     _        ( ` _)  (  -~( -_ `,  }\n|  ____|                (_)     | |   | |       (_-  _  ~_-~~~~`,  ,' )\n| |__ ___  _ __ ___  ___ _  __ _| |__ | |_        `~ -\u003e(    __;-,((()))\n|  __/ _ \\| '__/ _ \\/ __| |/ _` | '_ \\| __|             ~~~~ {_ -_(())\n| | | (_) | | |  __/\\__ \\ | (_| | | | | |_                     `\\  }\n|_|  \\___/|_|  \\___||___/_|\\__, |_| |_|\\__|                      { }\n                            __/ |\n                           |___/\n```\n[![Documentation Status](https://readthedocs.org/projects/torch-foresight/badge/?version=latest)](https://torch-foresight.readthedocs.io/en/latest/?badge=latest) [![Build Status](https://travis-ci.org/ejmichaud/torch-foresight.svg?branch=master)](https://travis-ci.org/ejmichaud/torch-foresight)\n\nThis package provides a collection of modules useful for characterizing and predicting the dynamics and performance of neural nets. These consist mostly of novel metrics, derived from fields like theoretical neuroscience and information theory, aimed at helping researchers to better understand how neural networks work. The repository is meant to advance a new \"Science of AI\" or \"Science of Deep Learning\" (see [neuralnet.science](https://neuralnet.science)). It currently includes modules for computing:\n\n* [Effective Information](https://arxiv.org/abs/1907.03902)\n\nWith the following under development:\n- [ ] [Gradient Noise Scale](https://openai.com/blog/science-of-ai/)\n- [ ] [Information Bottleneck](https://arxiv.org/abs/1503.02406)\n\n**[Check out the comprehensive documentation (click me!)](https://torch-foresight.readthedocs.io)**\n\n## Installation\n\nThe package currently only supports Python 3 (3.5-3.7). Pytorch is required as a dependency. If pytorch is already installed, simply use:\n\n```\npip install git+https://github.com/ejmichaud/torch-foresight.git\n```\n\nIf you don't have pytorch, installing it with anaconda is recommended. An `environment.yml` has been provided. Use it like so:\n```\nconda env create -f environment.yml\n```\nThis will create a conda environment called \"foresight\", and install pytorch and then this package. It can then be activated with `conda activate foresight`. A requirements.txt file has also been provided, if you'd like to use `pip install -r requirements.txt`, which will install pytorch via pip. \n\n## Usage:\n\nComputing effective information:\n\n```python\nimport foresight.ei as ei\n\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\n... (define model, data loaders) ...\n\ninput = next(iter(data_loader))[0].to(device) # get a batch to run model on\nEI = ei.ei(model, input=input, device=device)\n```\n\nEffective information may prove a useful metric in characterizing the learning (generalization) and overfitting phases of a neural network. Here is an example of how it evolves during the training of a single layer (no hidden layers) softmax network:\n\n\u003cp align=\"center\"\u003e\n\u003cimg width=\"75%\" src=\"docs/figures/single-layer-softmax-graph.png\"\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fejmichaud%2Ftorch-foresight","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fejmichaud%2Ftorch-foresight","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fejmichaud%2Ftorch-foresight/lists"}