https://github.com/ro-jefferson/entropy_dnn

Code for project on relative entropy in deep neural networks
https://github.com/ro-jefferson/entropy_dnn

Last synced: 11 days ago
JSON representation

Code for project on relative entropy in deep neural networks

• Host: GitHub
• URL: https://github.com/ro-jefferson/entropy_dnn
• Owner: ro-jefferson
• License: mit
• Created: 2021-07-14T12:05:31.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2023-09-11T16:10:35.000Z (about 1 year ago)
• Last Synced: 2024-08-01T16:46:57.108Z (3 months ago)
• Language: Jupyter Notebook
• Size: 985 KB
• Stars: 6
• Watchers: 1
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# entropy_dnn

Code for project on relative entropy in deep neural networks; corresponding paper available at https://arxiv.org/abs/2107.06898. Files include:

* `Gaussian_Feedforward.ipynb` -- jupyter notebook that creates and trains feedforward random networks used in the analysis; data written in HDF5 format.

* `Gaussian_Feedforward_Analysis.ipynb` -- jupyter notebook that reads HDF5 files created by `Gaussian_Feedforward.ipynb` and performs analysis (e.g., computes correlation length, generates plots).

* `Relative_Entropy.ipynb` -- jupyter notebook that reads HDF5 files created by `Gaussian_Feedforward.ipynb` and computes the relative entropy or Kullback-Leibler (KL) divergence as a function of depth.

It is recommended to open the `.ipynb` files with jupyter so that LaTeX expressions in markdown cells are rendered correctly.