https://github.com/thesofakillers/ponder-bayes
Official repository for the paper "PonderBayes: Uncertainty-Informed Pondering". Not published.
https://github.com/thesofakillers/ponder-bayes
Last synced: 4 months ago
JSON representation
Official repository for the paper "PonderBayes: Uncertainty-Informed Pondering". Not published.
- Host: GitHub
- URL: https://github.com/thesofakillers/ponder-bayes
- Owner: thesofakillers
- License: mit
- Created: 2022-04-28T13:47:29.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2022-10-16T17:34:41.000Z (over 3 years ago)
- Last Synced: 2025-05-07T19:03:32.104Z (about 1 year ago)
- Language: Python
- Homepage:
- Size: 7.29 MB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# PonderBayes
Official repository for the paper "PonderBayes: Rationally Pondering Neural
Networks" (Not submitted to peer review, but available for a browse in this
repository)
## Requirements and Setup
Details such as python and package versions can be found in the generated
[pyproject.toml](pyproject.toml) and [poetry.lock](poetry.lock) files.
We recommend using an environment manager such as
[conda](https://docs.conda.io/en/latest/). After setting up your environment
with the correct python version, please proceed with the installation of the
required packages
For [poetry](https://python-poetry.org/) users, getting setup is as easy as
running
```terminal
poetry install
```
We also provide a [requirements.txt](requirements.txt) file for
[pip](https://pypi.org/project/pip/) users who do not wish to use poetry. In
this case, simply run
```terminal
pip install -r requirements.txt
```
This `requirements.txt` file is generated by running the following
```terminal
sh gen_pip_reqs.sh
```
### Additional/Optional Requirements
Some additional requirements are necessary for running
[notebooks/accuracies.ipynb](notebooks/accuracies.ipynb), which is the notebook
we use for generating figures and tables:
- A complete TeXLive/MacTeX installation is required, since we make use of a TeX
backend for our figure plots.
- Our pretrained and pretested checkpoints and logs, available on
[The Internet Archive](http://archive.org/) at
[this link](https://archive.org/download/ponderbayes/models.zip)
- please unzip the checkpoints and place them as the `models/` directory under
the root of the project
## Project Organization
```plaintext
├── LICENSE
├── README.md <- The top-level README for developers using this project.
├── data/
│ ├── interim/ <- Intermediate data that has been transformed.
│ ├── processed/ <- The final, canonical data sets for modeling.
│ └── raw/ <- The original, immutable data dump.
├── models/ <- Trained and serialized models and logs
├── notebooks/ <- Jupyter notebooks.
│ └── accuracies.ipynb <- Notebook for generating figures and tables
├── reports/ <- Generated analysis as HTML, PDF, LaTeX, etc.
├── lisa/ <- LISA (slurm compute) scripts, jobs, config.
├── pyproject.toml <- project metadata, handled by poetry.
├── poetry.lock <- resolving and locking dependencies, handled by poetry.
├── requirements.txt <- for non-poetry users.
├── gen_pip_reqs.sh <- for generating the pip requirements.txt file
└── ponderbayes/ <- Source code for use in this project.
├── __init__.py <- Makes src a Python module
├── data/ <- Scripts to download or generate data
├── models/ <- Model definitions
├── run/ <- scripts to train, evaluate and use models
├── utils.py <- miscellaneous utils
└── visualization/ <- Scripts for visualization
```
The project structure is largely based on the
[cookiecutter data-science template](https://github.com/drivendata/cookiecutter-data-science).
This is purposely opinionated so that paths align over collaborators without
having to edit config files. Users may find the
[cookiecutter data-science opinions page](http://drivendata.github.io/cookiecutter-data-science/#opinions),
of relevance
The top level `data/` and `models/` directory are in version control only to
show structure. Their contents will not be committed and are ignored via
`.gitignore`.
## Usage
For training, refer to `ponderbayes/run/train.py`:
```stdout
usage: train.py [-h] [-s SEED] [--disable-logging]
[--model {pondernet,groupthink,RGT,lambdaGT,aRGT}]
[-c CHECKPOINT] [--n-elems N_ELEMS] [--n-hidden N_HIDDEN]
[--max-steps MAX_STEPS] [--lambda-p LAMBDA_P] [--beta BETA]
[--progress-bar] [--n-train-samples N_TRAIN_SAMPLES]
[--n-eval-samples N_EVAL_SAMPLES] [--mode MODE]
[--batch-size BATCH_SIZE] [--num-workers NUM_WORKERS]
[--early_stopping] [--val-check-interval VAL_CHECK_INTERVAL]
[--n-iter N_ITER] [--ensemble-size ENSEMBLE_SIZE]
Train a model
optional arguments:
-h, --help show this help message and exit
-s SEED, --seed SEED The seed to use for random number generation
--disable-logging Disable logging
--model {pondernet,groupthink,RGT,lambdaGT,aRGT}
What model variant to use
-c CHECKPOINT, --checkpoint CHECKPOINT
path to a checkpoint from which to resume training
from
--n-elems N_ELEMS Number of elements in the parity vectors
--n-hidden N_HIDDEN Number of hidden elements in the reccurent cell
--max-steps MAX_STEPS
Maximum number of pondering steps
--lambda-p LAMBDA_P Geometric prior distribution hyperparameter
--beta BETA Regularization loss coefficient
--progress-bar whether to show the progress bar
--n-train-samples N_TRAIN_SAMPLES
The number of training samples to comprising the
dataset
--n-eval-samples N_EVAL_SAMPLES
The number of training samples to comprising the
dataset
--mode MODE Whether to perform 'interpolation' or 'extrapolation'
--batch-size BATCH_SIZE
Batch size
--num-workers NUM_WORKERS
The number of workers
--early_stopping Whether to use early stopping
--val-check-interval VAL_CHECK_INTERVAL
Evaluate every x amount of steps, as opposed to every
epoch
--n-iter N_ITER Number of training steps to use
--ensemble-size ENSEMBLE_SIZE
Number of models to ensemble
```
For testing, refer to `ponderbayes/run/test.py`:
```stdout
usage: test.py [-h] [-s SEED] -c CHECKPOINT [--progress-bar]
[--n-test-samples N_TEST_SAMPLES] [--batch-size BATCH_SIZE]
[--num-workers NUM_WORKERS]
Test a pondernet checkpoint
optional arguments:
-h, --help show this help message and exit
-s SEED, --seed SEED The seed to use for random number generation
-c CHECKPOINT, --checkpoint CHECKPOINT
path (relative to root) to a checkpoint to evaluate
--progress-bar whether to show the progress bar
--n-test-samples N_TEST_SAMPLES
The number of testing samples to comprise the dataset
--batch-size BATCH_SIZE
Batch size
--num-workers NUM_WORKERS
The number of workers
```