{"id":18423305,"url":"https://github.com/xmodar/ptb","last_synced_at":"2025-04-13T14:44:46.453Z","repository":{"id":111509478,"uuid":"185709695","full_name":"xmodar/ptb","owner":"xmodar","description":"Probabilistically True and Tight Bounds for Robust Deep Neural Network Training","archived":false,"fork":false,"pushed_at":"2021-06-10T05:51:50.000Z","size":73,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-16T07:41:34.378Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xmodar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-09T02:11:46.000Z","updated_at":"2021-12-06T10:47:03.000Z","dependencies_parsed_at":"2023-11-22T07:45:13.093Z","dependency_job_id":null,"html_url":"https://github.com/xmodar/ptb","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xmodar%2Fptb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xmodar%2Fptb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xmodar%2Fptb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xmodar%2Fptb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xmodar","download_url":"https://codeload.github.com/xmodar/ptb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248732187,"owners_count":21152840,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T04:36:45.561Z","updated_at":"2025-04-13T14:44:46.430Z","avatar_url":"https://github.com/xmodar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Probabilistically True and Tight Bounds for Robust DNN Training\n\nTraining Deep Neural Networks (DNNs) that are robust to norm bounded adversarial attacks remains an elusive problem. While verification based methods are generally too expensive to robustly train large networks, it was demonstrated in Gowal et. al. that bounded input intervals can be inexpensively propagated per layer through large networks. This interval bound propagation (IBP) approach lead to high robustness and was the first to be employed on large networks. However, due to the very loose nature of the IBP bounds, particularly for large networks, the required training procedure is complex and involved. In this paper, we closely examine the bounds of a block of layers composed of an affine layer followed by a ReLU nonlinearity followed by another affine layer. In doing so, we propose _probabilistic_ bounds, true bounds with overwhelming probability, that are provably tighter than IBP bounds in expectation. We then extend this result to deeper networks through blockwise propagation and show that we can achieve orders of magnitudes tighter bounds compared to IBP. With such tight bounds, we demonstrate that a simple standard training procedure can achieve the best robustness-accuracy trade-off across several architectures on both MNIST and CIFAR10.\n\n### Usage\n\nInstall it using [poetry 0.12.11](https://github.com/sdispater/poetry) (the recommended way):\n```sh\ngit clone https://github.com/ModarTensai/ptb \u0026\u0026 cd ptb\npoetry install\n```\n\nFor Jupyter support:\n```sh\npoetry install --extras jupyter\npoetry run python -m ipykernel install --user --name ptb\n```\n\nOr using pip:\n```sh\npip install git+https://github.com/ModarTensai/ptb\n```\n\nThen, check this out:\n```\n$ ptb basic --help\nUsage: ptb basic [OPTIONS]\n\n  Start basic neural network training.\n\nOptions:\n  -v, --validate / -t, --train    Whether to run in validation or training\n                                  mode.  [default: True]\n  -d, --dataset TEXT              Which dataset to use.\n  -m, --model TEXT                Which model architecture to use.\n  -p, --pretrained / -s, --scratch\n                                  Whether to load a pretrained model.\n                                  [default: False]\n  -lr, --learning-rate FLOAT RANGE\n                                  Learning rate.\n  -mm, --momentum FLOAT RANGE     SGD momentum.\n  -w, --weight-decay FLOAT RANGE  SGD weight decay.\n  -f, --factor FLOAT RANGE        The trade-off coefficient.\n  -t, --temperature FLOAT RANGE   Softmax temperature.\n  -n, --number-of-epochs INTEGER RANGE\n                                  The maximum number of epochs.\n  -b, --batch-size INTEGER RANGE  Mini-batch size.\n  -j, --jobs INTEGER RANGE        Number of threads for data loading when\n                                  using cuda.\n  -c, --checkpoint PATH           A checkpoint file to save the best model.\n  -r, --resume PATH               A checkpoint file to resume from.\n  -l, --log-dir PATH              A tensorboard logs directory.\n  -sd, --seed INTEGER RANGE       Seed the random number generators (slow!).\n  -e, --epsilon FLOAT RANGE       Epsilon used for training with interval\n                                  bounds.\n  --help                          Show this message and exit.\n```\n\nPlease, note that in this current version, training using PTB on multiple GPUs is not supported. However, the code will try to use all available GPUs which will lead to an error with DataParallel. In case you have more than one GPU, don't forget to set the environment variable `CUDA_VISIBLE_DEVICES` to only one GPU.\n\n### Cite\n\nThis is the official implementation of the method described in [this paper](https://arxiv.org/pdf/1905.12418.pdf):\n\n```bibtex\n@misc{alsubaihi2019probabilistically,\n    title={Expected Tight Bounds for Robust Deep Neural Network Training},\n    author={Salman Alsubaihi and Adel Bibi and Modar Alfadly and Abdullah Hamdi and Bernard Ghanem},\n    year={2019},\n    eprint={1905.12418},\n    archivePrefix={arXiv},\n    primaryClass={cs.LG}\n}\n```\n\n### License\n\nMIT\n\n### Author\n\n[Modar M. Alfadly](https://modar.me/)\n\n### Contributors\n\nI would gladly accept any pull request that improves any aspect of this repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxmodar%2Fptb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxmodar%2Fptb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxmodar%2Fptb/lists"}