https://github.com/sayakpaul/robustness-foundation-models
This repository holds code and other relevant files for the NeurIPS 2022 tutorial: Foundational Robustness of Foundation Models.
https://github.com/sayakpaul/robustness-foundation-models
foundation-models representation-learning robustness
Last synced: 5 months ago
JSON representation
This repository holds code and other relevant files for the NeurIPS 2022 tutorial: Foundational Robustness of Foundation Models.
- Host: GitHub
- URL: https://github.com/sayakpaul/robustness-foundation-models
- Owner: sayakpaul
- License: apache-2.0
- Created: 2022-08-29T04:58:05.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2023-01-13T20:19:02.000Z (over 2 years ago)
- Last Synced: 2025-03-30T15:36:46.910Z (6 months ago)
- Topics: foundation-models, representation-learning, robustness
- Language: Jupyter Notebook
- Homepage: https://bit.ly/neurips-tut-22
- Size: 6.48 MB
- Stars: 71
- Watchers: 4
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Foundational Robustness of Foundation Models (NeurIPS 2022 tutorial)
This repository holds code and other relevant files for the NeurIPS 2022 tutorial: [**Foundational Robustness of Foundation Models**](https://sites.google.com/view/neurips2022-frfm-turotial/) by [Pin-Yu Chen (IBM Research)](https://sites.google.com/site/pinyuchenpage/home), [Sijia Liu (Michigan State University)](https://lsjxjtu.github.io/), and [Sayak Paul (Hugging Face)](https://sayak.dev).
For details on schedule and the tutorial outline, please refer to our [tutorial website](https://sites.google.com/view/neurips2022-frfm-turotial/). You can also find the tutorial listing on [IBM Research](https://research.ibm.com/publications/foundational-robustness-of-foundation-models).
**Update January 13, 2023**: Our tutorial video is now public. Find it [here](https://slideslive.com/38992789/foundational-robustness-of-foundation-models).
## Navigating the codebase
We provide code for analytical tools for two types of models: vision and code. Below provides a high-level
overview of what `code` and `vision_models` directories contain:
```bash
vision_models
├── probing_transformer_models
│ ├── attention_distance
│ ├── attention_maps
│ ├── linear_projections
│ └── positional_embeddings
├── representation_effectiveness
│ ├── fourier_heatmap
│ ├── masking
│ ├── pgd_attacks
│ └── spectral_decomposition
└── robustness_eval
```
```bash
code
├── Attack.ipynb
```
Each directory provides a standalone `README.md` with instructions about executing the
scripts / notebooks.
## Slides
You can find the slides in the `slides` directory.
## Acknowledgements
We thank [Jinghan Jia (Michigan State University)](https://www.linkedin.com/in/jinghan-jia-5194451ba) for contributing the code for evaluating "code" models.