https://github.com/microsoft/robustdg

Toolkit for building machine learning models that generalize to unseen domains and are robust to privacy and other attacks.
https://github.com/microsoft/robustdg

artificial-intelligence causality domain-generalization machine-learning privacy-preserving-machine-learning

Last synced: about 2 months ago
JSON representation

Toolkit for building machine learning models that generalize to unseen domains and are robust to privacy and other attacks.

• Host: GitHub
• URL: https://github.com/microsoft/robustdg
• Owner: microsoft
• License: mit
• Created: 2020-06-12T09:28:19.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2023-10-03T04:31:58.000Z (12 months ago)
• Last Synced: 2024-05-09T18:11:14.648Z (5 months ago)
• Topics: artificial-intelligence, causality, domain-generalization, machine-learning, privacy-preserving-machine-learning
• Language: Python
• Homepage:
• Size: 65.6 MB
• Stars: 169
• Watchers: 10
• Forks: 29
• Open Issues: 14
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md
  • Security: SECURITY.md

Awesome Lists containing this project

• awesome-machine-learning-resources - **[Library

README

        
Toolkit for Building Robust ML models that generalize to unseen domains (RobustDG)

==================================================================================

`Divyat Mahajan `_, 

`Shruti Tople `_, 

`Amit Sharma `_

`Privacy & Causal Learning (ICML 2020) `_ | `MatchDG: Causal View of DG (ICML 2021) `_ | `Privacy & DG Connection paper `_

For machine learning models to be reliable, they need to generalize to data

beyond the train distribution. In addition, ML models should be robust to

privacy attacks like membership inference and domain knowledge-based attacks like adversarial attacks.

To advance research in building robust and generalizable models, we are

releasing a toolkit for building and evaluating ML models, *RobustDG*. RobustDG contains implementations of domain

generalization algorithms and includes evaluation benchmarks based

on out-of-distribution accuracy and robustness to membership privacy attacks. We will be adding evaluation for adversarial attacks and more privacy attacks soon. 

It is easily extendable. Add your own DG algorithms and evaluate them on different benchmarks.

Installation

------------

To use the command-line interface of RobustDG, clone this repo and add the folder to your system's PATH (or alternatively, run the commands from the RobustDG root directory). 

**Load dataset**

Let's first load the rotatedMNIST dataset in a suitable format for the resnet18 architecture.

.. code:: shell

    python data/data_gen_mnist.py --dataset rot_mnist --model resnet18 --img_h 224 --img_w 224 --subset_size 2000

**Train and evaluate ML model**

The following commands would train and evalute the MatchDG method on the Rotated MNIST dataset.

.. code:: shell

    python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --epochs 50 --batch_size 64 --pos_metric cos --match_func_aug_case 1

    

    python train.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --epochs 25

    

    python test.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --epochs 25 --test_metric acc

    

    python test.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --pos_metric cos --test_metric match_score    

Demo

----

A quick introduction on how to use our repository can be accessed here in the `Getting Started notebook `_.

If you are interested in reproducing results from the MatchDG paper, check out the `Reproducing results notebook `_. 

Roadmap

-------

* Support for more domain generalization algorithms like CSD and IRM. If you are an author of a DG algorithm and would like to contribute, please raise a  pull request `here `_ or get in touch.

* More evaluation metrics based on adversarial attacks, privacy attacks like model inversion. If you'd like to see an evaluation metric implemented, please raise an issue `here `_.

Contributing

--------------

This project welcomes contributions and suggestions.  Most contributions require you to agree to a

Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us

the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide

a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions

provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the `Microsoft Open Source Code of Conduct `_.

For more information see the `Code of Conduct FAQ `_ or

contact `[email protected] `_ with any additional questions or comments.