{"id":19846107,"url":"https://github.com/koukyosyumei/aijack","last_synced_at":"2025-04-04T10:07:09.515Z","repository":{"id":37419304,"uuid":"330113257","full_name":"Koukyosyumei/AIJack","owner":"Koukyosyumei","description":"Security and Privacy Risk Simulator for Machine Learning (arXiv:2312.17667)","archived":false,"fork":false,"pushed_at":"2024-04-19T05:20:58.000Z","size":159554,"stargazers_count":325,"open_issues_count":11,"forks_count":62,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-05-02T03:38:59.914Z","etag":null,"topics":["adversarial-attacks","adversarial-examples","adversarial-machine-learning","dbms","deep-learning","differential-privacy","evasion-attack","federated-learning","homomorphic-encryption","k-anonymity","machine-learning","membership-inference","model-inversion-attacks","paillier","paillier-cryptosystem","poisoning-attacks","privacy","security"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Koukyosyumei.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["Koukyosyumei"]}},"created_at":"2021-01-16T07:59:53.000Z","updated_at":"2024-04-25T10:12:45.000Z","dependencies_parsed_at":"2024-05-12T02:45:24.044Z","dependency_job_id":null,"html_url":"https://github.com/Koukyosyumei/AIJack","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Koukyosyumei%2FAIJack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Koukyosyumei%2FAIJack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Koukyosyumei%2FAIJack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Koukyosyumei%2FAIJack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Koukyosyumei","download_url":"https://codeload.github.com/Koukyosyumei/AIJack/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247156936,"owners_count":20893201,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adversarial-attacks","adversarial-examples","adversarial-machine-learning","dbms","deep-learning","differential-privacy","evasion-attack","federated-learning","homomorphic-encryption","k-anonymity","machine-learning","membership-inference","model-inversion-attacks","paillier","paillier-cryptosystem","poisoning-attacks","privacy","security"],"created_at":"2024-11-12T13:10:27.509Z","updated_at":"2025-04-04T10:07:09.494Z","avatar_url":"https://github.com/Koukyosyumei.png","language":"C++","funding_links":["https://github.com/sponsors/Koukyosyumei"],"categories":[],"sub_categories":[],"readme":"\u003c!--\n  Title: AIJack\n  Description: AIJack is a fantastic framework demonstrating the security risks of machine learning and deep learning, such as Model Inversion, poisoning attack, and membership inference attack.\n  Author: Hideaki Takahashi\n  --\u003e\n\n# AIJack: Security and Privacy Risk Simulator for Machine Learning\n\n\u003cdiv align=\"left\"\u003e\n\u003cimg src=\"https://badgen.net/github/stars/Koukyosyumei/AIjack?color=green\"\u003e\n\u003cimg src=\"https://badgen.net/github/forks/Koukyosyumei/AIjack\"\u003e\n\u003cimg src=\"https://badgen.net/github/watchers/Koukyosyumei/AIjack\"\u003e\n\u003cimg src=\"https://img.shields.io/github/commit-activity/m/Koukyosyumei/AIJack\"\u003e\n\u003cimg src=\"https://img.shields.io/github/languages/code-size/Koukyosyumei/AIJack\"\u003e\n\u003cimg src=\"https://app.codacy.com/project/badge/Grade/529a33e6523e4bf2a79c26e815a070f4\"/\u003e\n\u003cimg src=\"https://img.shields.io/github/license/Koukyosyumei/AIJack\"\u003e\n\u003c/div\u003e\n\n❤️ \u003ci\u003eIf you like AIJack, please consider \u003ca href=\"https://github.com/sponsors/Koukyosyumei\"\u003ebecoming a GitHub Sponsor\u003c/a\u003e\u003c/i\u003e ❤️\n\n\u003ch1 align=\"center\"\u003e\n\n  \u003cbr\u003e\n  \u003cimg src=\"docs/source/_static/logo_wide_png.png\" alt=\"\" width=1000\u003e\n  \u003cbr\u003e\n  \u003cbr\u003e\n\n\u003c/h1\u003e\n\n# What is AIJack?\n\nAIJack is an easy-to-use open-source simulation tool for testing the security of your AI system against hijackers. It provides advanced security techniques like *Differential Privacy*, *Homomorphic Encryption*, *K-anonymity* and *Federated Learning* to guarantee protection for your AI. With AIJack, you can test and simulate defenses against various attacks such as *Poisoning*, *Model Inversion*, *Backdoor*, and *Free-Rider*. We support more than 30 state-of-the-art methods. For more information, check our [paper](https://arxiv.org/abs/2312.17667) and [documentation](https://koukyosyumei.github.io/AIJack/) and start securing your AI today with AIJack.\n\n# Installation\n\nYou can install AIJack with `pip`. AIJack requires Boost and pybind11.\n\n```\napt install -y libboost-all-dev\npip install -U pip\npip install \"pybind11[global]\"\n\npip install aijack\n```\n\nIf you want to use the latest-version, you can directly install from GitHub.\n\n```\npip install git+https://github.com/Koukyosyumei/AIJack\n```\n\nWe also provide [Dockerfile](Dockerfile).\n\n\n# Quick Start\n\nWe briefly introduce the overview of AIJack.\n\n## Features\n\n-   All-around abilities for both attack \u0026 defense\n-   PyTorch-friendly design\n-   Compatible with scikit-learn\n-   Fast Implementation with C++ backend\n-   MPI-Backend for Federated Learning\n-   Extensible modular APIs\n\n## Basic Interface\n\n### Python API\n\nFor standard machine learning algorithms, AIJack allows you to simulate attacks against machine learning models with `Attacker` APIs. AIJack mainly supports PyTorch or sklearn models.\n\n```Python\n# abstract code\n\nattacker = Attacker(target_model)\nresult = attacker.attack()\n```\n\nFor instance, we can implement Poisoning Attack against SVM implemented with sklearn as follows.\n\n```Python\nfrom aijack . attack import Poison_attack_sklearn\n\nattacker = Poison_attack_sklearn(clf , X_train , y_train)\nmalicious_data , log = attacker.attack(initial_data , 1, X_valid , y_valid)\n```\n\nFor distributed learning such as Federated Learning and Split Learning, AIJack offers four basic APIs: `Client`, `Server`, `API`, and `Manager`. `Client` and `Server` represent each client and server within each distributed learning scheme. You can execute training by registering the clients and servers to `API` and running it. `Manager` gives additional abilities such as attack, defense, or parallel computing to `Client`, `Server` or `API` via `attach` method.\n\n```Python\n# abstract code\n\nclient = [Client(), Client()]\nserver = Server()\napi = API(client, server)\napi.run() # execute training\n\nc_manager = ClientManagerForAdditionalAbility(...)\ns_manager = ServerManagerForAdditionalAbility(...)\nExtendedClient = c_manager.attach(Client)\nExtendedServer = c_manager.attach(Server)\n\nextended_client = [ExtendedClient(...), ExtendedClient(...)]\nextended_server = ExtendedServer(...)\napi = API(extended_client, extended_server)\napi.run() # execute training\n```\n\nFor example, the bellow code implements the scenario where the server in Federated Learning tries to steal the training data with gradient-based model inversion attack.\n\n```Python\nfrom aijack.collaborative.fedavg import FedAVGAPI, FedAVGClient, FedAVGServer\nfrom aijack.attack.inversion import GradientInversionAttackServerManager\n\nmanager = GradientInversionAttackServerManager(input_shape)\nFedAVGServerAttacker = manager.attach(FedAVGServer)\n\nclients = [FedAVGClient(model_1), FedAVGClient(model_2)]\nserver = FedAVGServerAttacker(clients, model_3)\n\napi = FedAVGAPI(server, clients, criterion, optimizers, dataloaders)\napi.run()\n```\n\n### AIValut: A simple DBMS for debugging ML Models\n\nWe also provide a simple DBMS named `AIValut` designed specifically for SQL-based algorithms. AIValut currently supports Rain, a SQL-based debugging system for ML models. In the future, we have plans to integrate additional advanced features from AIJack, including K-Anonymity, Homomorphic Encryption, and Differential Privacy. \n\nAIValut has its own storage engine and query parser, and you can train and debug ML models with SQL-like queries. For example, the `Complaint` query automatically removes problematic records given the specified constraint.\n\n```sql\n# We train an ML model to classify whether each customer will go bankrupt or not based on their age and debt.\n# We want the trained model to classify the customer as positive when he/she has more debt than or equal to 100.\n# The 10th record seems problematic for the above constraint.\n\u003e\u003eSelect * From bankrupt\nid age debt y\n1 40 0 0\n2 21 10 0\n3 22 10 0\n4 32 30 0\n5 44 50 1\n6 30 100 1\n7 63 310 1\n8 53 420 1\n9 39 530 1\n10 49 1000 0\n\n# Train Logistic Regression with the number of iterations of 100 and the learning rate of 1.\n# The name of the target feature is `y`, and we use all other features as training data.\n\u003e\u003eLogreg lrmodel id y 100 1 From Select * From bankrupt\nTrained Parameters:\n (0) : 2.771564\n (1) : -0.236504\n (2) : 0.967139\nAUC: 0.520000\nPrediction on the training data is stored at `prediction_on_training_data_lrmodel`\n\n# Remove one record so that the model will predict `positive (class 1)` for the samples with `debt` greater or equal to 100.\n\u003e\u003eComplaint comp Shouldbe 1 Remove 1 Against Logreg lrmodel id y 100 1 From Select * From bankrupt Where debt Geq 100\nFixed Parameters:\n (0) : -4.765492\n (1) : 8.747224\n (2) : 0.744146\nAUC: 1.000000\nPrediction on the fixed training data is stored at `prediction_on_training_data_comp_lrmodel`\n```\n\nFor more detailed information and usage instructions, please refer to [aivalut/README.md](aivalut/README.md).\n\n\u003e Please use AIValut only for research purpose. \n\n## Resources\n\nYou can also find more examples in our tutorials and documentation.\n\n-   [Examples](docs/source/notebooks)\n-   [Documentation](https://koukyosyumei.github.io/AIJack/)\n-   [API Reference](https://koukyosyumei.github.io/AIJack/api.html)\n\n# Supported Algorithms\n\n|               |                        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |\n| ------------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Collaborative | Horizontal FL          | [FedAVG](https://arxiv.org/abs/1602.05629), [FedProx](https://arxiv.org/abs/1812.06127), [FedKD](https://arxiv.org/abs/2108.13323), [FedGEMS](https://arxiv.org/abs/2110.11027), [FedMD](https://arxiv.org/abs/1910.03581), [DSFL](https://arxiv.org/abs/2008.06180), [MOON](https://arxiv.org/abs/2103.16257), [FedExP](https://arxiv.org/abs/2301.09604)                                                                                                                                                                                                                                                                                             |\n| Collaborative | Vertical FL            | [SplitNN](https://arxiv.org/abs/1812.00564), [SecureBoost](https://arxiv.org/abs/1901.08755)                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |\n| Attack        | Model Inversion        | [MI-FACE](https://dl.acm.org/doi/pdf/10.1145/2810103.2813677), [DLG](https://papers.nips.cc/paper/2019/hash/60a6c4002cc7b29142def8871531281a-Abstract.html), [iDLG](https://arxiv.org/abs/2001.02610), [GS](https://proceedings.neurips.cc/paper/2020/hash/c4ede56bbd98819ae6112b20ac6bf145-Abstract.html), [CPL](https://arxiv.org/abs/2004.10397), [GradInversion](https://openaccess.thecvf.com/content/CVPR2021/papers/Yin_See_Through_Gradients_Image_Batch_Recovery_via_GradInversion_CVPR_2021_paper.pdf), [GAN Attack](https://arxiv.org/abs/1702.07464) |\n| Attack        | Label Leakage          | [Norm Attack](https://arxiv.org/abs/2102.08504)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |\n| Attack        | Poisoning              | [History Attack](https://arxiv.org/abs/2203.08669), [Label Flip](https://arxiv.org/abs/2203.08669), [MAPF](https://arxiv.org/abs/2203.08669), [SVM Poisoning](https://arxiv.org/abs/1206.6389)                                                                                                                                                                                                                                                                                                                                                                   |\n| Attack        | Backdoor               | [DBA](https://openreview.net/forum?id=rkgyS0VFvr), [Model Replacement](https://proceedings.mlr.press/v108/bagdasaryan20a.html)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |\n| Attack        | Free-Rider             | [Delta-Weight](https://arxiv.org/pdf/1911.12560.pdf)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |\n| Attack        | Evasion                | [Gradient-Descent Attack](https://arxiv.org/abs/1708.06131), [FGSM](https://arxiv.org/abs/1412.6572), [DIVA](https://arxiv.org/abs/2204.10933)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |\n| Attack        | Membership Inference   | [Shadow Attack](https://arxiv.org/abs/1610.05820)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |\n| Defense       | Homomorphic Encryption | [Paillier](https://link.springer.com/chapter/10.1007/3-540-48910-X_16)                                                                                                                                                                                                                                                                                                                                                                                                                                             |\n| Defense       | Differential Privacy   | [DPSGD](https://arxiv.org/abs/1607.00133), [AdaDPS](https://arxiv.org/pdf/2202.05963.pdf), [DPlis](https://arxiv.org/pdf/2103.01496.pdf)                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |\n| Defense       | Anonymization          | [Mondrian](https://ieeexplore.ieee.org/document/1617393)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |\n| Defense       | Robust Training   | [PixelDP](https://arxiv.org/abs/1802.03471v4), [Cost-Aware Robust Tree Ensemble](https://arxiv.org/abs/1912.01149) |\n| Defense       | Debugging              | [Model Assertions](https://cs.stanford.edu/~matei/papers/2019/debugml_model_assertions.pdf), [Rain](https://arxiv.org/abs/2004.05722), [Neuron Coverage](https://dl.acm.org/doi/abs/10.1145/3132747.3132785) |\n| Defense       | Others                 | [Soteria](https://openaccess.thecvf.com/content/CVPR2021/papers/Sun_Soteria_Provable_Defense_Against_Privacy_Leakage_in_Federated_Learning_From_CVPR_2021_paper.pdf), [FoolsGold](https://arxiv.org/abs/1808.04866), [MID](https://arxiv.org/abs/2009.05241), [Sparse Gradient](https://aclanthology.org/D17-1045/)                                                                                                                                                                                                                                              |\n\n-----------------------------------------------------------------------\n\n# Citation\n\nIf you use AIJack for your research, please cite our arXiv paper.\n\n```\n@article{takahashi2023aijack,\n  title={AIJack: Security and Privacy Risk Simulator for Machine Learning},\n  author={Takahashi, Hideaki},\n  journal={arXiv preprint arXiv:2312.17667},\n  year={2023}\n}\n```\n\n# Related Publications\n\nBelow you can find a list of papers and books that either use or extend AIJack.\n\n- Huang, Shiyuan, et al. \"Video in 10 Bits: Few-Bit VideoQA for Efficiency and Privacy.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n- Song, Junzhe, and Dmitry Namiot. \"A Survey of the Implementations of Model Inversion Attacks.\" International Conference on Distributed Computer and Communication Networks. Cham: Springer Nature Switzerland, 2022.\n- Kapoor, Amita, and Sharmistha Chatterjee. Platform and Model Design for Responsible AI: Design and build resilient, private, fair, and transparent machine learning models. Packt Publishing Ltd, 2023.\n- Mi, Yuxi, et al. \"Flexible Differentially Private Vertical Federated Learning with Adaptive Feature Embeddings.\" arXiv preprint arXiv:2308.02362 (2023).\n- Mohammadi, Mohammadreza, et al. \"Privacy-preserving Federated Learning System for Fatigue Detection.\" 2023 IEEE International Conference on Cyber Security and Resilience (CSR). IEEE, 2023.\n- Huang, Shiyuan. A General Framework for Model Adaptation to Meet Practical Constraints in Computer Vision. Diss. Columbia University, 2024.\n- Liu, Can, Jin Wang, and Dongyang Yu. \"RAF-GI: Towards Robust, Accurate and Fast-Convergent Gradient Inversion Attack in Federated Learning.\" arXiv preprint arXiv:2403.08383 (2024).\n\n# Contribution\n\nAIJack welcomes contributions of any kind. If you'd like to address a bug or propose a new feature, please refer to [our guide](docs/source/contribution.rst).\n\n# Contact\n\nwelcome2aijack[@]gmail.com\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoukyosyumei%2Faijack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkoukyosyumei%2Faijack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkoukyosyumei%2Faijack/lists"}