{"id":13499487,"url":"https://github.com/protectai/modelscan","last_synced_at":"2026-02-18T23:03:16.087Z","repository":{"id":185944636,"uuid":"670705110","full_name":"protectai/modelscan","owner":"protectai","description":"Protection against Model Serialization Attacks","archived":false,"fork":false,"pushed_at":"2025-11-24T20:26:22.000Z","size":5644,"stargazers_count":646,"open_issues_count":45,"forks_count":133,"subscribers_count":13,"default_branch":"main","last_synced_at":"2026-02-18T17:29:16.573Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://modelscan.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/protectai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-07-25T16:37:05.000Z","updated_at":"2026-02-18T14:15:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"a93e8640-fa00-4314-90a4-b07c708718bc","html_url":"https://github.com/protectai/modelscan","commit_stats":{"total_commits":146,"total_committers":17,"mean_commits":8.588235294117647,"dds":0.595890410958904,"last_synced_commit":"1dcb5beca45702365895103d9793cda4d8e551fe"},"previous_names":["protectai/modelscan","protectai/model-scanner"],"tags_count":27,"template":false,"template_full_name":null,"purl":"pkg:github/protectai/modelscan","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fmodelscan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fmodelscan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fmodelscan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fmodelscan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/protectai","download_url":"https://codeload.github.com/protectai/modelscan/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/protectai%2Fmodelscan/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29597854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T22:25:43.180Z","status":"ssl_error","status_checked_at":"2026-02-18T22:25:42.766Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T22:00:33.588Z","updated_at":"2026-02-18T23:03:11.079Z","avatar_url":"https://github.com/protectai.png","language":"Python","funding_links":[],"categories":["🛡️ エージェントセキュリティ","🎯 Tool Categories","Open Source Security Tools","Tools","[↑](#table-of-contents)Tools \u003ca name=\"tools\"\u003e\u003c/a\u003e","Detection and Scanning Tools","GPT Security","5. MLSecOps, MLOps \u0026 Supply Chain Security","Model Security","Defense \u0026 Security Controls","Tools of Trade","AI应用"],"sub_categories":["その他の標準","🔐 ML Security \u0026 Governance","Security","Model Artifact Scanners","Open Source Tools","Standard","3.2 Tools \u0026 Frameworks","Model Scanning","Model \u0026 Artifact Scanning","Defensive / Scanning"],"readme":"![ModelScan Banner](https://github.com/protectai/modelscan/assets/18154355/eeec657b-0d8f-42a7-b693-f35f10101d2c)\n[![bandit](https://github.com/protectai/modelscan/actions/workflows/bandit.yml/badge.svg)](https://github.com/protectai/modelscan/actions/workflows/bandit.yml)\n[![build](https://github.com/protectai/modelscan/actions/workflows/build.yml/badge.svg)](https://github.com/protectai/modelscan/actions/workflows/build.yml)\n[![black](https://github.com/protectai/modelscan/actions/workflows/black.yml/badge.svg)](https://github.com/protectai/modelscan/actions/workflows/black.yml)\n[![mypy](https://github.com/protectai/modelscan/actions/workflows/mypy.yml/badge.svg)](https://github.com/protectai/modelscan/actions/workflows/mypy.yml)\n[![tests](https://github.com/protectai/modelscan/actions/workflows/test.yml/badge.svg)](https://github.com/protectai/modelscan/actions/workflows/test.yml)\n[![Supported Versions](https://img.shields.io/pypi/pyversions/modelscan.svg)](https://pypi.org/project/modelscan)\n[![pypi Version](https://img.shields.io/pypi/v/modelscan)](https://pypi.org/project/modelscan)\n[![License: Apache 2.0](https://img.shields.io/crates/l/apa)](https://opensource.org/license/apache-2-0/)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit)](https://github.com/pre-commit/pre-commit)\n\n# ModelScan: Protection Against Model Serialization Attacks\n\nMachine Learning (ML) models are shared publicly over the internet, within teams and across teams. The rise of Foundation Models have resulted in public ML models being increasingly consumed for further training/fine tuning. ML Models are increasingly used to make critical decisions and power mission-critical applications.\nDespite this, models are not yet scanned with the rigor of a PDF file in your inbox.\n\nThis needs to change, and proper tooling is the first step.\n\n![ModelScan Preview](/imgs/modelscan-unsafe-model.gif)\n\nModelScan is an open source project from [Protect AI](https://protectai.com/?utm_campaign=Homepage\u0026utm_source=ModelScan%20GitHub%20Page\u0026utm_medium=cta\u0026utm_content=Open%20Source) that scans models to determine if they contain\nunsafe code. It is the first model scanning tool to support multiple model formats.\nModelScan currently supports: H5, Pickle, and SavedModel formats. This protects you\nwhen using PyTorch, TensorFlow, Keras, Sklearn, XGBoost, with more on the way.\n\n## TL;DR\n\nIf you are ready to get started scanning your models, it is simple:\n\n```bash\npip install modelscan\n```\n\nWith it installed, scan a model:\n\n```bash\nmodelscan -p /path/to/model_file.pkl\n```\n\n## Why You Should Scan Models\n\nModels are often created from automated pipelines, others may come from a data scientist’s laptop. In either case the model needs to move from one machine to another before it is used. That process of saving a model to disk is called serialization.\n\nA **Model Serialization Attack** is where malicious code is added to the contents of a model during serialization(saving) before distribution — a modern version of the Trojan Horse.\n\nThe attack functions by exploiting the saving and loading process of models. When you load a model with `model = torch.load(PATH)`, PyTorch opens the contents of the file and begins to running the code within. The second you load the model the exploit has executed.\n\nA **Model Serialization Attack** can be used to execute:\n\n- Credential Theft(Cloud credentials for writing and reading data to other systems in your environment)\n- Data Theft(the request sent to the model)\n- Data Poisoning(the data sent after the model has performed its task)\n- Model Poisoning(altering the results of the model itself)\n\nThese attacks are incredibly simple to execute and you can view working examples in our 📓[notebooks](https://github.com/protectai/modelscan/tree/main/notebooks) folder.\n\n## Enforcing And Automating Model Security\n\nModelScan offers robust open-source scanning. If you need comprehensive AI security, consider [Guardian](https://protectai.com/guardian?utm_campaign=Guardian\u0026utm_source=ModelScan%20GitHub%20Page\u0026utm_medium=cta\u0026utm_content=Open%20Source). It is our enterprise-grade model scanning product.\n\n![Guardian Overview](/imgs/guardian_overview.png)\n\n### Guardian's Features:\n\n1. **Cutting-Edge Scanning**: Access our latest scanners, broader model support, and automatic model format detection.\n2. **Proactive Security**: Define and enforce security requirements for Hugging Face models before they enter your environment—no code changes required.\n3. **Enterprise-Wide Coverage**: Implement a cohesive security posture across your organization, seamlessly integrating with your CI/CD pipelines.\n4. **Comprehensive Audit Trail**: Gain full visibility into all scans and results, empowering you to identify and mitigate threats effectively.\n\n## Getting Started\n\n### How ModelScan Works\n\nIf loading a model with your machine learning framework automatically executes the attack,\nhow does ModelScan check the content without loading the malicious code?\n\nSimple, it reads the content of the file one byte at a time just like a string, looking for\ncode signatures that are unsafe. This makes it incredibly fast, scanning models in the time it\ntakes for your computer to process the total filesize from disk(seconds in most cases). It also secure.\n\nModelScan ranks the unsafe code as:\n\n- CRITICAL\n- HIGH\n- MEDIUM\n- LOW\n\n![ModelScan Flow Chart](/imgs/model_scan_flow_chart.png)\n\nIf an issue is detected, reach out to the author's of the model immediately to determine the cause.\n\nIn some cases, code may be embedded in the model to make things easier to reproduce as a data scientist, but\nit opens you up for attack. Use your discretion to determine if that is appropriate for your workloads.\n\n### What Models and Frameworks Are Supported?\n\nThis will be expanding continually, so look out for changes in our release notes.\n\nAt present, ModelScan supports any Pickle derived format and many others:\n\n| ML Library                                   | API                                                                                                        | Serialization Format                | modelscan support |\n|----------------------------------------------|------------------------------------------------------------------------------------------------------------|-------------------------------------|-------------------|\n| Pytorch                                      | [torch.save() and torch.load()](https://pytorch.org/tutorials/beginner/saving_loading_models.html )        | Pickle                              | Yes               |\n| Tensorflow                                   | [tf.saved_model.save()](https://www.tensorflow.org/guide/saved_model)                                      | Protocol Buffer                     | Yes               |\n| Keras                                        | [keras.models.save(save_format= 'h5')](https://www.tensorflow.org/guide/keras/serialization_and_saving)    | HD5 (Hierarchical Data Format)      | Yes               |\n|                                              | [keras.models.save(save_format= 'keras')](https://www.tensorflow.org/guide/keras/serialization_and_saving) | Keras V3 (Hierarchical Data Format) | Yes               |\n| Classic ML Libraries (Sklearn, XGBoost etc.) | pickle.dump(), dill.dump(), joblib.dump(), cloudpickle.dump()                                              | Pickle, Cloudpickle, Dill, Joblib   | Yes               |\n\n### Installation\n\nModelScan is installed on your systems as a Python package(Python 3.9 to 3.12 supported). As shown from above you can install\nit by running this in your terminal:\n\n```bash\npip install modelscan\n```\n\nTo include it in your project's dependencies so it is available for everyone, add it to your `requirements.txt`\nor `pyproject.toml` like this:\n\n```toml\nmodelscan = \"\u003e=0.1.1\"\n```\n\nScanners for Tensorflow or HD5 formatted models require installation with extras:\n\n```bash\npip install 'modelscan[ tensorflow, h5py ]'\n```\n\n### Using ModelScan via CLI\n\nModelScan supports the following arguments via the CLI:\n\n| Usage                                                                            | Argument         | Explanation                                             |\n|----------------------------------------------------------------------------------|------------------|---------------------------------------------------------|\n| ```modelscan -h```                                                              | -h or --help     | View usage help                                         |\n| ```modelscan -v```                                                              | -v or --version  | View version information                                |\n| ```modelscan -p /path/to/model_file```                                           | -p or --path     | Scan a locally stored model                             |\n| ```modelscan -p /path/to/model_file --settings-file ./modelscan-settings.toml``` | --settings-file  | Scan a locally stored model using custom configurations |\n| ```modelscan create-settings-file```                                             | -l or --location | Create a configurable settings file                     |\n| ```modelscan -r```                                             | -r or --reporting-format | Format of the output. Options are console,       json, or custom (to be defined in settings-file). Default is console                    |\n| ```modelscan -r reporting-format -o file-name```                                             | -o or --output-file | Optional file name for output report                  |\n| ```modelscan --show-skipped```                          | --show-skipped | Print a list of files that were skipped      during the scan   |\n\nRemember models are just like any other form of digital media, you should scan content from any untrusted source before use.\n\n#### CLI Exit Codes\n\nThe CLI exit status codes are:\n\n- `0`: Scan completed successfully, no vulnerabilities found\n- `1`: Scan completed successfully, vulnerabilities found\n- `2`: Scan failed, modelscan threw an error while scanning\n- `3`: No supported files were passed to the tool\n- `4`: Usage error, CLI was passed invalid or incomplete options\n\n### Understanding The Results\n\nOnce a scan has been completed you'll see output like this if an issue is found:\n\n![ModelScan Scan Output](https://github.com/protectai/modelscan/raw/main/imgs/cli_output.png)\n\nHere we have a model that has an unsafe operator for both `ReadFile` and `WriteFile` in the model.\nClearly we do not want our models reading and writing files arbitrarily. We would now reach out\nto the creator of this model to determine what they expected this to do. In this particular case\nit allows an attacker to read our AWS credentials and write them to another place.\n\nThat is a firm NO for usage.\n\n## Integrating ModelScan In Your ML Pipelines and CI/CD Pipelines\n\nAd-hoc scanning is a great first step, please drill it into yourself, peers, and friends to do\nthis whenever they pull down a new model to explore. It is not sufficient to improve security\nfor production MLOps processes.\n\nModel scanning needs to be performed more than once to accomplish the following:\n\n1. Scan all pre-trained models before loading it for further work to prevent a compromised\nmodel from impacting your model building or data science environments.\n2. Scan all models after training to detect a supply chain attack that compromises new models.\n3. Scan all models before deploying to an endpoint to ensure that the model has not been compromised after storage.\n\nThe red blocks below highlight this in a traditional ML Pipeline.\n![MLOps Pipeline with ModelScan](https://github.com/protectai/modelscan/raw/main/imgs/ml_ops_pipeline_model_scan.png)\n\nThe processes would be the same for fine-tuning or any modifications of LLMs, foundational models, or external model.\n\nEmbed scans into deployment processes in your CI/CD systems to secure usage\nas models are deployed as well if this is done outside your ML Pipelines.\n\n## Diving Deeper\n\nInside the 📓[**notebooks**](https://github.com/protectai/modelscan/tree/main/notebooks) folder you can explore a number of notebooks that showcase\nexactly how Model Serialization Attacks can be performed against various ML Frameworks like TensorFlow and PyTorch.\n\nTo dig more into the meat of how exactly these attacks work check out 🖹 [**Model Serialization Attack Explainer**](https://github.com/protectai/modelscan/blob/main/docs/model_serialization_attacks.md).\n\nIf you encounter any other approaches for evaluating models in a static context, please reach out, we'd love\nto learn more!\n\n## Licensing\n\nCopyright 2024 Protect AI\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n   \u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n## Acknowledgements\n\nWe were heavily inspired by [Matthieu Maitre](http://mmaitre314.github.io) who built [PickleScan](https://github.com/mmaitre314/picklescan).\nWe appreciate the work and have extended it significantly with ModelScan. ModelScan is OSS’ed in the similar spirit as PickleScan.\n\n## Contributing\n\nWe would love to have you contribute to our open source ModelScan project.\nIf you would like to contribute, please follow the details on [Contribution page](https://github.com/protectai/modelscan/blob/main/CONTRIBUTING.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprotectai%2Fmodelscan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprotectai%2Fmodelscan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprotectai%2Fmodelscan/lists"}