{"id":47707274,"url":"https://github.com/promptfoo/modelaudit","last_synced_at":"2026-06-06T01:01:43.913Z","repository":{"id":340494726,"uuid":"944201441","full_name":"promptfoo/modelaudit","owner":"promptfoo","description":"Security scanner for AI/ML model files. Detects malicious code, backdoors, and vulnerabilities before deployment","archived":false,"fork":false,"pushed_at":"2026-06-05T19:21:26.000Z","size":19175,"stargazers_count":55,"open_issues_count":24,"forks_count":11,"subscribers_count":5,"default_branch":"main","last_synced_at":"2026-06-05T19:22:49.594Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.promptfoo.dev/docs/model-audit/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/promptfoo.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":"SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":"MAINTAINERS.md","copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-03-07T00:27:41.000Z","updated_at":"2026-06-05T18:39:38.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/promptfoo/modelaudit","commit_stats":null,"previous_names":["promptfoo/modelaudit"],"tags_count":34,"template":false,"template_full_name":null,"purl":"pkg:github/promptfoo/modelaudit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptfoo%2Fmodelaudit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptfoo%2Fmodelaudit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptfoo%2Fmodelaudit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptfoo%2Fmodelaudit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/promptfoo","download_url":"https://codeload.github.com/promptfoo/modelaudit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptfoo%2Fmodelaudit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33965591,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-05T02:00:06.157Z","response_time":120,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-02T18:01:13.197Z","updated_at":"2026-06-06T01:01:43.868Z","avatar_url":"https://github.com/promptfoo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ModelAudit\n\n**Secure your AI models before deployment.** Static scanner that detects malicious code, potential backdoor indicators, and security vulnerabilities in ML model files — without ever loading or executing them.\n\n[![PyPI version](https://badge.fury.io/py/modelaudit.svg)](https://pypi.org/project/modelaudit/)\n[![Python versions](https://img.shields.io/pypi/pyversions/modelaudit.svg)](https://pypi.org/project/modelaudit/)\n[![Code Style: ruff](https://img.shields.io/badge/code%20style-ruff-005cd7.svg)](https://github.com/astral-sh/ruff)\n[![License](https://img.shields.io/github/license/promptfoo/modelaudit)](https://github.com/promptfoo/modelaudit/blob/main/LICENSE)\n[![Security policy](https://img.shields.io/badge/security-policy-brightgreen.svg)](https://github.com/promptfoo/modelaudit/security/policy)\n\n\u003cimg width=\"989\" alt=\"ModelAudit scan results\" src=\"https://www.promptfoo.dev/img/docs/modelaudit/modelaudit-result.png\" /\u003e\n\n**[Full Documentation](https://www.promptfoo.dev/docs/model-audit/)** | **[Usage Examples](https://www.promptfoo.dev/docs/model-audit/usage/)** | **[Supported Formats](https://www.promptfoo.dev/docs/model-audit/scanners/)**\n\n## Why ModelAudit\n\nModels download from untrusted registries, pass through CI, and end up running in production. Traditional SAST tools do not look at pickle opcodes, HDF5 group layouts, ONNX proto graphs, or TensorFlow SavedModel signatures — ModelAudit does:\n\n- **Scan statically.** No model is ever loaded, unpickled, or executed.\n- **Cover the formats you actually ship.** 40+ scanners spanning pickle, PyTorch, SafeTensors, ONNX, TensorFlow, Keras, GGUF, archives, and configs.\n- **Fit into CI.** Machine-readable output (JSON, SARIF), strict mode, exit codes, and [selectable scanners](https://github.com/promptfoo/modelaudit/blob/main/docs/user/scanner-selection.md).\n- **Surface coverage limits.** Recognized scanners report bounded-analysis gaps such as truncated reads or exhausted budgets instead of presenting them as fully covered results.\n\nComparable tools: [`picklescan`](https://github.com/mmaitre314/picklescan) (pickle only, Python-based), [`fickling`](https://github.com/trailofbits/fickling) (pickle only, AST-based), [`modelscan`](https://github.com/protectai/modelscan) (pickle + TensorFlow + Keras subset). ModelAudit is broader in coverage and ships a native Rust pickle engine via its companion package [`modelaudit-picklescan`](https://pypi.org/project/modelaudit-picklescan/).\n\n## Quick Start\n\n**Requires Python 3.10-3.13**\n\n```bash\npip install \"modelaudit[all]\"\n\n# Scan a file or directory\nmodelaudit model.pkl\nmodelaudit ./models/\n\n# Export results for CI/CD\nmodelaudit model.pkl --format json --output results.json\n```\n\n```\n$ modelaudit suspicious_model.pkl\n\nFiles scanned: 1 | Issues found: 2 critical, 1 warning\n\n1. suspicious_model.pkl (pos 28): [CRITICAL] Malicious code execution attempt\n   Why: Contains os.system() call that could run arbitrary commands\n\n2. suspicious_model.pkl (pos 52): [WARNING] Dangerous pickle deserialization\n   Why: Could execute code when the model loads\n```\n\n## What It Detects\n\n- **Code execution attacks** in Pickle, PyTorch, NumPy, and Joblib files\n- **Potential backdoor indicators** — suspicious weight patterns, anomalous tensors, or hidden-code signals\n- **Embedded secrets** — API keys, tokens, and credentials in model weights or metadata\n- **Network indicators** — URLs, IPs, and socket usage that could enable data exfiltration\n- **Archive exploits** — path traversal, symlink attacks in ZIP/TAR/7z files\n- **Unsafe ML operations** — Lambda layers, custom ops, TorchScript/JIT, template injection\n- **Supply chain risks** — tampering, license violations, suspicious configurations\n\n## Supported Formats\n\nModelAudit includes 44 registered scanners covering model, archive, and configuration formats:\n\n| Format                  | Extensions                                                                | Risk   |\n| ----------------------- | ------------------------------------------------------------------------- | ------ |\n| **Pickle**              | `.pkl`, `.pickle`, `.dill`                                                | HIGH   |\n| **PyTorch**             | `.pt`, `.pth`, `.ckpt`, `.bin`                                            | HIGH   |\n| **Joblib**              | `.joblib`                                                                 | HIGH   |\n| **NumPy**               | `.npy`, `.npz`                                                            | HIGH   |\n| **R Serialized**        | `.rds`, `.rda`, `.rdata`                                                  | HIGH   |\n| **TensorFlow**          | `.pb`, `.meta`, SavedModel dirs                                           | MEDIUM |\n| **Keras**               | `.h5`, `.hdf5`, `.keras`                                                  | MEDIUM |\n| **ONNX**                | `.onnx`                                                                   | MEDIUM |\n| **CoreML**              | `.mlmodel`                                                                | LOW    |\n| **MXNet**               | `*-symbol.json`, `*-NNNN.params`                                          | LOW    |\n| **NeMo**                | `.nemo`                                                                   | MEDIUM |\n| **CNTK**                | `.dnn`, `.cmf`                                                            | MEDIUM |\n| **RKNN**                | `.rknn`                                                                   | MEDIUM |\n| **Torch7**              | `.t7`, `.th`, `.net`                                                      | HIGH   |\n| **CatBoost**            | `.cbm`                                                                    | MEDIUM |\n| **XGBoost**             | `.bst`, `.model`, `.json`, `.ubj`                                         | MEDIUM |\n| **LightGBM**            | `.lgb`, `.lightgbm`, `.model`                                             | MEDIUM |\n| **Llamafile**           | `.llamafile`, extensionless, `.exe`                                       | MEDIUM |\n| **TorchServe**          | `.mar`                                                                    | HIGH   |\n| **SafeTensors**         | `.safetensors`                                                            | LOW    |\n| **GGUF/GGML**           | `.gguf`, `.ggml`, `.ggmf`, `.ggjt`, `.ggla`, `.ggsa`                      | LOW    |\n| **JAX/Flax**            | `.msgpack`, `.flax`, `.orbax`, `.jax`, `.checkpoint`, `.orbax-checkpoint` | LOW    |\n| **TFLite**              | `.tflite`                                                                 | LOW    |\n| **ExecuTorch**          | `.ptl`, `.pte`                                                            | LOW    |\n| **TensorRT**            | `.engine`, `.plan`, `.trt`                                                | LOW    |\n| **PaddlePaddle**        | `.pdmodel`, `.pdiparams`                                                  | LOW    |\n| **OpenVINO**            | `.xml`                                                                    | LOW    |\n| **Skops**               | `.skops`                                                                  | HIGH   |\n| **PMML**                | `.pmml`                                                                   | LOW    |\n| **Compressed Wrappers** | `.gz`, `.bz2`, `.xz`, `.lz4`, `.zlib`                                     | MEDIUM |\n\nPlus scanners for ZIP, TAR, 7-Zip, OCI layers, Jinja2 templates, JSON/YAML metadata, manifests, model cards, text files, and RAR recognition. RAR archives are reported as unsupported/fail-closed instead of being skipped.\n\n[View complete format documentation](https://www.promptfoo.dev/docs/model-audit/scanners/)\n\n## Remote Sources\n\nScan models directly from remote registries and cloud storage:\n\n```bash\n# Hugging Face\nmodelaudit https://huggingface.co/gpt2\nmodelaudit hf://microsoft/DialoGPT-medium\n\n# Cloud storage\nmodelaudit s3://bucket/model.pt\nmodelaudit gs://bucket/models/\n\n# MLflow registry\nmodelaudit models:/MyModel/Production\n\n# JFrog Artifactory (files and folders)\n# Auth: export JFROG_API_TOKEN=...\nmodelaudit https://company.jfrog.io/artifactory/repo/model.pt\nmodelaudit https://company.jfrog.io/artifactory/repo/models/\n\n# DVC-tracked models\nmodelaudit model.dvc\n```\n\n### Authentication Environment Variables\n\n- `HF_TOKEN` for private Hugging Face repositories\n- `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` (and optional `AWS_SESSION_TOKEN`) for S3\n- `GOOGLE_APPLICATION_CREDENTIALS` for GCS\n- `MLFLOW_TRACKING_URI` for MLflow registry access\n- `JFROG_API_TOKEN` or `JFROG_ACCESS_TOKEN` for JFrog Artifactory\n- Store credentials in environment variables or a secrets manager, and never commit tokens/keys.\n\n## Installation\n\n```bash\n# Broad scanner coverage (recommended; excludes the TensorFlow runtime and platform-specific TensorRT)\npip install \"modelaudit[all]\"\n\n# Core only (static scanners, pickle, NumPy, archives, manifests, metadata)\npip install modelaudit\n\n# Specific frameworks (TensorFlow installs on Python 3.11-3.12; ONNX installs on Python 3.10-3.12)\npip install \"modelaudit[tensorflow,pytorch,h5,onnx,safetensors]\"\n\n# CI/CD environments\npip install \"modelaudit[all-ci]\"\n\n# On Python 3.11-3.12, add TensorFlow only when you need runtime-dependent checkpoint or weight analysis\npip install \"modelaudit[all,tensorflow]\"\n\n# Docker\ndocker run --rm -v \"$(pwd)\":/app ghcr.io/promptfoo/modelaudit:latest model.pkl\n```\n\nThe ONNX extra, including the ONNX portion of `modelaudit[all]`, is packaged for Python 3.10-3.12.\n\n## CLI Options\n\nPrimary commands:\n\n```bash\nmodelaudit [PATHS...]                           # Default scan command\nmodelaudit scan [OPTIONS] PATHS...              # Explicit scan command\nmodelaudit scan --list-scanners                 # List scanner IDs for targeted scans\nmodelaudit metadata [OPTIONS] PATH              # Extract model metadata safely (no deserialization by default)\nmodelaudit doctor [--show-failed]               # Diagnose scanner/dependency availability\nmodelaudit debug [--json] [--verbose]           # Environment and configuration diagnostics\nmodelaudit cache [stats|clear|cleanup] [OPTIONS]\n```\n\nCommon scan options:\n\n```text\n--format {text,json,sarif}   Output format (default: auto-detected)\n--output FILE                Write results to file\n--strict                     Fail on warnings, scan all file types, strict license validation\n--sbom FILE                  Generate CycloneDX SBOM\n--stream                     Process files one-by-one; remote downloads are deleted after scanning\n--max-size SIZE              Size limit (e.g., 10GB)\n--timeout SECONDS            Override scan timeout\n--dry-run                    Preview what would be scanned\n--verbose / --quiet          Control output detail\n--blacklist PATTERN          Additional patterns to flag\n--no-cache                   Disable result caching\n--cache-dir DIR              Set cache directory for downloads and scan results\n--progress                   Force progress display\n--scanners LIST              Only run selected scanners (IDs/classes; comma-separated or repeated)\n--exclude-scanner NAME       Exclude a scanner from the active set (comma-separated or repeated)\n--list-scanners              List scanner IDs, class names, extensions, and dependencies\n```\n\nTargeted scanner selection:\n\n```bash\n# Discover scanner IDs and class names\nmodelaudit scan --list-scanners\nmodelaudit scan --list-scanners --format json\n\n# Run only selected scanners\nmodelaudit scan ./models --scanners pickle,tf_savedmodel\nmodelaudit scan ./model.pkl --scanners PickleScanner\n\n# Run the default scanner set except a noisy or slow scanner\nmodelaudit scan ./models --exclude-scanner weight_distribution\n\n# For container formats, include both the container scanner and nested scanner\nmodelaudit scan ./archive.zip --scanners zip,pickle\n```\n\n`--scanners` starts from an explicit allowlist. `--exclude-scanner` subtracts scanners from either that allowlist or the default scanner set. Scanner selection is reflected in JSON output under `scanner_selection`.\n\nFor remote folders, ModelAudit narrows downloads by selected scanner extensions when safe, and keeps filtering conservative for container or header-routed scanners to avoid dropping extension-spoofed artifacts before scanning.\n\n## Metadata Extraction\n\n```bash\n# Human-readable summary (safe default: no model deserialization)\nmodelaudit metadata model.safetensors\n\n# Machine-readable output\nmodelaudit metadata ./models --format json --output metadata.json\n\n# Focus only on security-relevant metadata fields\nmodelaudit metadata model.onnx --security-only\n```\n\n`--trust-loaders` enables scanner metadata loaders that may deserialize model content. Only use this on trusted artifacts in isolated environments.\n\n## Exit Codes\n\n- `0`: No security issues detected\n- `1`: Security issues detected\n- `2`: Scan errors\n\n## Telemetry and Privacy\n\nModelAudit includes telemetry for product reliability and usage analytics.\n\n- Collected metadata can include command usage, scan timing, scanner/file-type usage, issue severity/type aggregates, sanitized model names/references, and coarse metadata like file extension/domain.\n- URL telemetry strips userinfo, query strings, and fragments from model references. Avoid putting credentials in model names, file names, or artifact paths when telemetry is enabled.\n- Model files are scanned locally and ModelAudit does not upload model binary contents as telemetry events.\n- Telemetry is disabled automatically when `CI=true` is set or `IS_TESTING=true` is set, and in editable development installs by default. Events that are sent from other CI providers (TeamCity, CodeBuild, Bitbucket Pipelines, Jenkins) are tagged with `isRunningInCi=true` so they can be filtered downstream.\n- The anonymous user identifier is stored in `~/.promptfoo/promptfoo.yaml` for cross-tool correlation with [Promptfoo](https://www.promptfoo.dev/). Existing IDs from `~/.modelaudit/user_config.json` are migrated on first run after upgrade.\n\nOpt out explicitly with either environment variable:\n\n```bash\nexport PROMPTFOO_DISABLE_TELEMETRY=1\n# or\nexport NO_ANALYTICS=1\n```\n\nTo opt in during editable/development installs:\n\n```bash\nexport MODELAUDIT_TELEMETRY_DEV=1\n```\n\n## Output Examples\n\n```bash\n# JSON for CI/CD pipelines\nmodelaudit model.pkl --format json --output results.json\n\n# SARIF for code scanning platforms\nmodelaudit model.pkl --format sarif --output results.sarif\n```\n\n## Troubleshooting\n\n- Run `modelaudit doctor --show-failed` to list unavailable scanners and missing optional deps.\n- Run `modelaudit debug --json` to collect environment/config diagnostics for bug reports.\n- Use `modelaudit cache cleanup --max-age 30` to remove stale cache entries safely.\n- If `pip` installs an older release, verify Python is supported (`python --version`; ModelAudit supports Python 3.10-3.13).\n- For additional troubleshooting and cloud auth guidance, see:\n  - https://www.promptfoo.dev/docs/model-audit/\n  - https://www.promptfoo.dev/docs/model-audit/usage/\n\n## Documentation\n\n- **[Full docs](https://www.promptfoo.dev/docs/model-audit/)** — setup, configuration, and advanced usage\n- **[Usage examples](https://www.promptfoo.dev/docs/model-audit/usage/)** — CI/CD integration, remote scanning, SBOM generation\n- **[Supported formats](https://www.promptfoo.dev/docs/model-audit/scanners/)** — detailed scanner documentation\n- **[Support policy](https://github.com/promptfoo/modelaudit/blob/main/SUPPORT.md)** — supported Python/OS versions and maintenance policy\n- **[Security model and limitations](https://github.com/promptfoo/modelaudit/blob/main/docs/user/security-model.md)** — what ModelAudit does and does not guarantee\n- **[Compatibility matrix](https://github.com/promptfoo/modelaudit/blob/main/docs/user/compatibility-matrix.md)** — file formats vs optional dependencies\n- **[Scanner selection](https://github.com/promptfoo/modelaudit/blob/main/docs/user/scanner-selection.md)** — targeted scanner allowlists and exclusions\n- **[Metadata extraction guide](https://github.com/promptfoo/modelaudit/blob/main/docs/user/metadata-extraction.md)** — safe metadata workflows and `--trust-loaders` guidance\n- **[Offline/air-gapped guide](https://github.com/promptfoo/modelaudit/blob/main/docs/user/offline-air-gapped.md)** — secure operation without internet access\n- **Troubleshooting** — run `modelaudit doctor --show-failed` to check scanner availability\n\n## Related Packages\n\n- **[`modelaudit-picklescan`](https://pypi.org/project/modelaudit-picklescan/)** — the standalone Rust-backed pickle scanner used by ModelAudit's pickle, PyTorch, ExecuTorch, and PyTorch-ZIP scanners. Install it directly if you only need pickle analysis (as a library, not a CLI) and do not want the full scanner bundle.\n\n## Reporting Vulnerabilities\n\nDo not open public issues for suspected vulnerabilities. See [SECURITY.md](https://github.com/promptfoo/modelaudit/blob/main/SECURITY.md) for coordinated disclosure.\n\n## Contributing\n\nIssues, feature requests, and PRs are welcome. See [CONTRIBUTING.md](https://github.com/promptfoo/modelaudit/blob/main/CONTRIBUTING.md).\n\n## License\n\nMIT License — see [LICENSE](https://github.com/promptfoo/modelaudit/blob/main/LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpromptfoo%2Fmodelaudit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpromptfoo%2Fmodelaudit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpromptfoo%2Fmodelaudit/lists"}