{"id":13717494,"url":"https://github.com/AppThreat/vulnerability-db","last_synced_at":"2025-05-07T07:31:37.622Z","repository":{"id":45427490,"uuid":"235073284","full_name":"AppThreat/vulnerability-db","owner":"AppThreat","description":"Vulnerability database and package search for sources such as Linux, OSV, NVD, GitHub and npm. Powered by sqlite, CVE 5.1, purl, and vers.","archived":false,"fork":false,"pushed_at":"2025-04-15T08:11:56.000Z","size":6468,"stargazers_count":114,"open_issues_count":28,"forks_count":22,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-28T08:07:50.671Z","etag":null,"topics":["advisories","cli","cve","database","nvd","purl","sca","vers","vulnerability-database","vulnerability-detection"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AppThreat.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"prabhu"}},"created_at":"2020-01-20T10:16:06.000Z","updated_at":"2025-04-08T07:31:03.000Z","dependencies_parsed_at":"2024-01-17T09:21:24.474Z","dependency_job_id":"4aa77602-d22d-4b65-bb50-14321c3c1ccf","html_url":"https://github.com/AppThreat/vulnerability-db","commit_stats":{"total_commits":124,"total_committers":6,"mean_commits":"20.666666666666668","dds":"0.42741935483870963","last_synced_commit":"48afbc5a9e722e6f2ad989338d1c33dec285cde7"},"previous_names":["appthreat/vulndb"],"tags_count":156,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AppThreat%2Fvulnerability-db","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AppThreat%2Fvulnerability-db/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AppThreat%2Fvulnerability-db/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AppThreat%2Fvulnerability-db/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AppThreat","download_url":"https://codeload.github.com/AppThreat/vulnerability-db/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252833559,"owners_count":21811209,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["advisories","cli","cve","database","nvd","purl","sca","vers","vulnerability-database","vulnerability-detection"],"created_at":"2024-08-03T00:01:23.093Z","updated_at":"2025-05-07T07:31:37.613Z","avatar_url":"https://github.com/AppThreat.png","language":"Python","funding_links":["https://github.com/sponsors/prabhu"],"categories":["Dependency intelligence","Python"],"sub_categories":["Vulnerability information exchange"],"readme":"# Introduction\n\nThis repo is a vulnerability database and package search for sources such as AppThreat vuln-list, OSV, NVD, and GitHub. Vulnerability data are downloaded from the sources and stored in a sqlite based storage with indexes to allow offline access and efficient searches.\n\n## Why vulnerability db?\n\nA good vulnerability database must have the following properties:\n\n- Accuracy\n- Easy to [download](#download-pre-built-database-recommended), [integrate](./INTEGRATION.md), and use\n- Performance\n\nMultiple upstream sources are used by vdb to improve accuracy and reduce false negatives. SQLite database containing data in CVE 5.1 schema format is precompiled and distributed as files via ghcr to simplify download. With automatic purl prefix generation even for git repos, searches on the database can be performed with purl, cpe, or even http git url string. Every row in the database uses an open specification such as CVE 5.0 or Package URL (purl and vers) thus preventing the possibility of vendor lock-in.\n\n## Vulnerability Data sources\n\n- Linux [vuln-list](https://github.com/appthreat/vuln-list) (Forked from AquaSecurity)\n- OSV (1)\n- NVD\n- GitHub\n\n1 - We exclude Linux and oss-fuzz feeds by default. Set the environment variable `OSV_INCLUDE_FUZZ=true` to include them.\n2 - Malware feeds are included by default, thus increasing the db size slightly. Set the environment variable `OSV_EXCLUDE_MALWARE=true` to exclude them.\n\n## Linux distros\n\n- AlmaLinux\n- Debian\n- Alpine\n- Amazon Linux\n- Arch Linux\n- RHEL/CentOS\n- Rocky Linux\n- Ubuntu\n- OpenSUSE\n- Photon\n- Chainguard\n- Wolfi OS\n\n## Installation\n\n```shell\npip install appthreat-vulnerability-db\u003e=6.2.0\n```\n\nTo install vdb with optional dependencies such as `oras` use the `[oras]` or `[all]` dependency group.\n\n```shell\npip install appthreat-vulnerability-db[all]\n```\n\n**NOTE:** VDB v6 is a major rewrite to use SQLite database. Current users of depscan v5 must continue using version 5.8.x\n\n```shell\npip install appthreat-vulnerability-db==5.8.0\n```\n\n## Usage\n\nThis package is ideal as a library for managing vulnerabilities. This is used by [owasp-dep-scan](http://github.com/owasp-dep-scan/dep-scan), a free open-source dependency audit tool. However, there is a limited cli capability available with few features to test this tool directly.\n\n### Option 1: Download pre-built database (Recommended)\n\nTo download a pre-built SQLite database ([refreshed](https://github.com/AppThreat/vdb/actions) every 6 hours) containing all application vulnerabilities (~ 700MB). This step is recommended for all users.\n\n```shell\n# pip install appthreat-vulnerability-db[all]\nvdb --download-image\n```\n\nYou can execute this command daily or when a fresh database is required.\n\nTo perform containers and OS scans, download the full image (~ 7.5GB) which includes all application and OS vulnerabilities.\n\n```\nvdb --download-full-image\n```\n\nUse any sqlite browser or cli tools to load and query the two databases.\n\n**data.index.vdb6** - index db with purl prefix and vers\n\n\u003cimg src=\"./docs/index-vdb6.png\" alt=\"index\" width=\"400\"\u003e\n\n**data.vdb6** - Contains source data in CVE 5.1 format stored as a jsonb blob.\n\n\u003cimg src=\"./docs/vdb6.png\" alt=\"database\" width=\"400\"\u003e\n\n### Option 2: Download pre-built database (ORAS)\n\nUsing [ORAS cli](https://oras.land/) might be slightly faster.\n\n```\nexport VDB_HOME=$HOME/vdb\noras pull ghcr.io/appthreat/vdbxz:v6.4.x -o $VDB_HOME\ntar -xvf *.tar.xz\nrm *.tar.xz\n```\n\n### Option 3: Use HuggingFace cli\n\nDownload one of the databases.\n\n```shell\npip install -U \"huggingface_hub[cli]\"\n```\n\napp only database\n\n```shell\nexport VDB_HOME=$(pwd)/app\nhuggingface-cli download AppThreat/vdb --include \"app/*.vdb6\" --repo-type dataset --local-dir .\n```\n\napp only 10 year database\n\n```shell\nexport VDB_HOME=$(pwd)/app-10y\nhuggingface-cli download AppThreat/vdb --include \"app-10y/*.vdb6\" --repo-type dataset --local-dir .\n```\n\napp and os database\n\n```shell\nexport VDB_HOME=$(pwd)/app-os\nhuggingface-cli download AppThreat/vdb --include \"app-os/*.vdb6\" --repo-type dataset --local-dir .\n```\n\napp and os 10 year database\n\n```shell\nexport VDB_HOME=$(pwd)/app-os-10y\nhuggingface-cli download AppThreat/vdb --include \"app-os-10y/*.vdb6\" --repo-type dataset --local-dir .\n```\n\n#### Citation\n\nUse the below citation in your research.\n\n```text\n@misc{vdb,\n  author = {Team AppThreat},\n  month = Feb,\n  title = {{AppThreat vulnerability-db}},\n  howpublished = {{https://huggingface.co/datasets/AppThreat/vdb}},\n  year = {2025}\n}\n```\n\n### Option 4: Manually create the vulnerability database (ADVANCED users)\n\nCache application vulnerabilities\n\n```shell\nvdb --cache\n```\n\nThe typical size of this database is over 700 MB.\n\nCache from just [OSV](https://osv.dev)\n\n```shell\nvdb --cache --only-osv\n```\n\nIt is possible to customize the cache behavior by increasing the historic data period to cache by setting the following environment variables.\n\n- NVD_START_YEAR - Default: 2018. Supports up to 2002\n- GITHUB_PAGE_COUNT - Default: 2. Supports up to 20\n\nCache application and OS vulnerabilities\n\n```shell\nvdb --cache-os\n```\n\nNote the size of the database with OS vulnerabilities is over 7.5 GB. It is possible to ignore/exlude specific OS distros using environment variables.\n\nExample to ignore almalinux and ubuntu data from getting included, set the below environment variables:\n\n```shell\nexport VDB_IGNORE_ALMALINUX=true\nexport VDB_IGNORE_UBUNTU=true\n```\n\nRefer to the variable `LINUX_DISTRO_VULN_LIST_PATHS` in [config.py](./vdb/lib/config.py) for the full list of distro strings supported.\n\n## CLI Usage\n\n```shell\nusage: vdb [-h] [--clean] [--cache] [--cache-os] [--only-osv] [--only-aqua] [--only-ghsa] [--search SEARCH] [--list-malware] [--bom BOM_FILE]\n\nAppThreat's vulnerability database and package search library with a built-in sqlite based storage.\n\noptions:\n  -h, --help       show this help message and exit\n  --clean          Clear the vulnerability database cache from platform specific user_data_dir.\n  --cache          Cache vulnerability information in platform specific user_data_dir.\n  --cache-os       Cache OS vulnerability information in platform specific user_data_dir.\n  --only-osv       Use only OSV as the source. Use with --cache.\n  --only-aqua      Use only Aqua vuln-list as the source. Use with --cache.\n  --only-ghsa      Use only recent ghsa as the source. Use with --cache.\n  --search SEARCH  Search for the package or CVE ID in the database. Use purl, cpe, or git http url.\n  --list-malware   List latest malwares with CVE ID beginning with MAL-.\n  --bom BOM_FILE   Search for packages in the CycloneDX BOM file.\n  --download-image      Downloaded pre-created vdb image to platform specific user_data_dir. Application vulnerabilities only.\n  --download-full-image\n                        Downloaded pre-created vdb image to platform specific user_data_dir. All vulnerabilities including OS.\n  --print-vdb-metadata  Display metadata about the current vdb in user_data_dir.\n```\n\n### CLI search\n\nIt is possible to perform a range of searches using the cli.\n\n```shell\nvdb --search pkg:pypi/xml2dict@0.2.2\n\n# Search based on a purl prefix\nvdb --search pkg:pypi/xml2dict\n\n# Full url and short form for swift\nvdb --search \"pkg:swift/github.com/vapor/vapor@4.39.0\"\n\nvdb --search \"pkg:swift/vapor/vapor@4.89.0\"\n\n# Search by cpe\nvdb --search \"cpe:2.3:a:npm:gitblame:*:*:*:*:*:*:*:*\"\n\n# Search by colon separated values\nvdb --search \"npm:gitblame:0.0.1\"\n\n# Search by vulnerability id (CVE, GHSA, ALSA, DSA, etc.)\nvdb --search CVE-2024-25169\n\n# Search with wildcard for CVE\nvdb --search CVE-2024-%\n\n# Search by git url\nvdb --search \"https://github.com/electron/electron\"\n\n# Search by CycloneDX SBOM\nvdb --bom bom.json\n```\n\n### List recent malware\n\nTo list malware entries with the `MAL-` prefix, use the following command.\n\n```shell\nvdb --list-malware\n```\n\n## Library usage\n\nRefer to [cli.py](./vdb/cli.py) for usage. Below is a sample code snippet.\n\n```python\nfrom vdb.lib import config, db6 as db_lib, search\nfrom vdb.lib.cve_model import CVE\n\n# Perform search\nresults = search.search_by_any(\"purl or cpe or url\", with_data=True)\n\n# Download prebuilt database\nfrom vdb.lib.orasclient import download_image\n\ndownload_image(config.VDB_DATABASE_URL, config.DATA_DIR)\n\n# Check if database needs updating\nif db_lib.needs_update():\n    print(\"Database needs to be updated\")\n    download_image(...)\n```\n\n## Model Context Protocol (MCP) server\n\nRefer to the [readme](./packages/mcp-server-vdb/README.md)\n\n## Read .vdb6 files in other languages\n\n.vdb6 files are standard SQLite database files. Use any modern sqlite library to read and query them. There is a mini [deno example](./contrib/deno-vdb/README.md) in this repo for demonstration.\n\n## Local development\n\nSetup uv by following the official [documentation](https://docs.astral.sh/uv/).\n\n```shell\nuv sync --all-extras --dev\nuv run depscan --help\nuv run pytest\n```\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAppThreat%2Fvulnerability-db","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAppThreat%2Fvulnerability-db","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAppThreat%2Fvulnerability-db/lists"}