{"id":13514008,"url":"https://github.com/ankane/pdscan","last_synced_at":"2025-11-17T14:21:38.129Z","repository":{"id":34242024,"uuid":"172596857","full_name":"ankane/pdscan","owner":"ankane","description":"Scan your data stores for unencrypted personal data (PII)","archived":false,"fork":false,"pushed_at":"2025-03-27T22:01:33.000Z","size":269,"stargazers_count":259,"open_issues_count":6,"forks_count":35,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-04T00:44:35.029Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ankane.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-25T22:40:01.000Z","updated_at":"2025-03-27T22:01:36.000Z","dependencies_parsed_at":"2023-01-15T05:34:19.331Z","dependency_job_id":"f4695f51-ddcd-4940-a82b-ab48f50cebb4","html_url":"https://github.com/ankane/pdscan","commit_stats":{"total_commits":267,"total_committers":3,"mean_commits":89.0,"dds":0.1460674157303371,"last_synced_commit":"4977b6f281847aa9d09a355da8f4bcacbd3397f0"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankane%2Fpdscan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankane%2Fpdscan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankane%2Fpdscan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankane%2Fpdscan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ankane","download_url":"https://codeload.github.com/ankane/pdscan/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248631688,"owners_count":21136556,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T05:00:42.841Z","updated_at":"2025-11-17T14:21:38.123Z","avatar_url":"https://github.com/ankane.png","language":"Go","funding_links":[],"categories":["Go","others"],"sub_categories":[],"readme":"# pdscan\n\nScan your data stores for unencrypted personal data (PII)\n\n- Last names (US)\n- Email addresses\n- IP addresses (IPv4)\n- Street addresses (US)\n- Phone numbers\n- Credit card numbers\n- Social Security numbers (US)\n- Dates of birth\n- Location data\n- OAuth tokens\n- MAC addresses\n\nUses data sampling and naming, and works with compressed files\n\n:boom: Zero runtime dependencies and minimal database load\n\n[![Build Status](https://github.com/ankane/pdscan/actions/workflows/build.yml/badge.svg)](https://github.com/ankane/pdscan/actions)\n\n## Installation\n\nDownload the latest version:\n\n- Linux - [x86_64](https://github.com/ankane/pdscan/releases/download/v0.1.9/pdscan-0.1.9-x86_64-linux.zip) or [arm64](https://github.com/ankane/pdscan/releases/download/v0.1.9/pdscan-0.1.9-arm64-linux.zip)\n- Mac - [x86_64](https://github.com/ankane/pdscan/releases/download/v0.1.9/pdscan-0.1.9-x86_64-darwin.zip) or [arm64](https://github.com/ankane/pdscan/releases/download/v0.1.9/pdscan-0.1.9-arm64-darwin.zip)\n- Windows - [x86_64](https://github.com/ankane/pdscan/releases/download/v0.1.9/pdscan-0.1.9-x86_64-windows.zip)\n\nYou can also install it with [Homebrew](#homebrew) or [Docker](#docker).\n\n## Data Stores\n\n- [Elasticsearch](#elasticsearch)\n- [Files](#files)\n- [MariaDB](#mariadb)\n- [MongoDB](#mongodb)\n- [MySQL](#mysql)\n- [OpenSearch](#opensearch)\n- [Postgres](#postgres)\n- [Redis](#redis)\n- [S3](#s3)\n- [SQLite](#sqlite)\n- [SQL Server](#sql-server)\n\n### Elasticsearch\n\n```sh\npdscan elasticsearch+http://user:pass@host:9200\n```\n\nFor HTTPS, use `elasticsearch+https://`.\n\nYou can also specify indices.\n\n```sh\npdscan elasticsearch+http://user:pass@host:9200/index1,index2\n```\n\nWildcards are also supported.\n\n```sh\npdscan \"elasticsearch+http://user:pass@host:9200/index*\"\n```\n\n### Files\n\n```sh\npdscan file://path/to/file.txt\n```\n\nYou can also specify a directory.\n\n```sh\npdscan file://path/to/directory\n```\n\nFor absolute paths, use `file:///`.\n\n```sh\npdscan file:///absolute/path/to/file.txt\n```\n\nFor paths relative to your home directory on Mac and Linux, use:\n\n```sh\npdscan file://$HOME/file.txt\n```\n\n### MariaDB\n\n```sh\npdscan mariadb://user:pass@host:3306/dbname\n```\n\n### MongoDB\n\n```sh\npdscan mongodb://user:pass@host:27017/dbname\n```\n\n### MySQL\n\n```sh\npdscan mysql://user:pass@host:3306/dbname\n```\n\n### OpenSearch\n\n```sh\npdscan opensearch+http://user:pass@host:9200\n```\n\nFor HTTPS, use `opensearch+https://`.\n\nYou can also specify indices.\n\n```sh\npdscan opensearch+http://user:pass@host:9200/index1,index2\n```\n\nWildcards are also supported.\n\n```sh\npdscan \"opensearch+http://user:pass@host:9200/index*\"\n```\n\n### Postgres\n\n```sh\npdscan postgres://user:pass@host:5432/dbname\n```\n\nAlways make sure your [connection is secure](https://ankane.org/postgres-sslmode-explained) when connecting to a database over a network you don’t fully trust. Your best option is to connect over SSH or a VPN. Another option is to use `sslmode=verify-full`. If you don’t do this, your database credentials can be compromised.\n\nIf your connection doesn’t use SSL, append to the URI:\n\n```\n?sslmode=disable\n```\n\nFor best sampling, enable the [tsm_system_rows](https://www.postgresql.org/docs/current/tsm-system-rows.html) extension (ships with Postgres 9.5+).\n\n```sql\nCREATE EXTENSION tsm_system_rows;\n```\n\n### Redis\n\n```sh\npdscan redis://user:pass@host:6379/db\n```\n\n### S3\n\n```sh\npdscan s3://bucket/path/to/file.txt\n```\n\n\u003e Requires `s3:GetObject` permission\n\nYou can also specify a prefix by ending with a `/`.\n\n```sh\npdscan s3://bucket/path/to/directory/\n```\n\n\u003e Requires `s3:ListBucket` and `s3:GetObject` permissions\n\n### SQLite\n\n```sh\npdscan sqlite://path/to/dbname.sqlite3\n```\n\n\u003e Not available with prebuilt binaries\n\n### SQL Server\n\n```sh\npdscan \"sqlserver://user:pass@host:1433?database=dbname\"\n```\n\n## Options\n\nShow the data found\n\n```sh\npdscan --show-data\n```\n\nShow low confidence matches\n\n```sh\npdscan --show-all\n```\n\nChange the sample size\n\n```sh\npdscan --sample-size 50000\n```\n\nSpecify the number of processes to use (defaults to 1)\n\n```sh\npdscan --processes 4\n```\n\nScan for only certain types of data\n\n```sh\npdscan --only email,phone,location\n```\n\nScan for all except certain types of data\n\n```sh\npdscan --except ip,mac\n```\n\nSpecify the minimum number of rows/documents/lines for a match (experimental)\n\n```sh\npdscan --min-count 10\n```\n\nSpecify a custom pattern (experimental)\n\n```sh\npdscan --pattern \"\\d{16}\"\n```\n\nOutput newline delimited JSON (experimental)\n\n```sh\npdscan --format ndjson\n```\n\n## Additional Installation Methods\n\n### Homebrew\n\nWith Homebrew, you can use:\n\n```sh\nbrew install ankane/brew/pdscan\n```\n\n### Docker\n\nGet the [Docker image](https://hub.docker.com/r/ankane/pdscan) with:\n\n```sh\ndocker pull ankane/pdscan\n```\n\nAnd run it with:\n\n```sh\ndocker run -ti ankane/pdscan \u003cconnection-uri\u003e\n```\n\nFor data stores on the host machine, use `host.docker.internal` as the hostname\n\n```sh\ndocker run -ti ankane/pdscan \"postgres://user@host.docker.internal:5432/dbname?sslmode=disable\"\n```\n\n\u003e On Linux, this requires `--add-host=host.docker.internal:host-gateway`\n\nFor files on the host machine, use:\n\n```sh\ndocker run -ti -v /path/to/files:/data ankane/pdscan file:///data\n```\n\n## History\n\nView the [changelog](https://github.com/ankane/pdscan/blob/master/CHANGELOG.md)\n\n## Contributing\n\nEveryone is encouraged to help improve this project. Here are a few ways you can help:\n\n- [Report bugs](https://github.com/ankane/pdscan/issues)\n- Fix bugs and [submit pull requests](https://github.com/ankane/pdscan/pulls)\n- Write, clarify, or fix documentation\n- Suggest or add new features\n\nTo get started with development:\n\n```sh\ngit clone https://github.com/ankane/pdscan.git\ncd pdscan\nmake test\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankane%2Fpdscan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fankane%2Fpdscan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankane%2Fpdscan/lists"}