{"id":19435957,"url":"https://github.com/ralscha/selfhost-hibp-passwords","last_synced_at":"2025-04-24T21:30:45.890Z","repository":{"id":57715629,"uuid":"191983277","full_name":"ralscha/selfhost-hibp-passwords","owner":"ralscha","description":"Tools for self-hosting the Have I Been Pwned Password database. Xodus importer and query library","archived":false,"fork":false,"pushed_at":"2025-03-29T08:38:15.000Z","size":272,"stargazers_count":8,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-29T09:28:22.810Z","etag":null,"topics":["hibp","hibpwned","java","passwords","spring-boot","xodus"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ralscha.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-14T17:38:55.000Z","updated_at":"2025-03-29T08:38:18.000Z","dependencies_parsed_at":"2023-01-30T19:15:48.024Z","dependency_job_id":"6f7af43e-e614-4b83-8f47-227d138fff81","html_url":"https://github.com/ralscha/selfhost-hibp-passwords","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fselfhost-hibp-passwords","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fselfhost-hibp-passwords/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fselfhost-hibp-passwords/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ralscha%2Fselfhost-hibp-passwords/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ralscha","download_url":"https://codeload.github.com/ralscha/selfhost-hibp-passwords/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250712880,"owners_count":21475093,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hibp","hibpwned","java","passwords","spring-boot","xodus"],"created_at":"2024-11-10T15:08:35.678Z","updated_at":"2025-04-24T21:30:45.557Z","avatar_url":"https://github.com/ralscha.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Self-host Have I Been Pwned Passwords\r\n\r\nTool for importing the Have I Been Pwned Passwords list into a local [Jetbrain Xodus database](https://github.com/JetBrains/xodus). Java library for querying the database. \r\n\r\n\r\n## Prerequisites \r\n   - Sufficient drive storage. About 57GB during import, 24GB for normal operation.\r\n\r\n## Import\r\n\r\n   1. Download Java 17 or newer. JRE is sufficient\r\n       - https://adoptium.net/\r\n       - https://jdk.java.net/17/\r\n       - https://aws.amazon.com/corretto/\r\n       - https://www.azul.com/downloads/?package=jdk\r\n\r\n   2. Download the Pwned Passwords. Either use the [official downloader](https://github.com/HaveIBeenPwned/PwnedPasswordsDownloader) or [my downloader](https://github.com/ralscha/hibp-passwords-downloader) written in Go. The importer expects the hashes in individual files. \r\n\r\n   3. Download [importer](https://github.com/ralscha/selfhost-hibp-passwords/releases/download/importer-1.1.0/hibp-passwords-importer.jar)\r\n   4. Run the import tool. Point it to the directory that contains the downloaded hash files        \r\n      `java -jar hibp-passwords-importer.jar import \u003chashesdir\u003e \u003cdatabase_directory_name\u003e`\r\n\r\n\r\n## Query\r\n\r\n### Library\r\n\r\nAdd library to your project.\r\n\r\n```\r\n\u003c!-- Maven --\u003e\r\n\u003cdependency\u003e\r\n   \u003cgroupId\u003ech.rasc.hibppasswords\u003c/groupId\u003e\r\n   \u003cartifactId\u003equery\u003c/artifactId\u003e\r\n   \u003cversion\u003e1.0.0\u003c/version\u003e\r\n\u003c/dependency\u003e\r\n```      \r\n\r\n```\r\n// Gradle\r\ndependencies {\r\n    compile 'ch.rasc.hibppasswords:query:1.0.0'\r\n}\r\n````\r\n\r\nAn application can query the database with a plain text password or SHA-1 hash. Both\r\nmethods return either how many times a string or SHA-1 hash appears in the data set, or `null` if\r\nthe given password is not found.\r\n\r\n`haveIBeenPwnedRange` implements a k-Anonymity model that supports searching with a partial hash:    \r\nhttps://haveibeenpwned.com/API/v2#SearchingPwnedPasswordsByRange\r\n\r\n```java\r\nimport ch.rasc.hibppasswords.query.HibpPasswordsQuery;\r\nimport ch.rasc.hibppasswords.query.RangeQueryResult;\r\nimport java.nio.file.Path;\r\n\r\nPath db = Paths.get(\"...\"); // Path to local database\r\nInteger count = HibpPasswordsQuery.haveIBeenPwnedPlain(db, \"123456\");\r\ncount = HibpPasswordsQuery.haveIBeenPwnedSha1(db, \"FFFFFFBFAD0B653BDAC698485C6D105F3C3682B2\");\r\n\r\nList\u003cRangeQueryResult\u003e result = HibpPasswordsQuery.haveIBeenPwnedRange(db, \"FFFFF\");\r\n```\r\nThese three methods open and close the database for each call. To speed up queries, an application can instantiate the Xodus environment once and pass it as the first argument. \r\n\r\n```java\r\nimport jetbrains.exodus.env.Environment;\r\nimport jetbrains.exodus.env.Environments;\r\n\r\nPath db = Paths.get(\"...\"); // Path to local database\r\n\r\ntry (Environment environment = Environments.newInstance(db.toFile())) {\r\n  Integer count = HibpPasswordsQuery.haveIBeenPwnedPlain(environment, \"123456\");\r\n  count = HibpPasswordsQuery.haveIBeenPwnedSha1(environment, \"FFFFFFBFAD0B653BDAC698485C6D105F3C3682B2\");\r\n\r\n  List\u003cRangeQueryResult\u003e result = HibpPasswordsQuery.haveIBeenPwnedRange(environment, \"FFFFF\");\r\n}\r\n```\r\nSee [Spring Boot example](https://github.com/ralscha/selfhost-hibp-passwords/blob/master/demo/src/main/java/ch/rasc/hibppasswords/Application.java)\r\n\r\n\r\n\r\n### Command Line\r\n\r\nThe local database can be queried with the import tool.\r\n  - With plain text password:    \r\n    `java -jar hibp-passwords-importer.jar query-plain 123456 \u003cpath_to_database\u003e`\r\n\r\n  - With SHA1 hash:    \r\n    `java -jar hibp-passwords-importer.jar query-sha1 FFFFFFFEE791CBAC0F6305CAF0CEE06BBE131160 \u003cpath_to_database\u003e`\r\n\r\n\r\n## HTTP Demo\r\n\r\nThe repository hosts a Spring Boot demo with the HTTP endpoints. \r\n\r\n```sh\r\n$ git clone https://github.com/ralscha/selfhost-hibp-passwords.git\r\n$ cd selfhost-hibp-passwords/demo\r\n$ JAVA_HOME=\u003cpath_to_jdk\u003e ../mvnw spring-boot:run -Dspring-boot.run.arguments=--app.hibp-database-dir=\u003cpath_to_database\u003e\r\n\r\n\r\n# in another shell\r\n\r\n$ curl http://localhost:8080/range/7C4A8\r\n001CE884342580D934A29D94060B3796C30:2\r\n00AD0FC3FA522D0474F9A28FD478C06669D:1\r\n...\r\n\r\n$ curl http://localhost:8080/plain/mypassword\r\n38621\r\n\r\n$ curl http://localhost:8080/sha1/7C4A8D7F20D435D1F9F7FFA96C28E216E98163\r\n13\r\n```\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fralscha%2Fselfhost-hibp-passwords","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fralscha%2Fselfhost-hibp-passwords","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fralscha%2Fselfhost-hibp-passwords/lists"}