{"id":41052211,"url":"https://github.com/felix/dhtsearch","last_synced_at":"2026-01-22T11:28:54.405Z","repository":{"id":40633434,"uuid":"93838250","full_name":"felix/dhtsearch","owner":"felix","description":"[mirror] Standalone DHT search","archived":false,"fork":false,"pushed_at":"2025-10-18T22:21:02.000Z","size":3679,"stargazers_count":61,"open_issues_count":1,"forks_count":10,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-10-19T13:29:39.783Z","etag":null,"topics":["bittorrent","dht","dht-network","go","golang","self-hosted","torrent-search-engine"],"latest_commit_sha":null,"homepage":"https://src.userspace.com.au/dhtsearch/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/felix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-09T08:43:20.000Z","updated_at":"2025-10-04T14:03:13.000Z","dependencies_parsed_at":"2022-07-14T04:20:30.825Z","dependency_job_id":null,"html_url":"https://github.com/felix/dhtsearch","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/felix/dhtsearch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felix%2Fdhtsearch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felix%2Fdhtsearch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felix%2Fdhtsearch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felix%2Fdhtsearch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/felix","download_url":"https://codeload.github.com/felix/dhtsearch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/felix%2Fdhtsearch/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28662071,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T01:17:37.254Z","status":"online","status_checked_at":"2026-01-22T02:00:07.137Z","response_time":144,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bittorrent","dht","dht-network","go","golang","self-hosted","torrent-search-engine"],"created_at":"2026-01-22T11:28:53.744Z","updated_at":"2026-01-22T11:28:54.400Z","avatar_url":"https://github.com/felix.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DHT Search\n\nThis is a Mainline DHT crawler and BitTorrent client which also provides an\nHTTP interface to query the indexed data.\n\nDistributed Hash Table (DHT) is a distributed system storing key/value pairs,\nin this case it is specifically Mainline DHT, the type used by BitTorrent\nclients. The crawler also implements a number of extensions which enable it to\nget the metadata for the torrent enabling indexing and later searching.\n\nThe crawler joins the DHT network and listens to the conversations between\nnodes, keeping track of interesting packets. The most interesting packets are\nthose where another node announces they have a torrent available.\n\nThis BitTorrent client only downloads the torrent metadata. The actual files\nhosted by the remote nodes are not retrieved.\n\n## Features\n\n- **Tagging** of torrents metadata is fetched. The torrent is tagged using a\n  set of regular expressions matched against the torrent name and the files in\n  the torrent.\n\n- **Filtering** can be done by tags. By default all torrents tagged 'adult' are\n  not indexed. See the SkipTags option in the configuration file.\n\n- **Full Text Search** using PostgreSQL's or Sqlite's text search vectors.\n  Torrent names are weighted more than file names.\n\n- **Statistics** for the crawler process are available when the HTTP server is\n  enabled. Fetch the JSON from the `/status` endpoint.\n\n- **Custom tags** can be defined in the configuration file.\n\n- **Cross-platform** builds for Windows, Macos, Linux, FreeBSD, and OpenBSD\n\n## Installation\n\nThere is a Makefile for GNU make:\n\n```shell\n$ make build\n```\n\n## Usage\n\nYou are going to need to sort out any port forwarding if you are behind NAT so\nremote nodes can get to yours.\n\nConfiguration is done via a [TOML](https://github.com/toml-lang/toml) formatted\nfile or via flags passed to the daemon.\n\nThe following command line flags are available:\n\n      -base-port int\n            listen port (and first of multiple ports) (default 6881)\n      -debug\n            provide debug output\n      -dsn string\n            Database DSN (default \"postgres://dht:dht@localhost/dht?sslmode=disable\")\n      -http-address string\n            HTTP listen address:port (default \"localhost:6880\")\n      -no-http\n            no HTTP service\n      -num-nodes int\n            number of nodes to start (default 1)\n      -quiet\n            log only errors\n\nand the following \"advanced\" options:\n\n      -max-bt-workers int\n            max number of BT workers (default 256)\n      -max-dht-workers int\n            max number of DHT workers (default 256)\n      -peer-cache-size int\n            memory cache of seen peers (default 200)\n      -routing-table-size int\n            number of remote nodes in routing table (default 1000)\n      -tcp-timeout int\n            TCP timeout in seconds (default 10)\n      -udp-timeout int\n            UDP timeout in seconds (default 10)\n\nThese options enable you to start a number of DHT nodes thus implementing a\nsmall scale [Sybil attack](https://en.wikipedia.org/wiki/Sybil_attack). The\nfirst DHT node will take the port specified and each subsequent port is for the\nfollowing nodes.\n\n## TODO\n\n- Enable rate limiting.\n- Improve our manners on the DHT network (replies etc.).\n- Improve the routing table implementation.\n- Add results pagination.\n- Add tests!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffelix%2Fdhtsearch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffelix%2Fdhtsearch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffelix%2Fdhtsearch/lists"}