https://github.com/tb0hdan/domains

World’s single largest Internet domains dataset
https://github.com/tb0hdan/domains

colly dataset internet-domains scrapy search-engines yacy

Last synced: 21 days ago
JSON representation

World’s single largest Internet domains dataset

• Host: GitHub
• URL: https://github.com/tb0hdan/domains
• Owner: tb0hdan
• License: bsd-3-clause
• Created: 2020-01-12T22:39:04.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2026-01-15T18:08:44.000Z (about 2 months ago)
• Last Synced: 2026-01-15T20:48:12.778Z (about 2 months ago)
• Topics: colly, dataset, internet-domains, scrapy, search-engines, yacy
• Language: JavaScript
• Homepage: https://domainsproject.org
• Size: 1.68 GB
• Stars: 1,023
• Watchers: 39
• Forks: 165
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • Changelog: NEWS.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE

Awesome Lists containing this project

• my-awesome - tb0hdan/domains - domains,scrapy,search-engines,yacy pushed_at:2026-02 star:1.1k fork:0.2k World’s single largest Internet domains dataset (JavaScript)
• awesome-hacking-lists - tb0hdan/domains - World’s single largest Internet domains dataset (HTML)