https://github.com/n0tan3rd/squidwarc

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
https://github.com/n0tan3rd/squidwarc

browser-automation chrome chrome-headless crawler crawling headless-chrome high-fidelity-preservation puppeteer webarchives webarchiving

Last synced: about 1 month ago
JSON representation

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

• Host: GitHub
• URL: https://github.com/n0tan3rd/squidwarc
• Owner: N0taN3rd
• License: apache-2.0
• Created: 2017-07-20T06:57:07.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2020-05-19T04:31:59.000Z (over 4 years ago)
• Last Synced: 2024-12-15T01:41:24.519Z (about 2 months ago)
• Topics: browser-automation, chrome, chrome-headless, crawler, crawling, headless-chrome, high-fidelity-preservation, puppeteer, webarchives, webarchiving
• Language: JavaScript
• Homepage: https://n0tan3rd.github.io/Squidwarc/
• Size: 2.39 MB
• Stars: 170
• Watchers: 10
• Forks: 26
• Open Issues: 11

Awesome Lists containing this project

• awesome-nodejs - Squidwarc - High fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head.  (Repository / Crawler)