https://github.com/devjiwonchoi/frameset-crawler

Crawler built with Puppeteer.js for crawling into Classic HTML4 frameset websites.
https://github.com/devjiwonchoi/frameset-crawler

Last synced: about 1 year ago
JSON representation

Crawler built with Puppeteer.js for crawling into Classic HTML4 frameset websites.

• Host: GitHub
• URL: https://github.com/devjiwonchoi/frameset-crawler
• Owner: devjiwonchoi
• License: mit
• Created: 2023-04-12T07:33:33.000Z (about 3 years ago)
• Default Branch: main
• Last Pushed: 2023-04-17T18:23:34.000Z (about 3 years ago)
• Last Synced: 2024-10-11T12:04:11.736Z (over 1 year ago)
• Language: TypeScript
• Size: 25.4 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
Requirements:

- Chrome

- node.js

- yarn

- patience

Scripts:

"yarn crawl" - Saves data as data.json on root dir.

"yarn upload" - Uploads data.json file to the database.

Note: This crawler was designed for crawling websites built with frameset(HTML4).

Therefore please be aware that the process of this crawling system might be differ from the latest(2023/03/03) puppeteer method.