https://github.com/pulipulichen/api-crawler-example

An example to build a crawler by GitHub Action.
https://github.com/pulipulichen/api-crawler-example

Last synced: about 14 hours ago
JSON representation

An example to build a crawler by GitHub Action.

• Host: GitHub
• URL: https://github.com/pulipulichen/api-crawler-example
• Owner: pulipulichen
• License: mit
• Created: 2023-02-07T11:22:33.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-08-03T09:34:33.000Z (6 months ago)
• Last Synced: 2024-08-03T10:42:49.194Z (6 months ago)
• Language: JavaScript
• Size: 34.2 KB
• Stars: 3
• Watchers: 4
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# API-Crawler-Example

An example to build a crawler by GitHub Action.

# Instruction

1. Fork this repo.

2. Edit `app/index.js`

  * Use `await GetHTML(url, options)` to download the HTML source code.

  * Use `$` jQuery to traverse and manipulate the content.

3. Setup GitHub Pages for Branch `gh-pages`

4. Schedule the job in `.github/workflows/crawler.yml`

# Output API

- https://pulipulichen.github.io/API-Crawler-Example/output.json