https://github.com/miscos/discord-web-crawler

A puppeteer based webcrawler posting results to a discord webhook
https://github.com/miscos/discord-web-crawler

discord puppeteer webcrawler

Last synced: 11 days ago
JSON representation

A puppeteer based webcrawler posting results to a discord webhook

• Host: GitHub
• URL: https://github.com/miscos/discord-web-crawler
• Owner: miscOS
• License: mit
• Created: 2024-08-08T13:36:05.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2024-08-10T18:46:57.000Z (3 months ago)
• Last Synced: 2024-10-17T22:26:48.389Z (29 days ago)
• Topics: discord, puppeteer, webcrawler
• Language: TypeScript
• Homepage:
• Size: 21.5 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: news.ts
  • License: LICENSE

Awesome Lists containing this project

README

        

# Discord-web-crawler

This web crawler based on puppeteer is designed to run in an docker environment.

## Installation

Clone the repository...

```bash

  git clone https://github.com/miscOS/discord-web-crawler.git

```

...and move to the directory to run the compose file. You might want to consider changing some configuration first.

```bash

  cd discord-web-crawler

  docker-compose up

```

## Configuration

The configuration is stored in `data/config.json`. The included example will give you an idead of what the configuration should look like. It is easiest to copy it and adapt it to your own requirements.

```bash

  cp data/example.config.json data/config.json

```

## Support

[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/K3K2OQ0GL)