https://github.com/firefart/pastebin_scraper
golang program to parse Pastebin for keywords and send them per E-Mail
https://github.com/firefart/pastebin_scraper
go golang pastebin pastebin-scraper scraper security
Last synced: 4 months ago
JSON representation
golang program to parse Pastebin for keywords and send them per E-Mail
- Host: GitHub
- URL: https://github.com/firefart/pastebin_scraper
- Owner: firefart
- License: mit
- Created: 2018-04-24T12:59:28.000Z (about 8 years ago)
- Default Branch: main
- Last Pushed: 2025-08-30T23:04:14.000Z (10 months ago)
- Last Synced: 2025-08-31T01:05:36.788Z (10 months ago)
- Topics: go, golang, pastebin, pastebin-scraper, scraper, security
- Language: Go
- Homepage:
- Size: 255 KB
- Stars: 34
- Watchers: 3
- Forks: 7
- Open Issues: 0
-
Metadata Files:
- Readme: Readme.md
- License: LICENSE
Awesome Lists containing this project
README
# Pastebin Scraper
Constantly monitors the Pastebin scrape API and sends E-Mails when a keyword matches. This program needs a paid [Pastebin PRO account](https://pastebin.com/pro).
You need to put the IP you are scraping from into the [Pastebin admin panel](https://pastebin.com/api_scraping_faq).
The sent email contains the Paste metadata, the first matched line per keyword and the zipped paste as an attachment.
Keywords are set to match with a starting [regex boundary](https://www.regular-expressions.info/wordboundaries.html). Matching of CIDRs is also supported (see config.json.sample).
Expected errors during execution are also sent via E-Mail to the E-Mail address configured in `config.json`.
For sending mails you should setup a local SMTP server like postfix to handle resubmission, signing and so on for you. SMTP authentication is currently not implemented.
## Installation on a systemd based system
- Build binary or download it
```bash
make
```
or
```bash
go get
go build
```
or
```bash
make_linux.bat
make_windows.bat
```
- Add a user to run the binary
```bash
adduser --system pastebin
```
- Copy everything to home dir
```bash
cp -R checkout_dir /home/pastebin/
```
- Edit the config
```bash
cp /home/pastebin/config.json.sample /home/pastebin/config.json
vim /home/pastebin/config.json
```
- Install the service
```bash
cd /home/pastebin
./install_service.sh
```
- Watch the logs
```bash
journalctl -u pastebin_scraper.service -f
```
## Example Config
```json
{
"mailserver": "localhost",
"mailport": 25,
"mailfrom": "Pastebin Alert ",
"mailto": "Unknown Person ",
"mailonerror": true,
"mailtoerror": "error@xxx.xom",
"timeout": "10s",
"keywords": [
{
"keyword": "keyword1",
"exceptions": ["exception1", "exception2", "exception3"]
},
{
"keyword": "keyword2",
"exceptions": ["exception1", "exception2", "exception3"]
},
{
"keyword": "keyword3",
"exceptions": ["exception1", "exception2", "exception3"]
}
],
"cidrs": ["10.0.0.0/8", "192.168.0.0/16"]
}
```