Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/schollz/websitechanges
Alerts you via email about a website change.
https://github.com/schollz/websitechanges
change-detection change-monitor website-change-monitor website-checker website-monitor
Last synced: 2 months ago
JSON representation
Alerts you via email about a website change.
- Host: GitHub
- URL: https://github.com/schollz/websitechanges
- Owner: schollz
- License: mit
- Created: 2020-02-16T22:44:05.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2023-04-23T10:34:42.000Z (over 1 year ago)
- Last Synced: 2024-10-18T19:54:18.512Z (3 months ago)
- Topics: change-detection, change-monitor, website-change-monitor, website-checker, website-monitor
- Language: Go
- Homepage:
- Size: 30.3 KB
- Stars: 57
- Watchers: 6
- Forks: 4
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# websitechanges
Alerts you via email about a website change, with an image of the change.
After writing a [website change script in Python](https://github.com/schollz/websitechanges), I realized I could make it much better using Go+Node. This code was very helpful for me to get registered for a high-demand [pottery class](https://schollz.com/blog/pottery/).
In this version I'm using [puppeteer](https://github.com/puppeteer/puppeteer) to do the capturing. The puppeteer code first filters out any ads (to prevent new things appearing in screen) and then captures an image of a CSS-selected content. The change-detection happens in Go and sends an email with an image of the change when any is detected.
## Example
Here is an example of a change detected from the New York times headlines.
The previous state is shown in red and the new state is shown in green.
The `config.json` used was (note the folder key is optional and is used to customize the folder name of images for easier recognition.):
```javascript
{
"watchers": [
{
"url": "https://www.nytimes.com",
"css": "h2 > span",
"folder": "nytimes"
}]
}
```
## Usage
First make sure you have `node` and `go` installed on your system.
Then download and build:
```
> git clone https://github.com/schollz/websitechanges
> cd websitechanges
> go build -v
```
Now copy the config file and fill it in (you don't have to include email, just erase that section if you don't need it):
```
> cp config-example.json config.json
> vim config.json
```
Now run. The first time you run it will download a HOSTS file that is used for filtering adds and it will install puppeteer if not installed.
```
> ./websitechanges
```
By default this will run every 5 minutes. You can adjust the delay by passing the `-delay 10` where 10 is the number of minutes to wait before the next run.
```
> ./websitechanges -delay 10
```
It automatically generates diff images everytime it encounters a change.
## Contributing
Pull requests are welcome. Feel free to...
- Revise documentation
- Add new features
- Fix bugs
- Suggest improvements
## License
MIT