https://github.com/abhijithvijayan/site-scrapper
A simple websitesite scrapper
https://github.com/abhijithvijayan/site-scrapper
Last synced: 3 months ago
JSON representation
A simple websitesite scrapper
- Host: GitHub
- URL: https://github.com/abhijithvijayan/site-scrapper
- Owner: abhijithvijayan
- Created: 2022-10-16T10:47:22.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2023-03-23T21:18:59.000Z (over 2 years ago)
- Last Synced: 2024-10-11T20:32:48.771Z (9 months ago)
- Language: JavaScript
- Homepage:
- Size: 34.2 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# site-scrapper
A simple websitesite scrapper
## Docs
### Set up deta.sh
>
### Deploy to Deta
```shell
deta new --node site-scrapper
yarn install
yarn deploy
```
#### Note: For subsequent deployment run only
```shell
yarn deploy
```
### Create Slack Bot For Workspace
>
### Sample Curl
```
curl --request GET \
--url 'https://api.domain/api/v1/html?url=https://example.com&cacheTTL=1000' \
--header 'content-type: application/json'
```
- `cacheTTL` Optional. when given a number in milli-seconds, returns from cache if found in that time window. Defaults to `300000` (5 minutes)
- `url` Website URL to Fetch HTML from