https://github.com/thiti-dev/scraperor
scraping service for public usage but with some catch xD
https://github.com/thiti-dev/scraperor
batch-script colly data-collector golang graphql migration nethttp postgresql scraper
Last synced: about 2 months ago
JSON representation
scraping service for public usage but with some catch xD
- Host: GitHub
- URL: https://github.com/thiti-dev/scraperor
- Owner: Thiti-Dev
- Created: 2021-07-22T12:49:13.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2021-07-23T20:58:25.000Z (almost 5 years ago)
- Last Synced: 2025-01-06T03:42:19.646Z (over 1 year ago)
- Topics: batch-script, colly, data-collector, golang, graphql, migration, nethttp, postgresql, scraper
- Language: Go
- Homepage: https://scraperor.herokuapp.com/scrape
- Size: 1.77 MB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
[](https://circleci.com/gh/Thiti-Dev/scraperor/tree/master)
## 🎓 Scraperor [scraping service]
A free services that helps you in seeking any information/data of any website in the world
## 💡 Usage
##### GRAPHQL
```url
https://scraperor.herokuapp.com/query
```
###### Example
```gql
mutation{
scrapeNow(input:{
pointer: ".user-profile-bio div",
target_url: "https://github.com/Thiti-Dev"
}){
elements
}
}
```
##### API-ENDPOINT
```url
{{POST}}: https://scraperor.herokuapp.com/scrape
```
###### Example body
```json
{
"target_url": "https://github.com/Thiti-Dev",
"pointer": ".user-profile-bio div"
}
```
## 📕 Important notes
- This is an alpha version so you can use our scraping engine without any registration required
- In the official version, you will have to make a registration before using our service and we'll collect all of the data ex. transmission data from host -> target , collected elements, etc . . .