https://github.com/neotoxic-off/juice
🧃 Scrap the world
https://github.com/neotoxic-off/juice
api scraper scraper-api
Last synced: 16 days ago
JSON representation
🧃 Scrap the world
- Host: GitHub
- URL: https://github.com/neotoxic-off/juice
- Owner: Neotoxic-off
- License: gpl-3.0
- Created: 2024-02-08T13:33:56.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2024-02-12T10:31:06.000Z (almost 2 years ago)
- Last Synced: 2024-02-12T11:38:30.406Z (almost 2 years ago)
- Topics: api, scraper, scraper-api
- Language: Python
- Homepage:
- Size: 1.47 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Install
```BASH
git clone https://github.com/Neotoxic-off/Juice
cd Juice
docker-compose up --build
```
## Usage
```PYTHON
r = requests.post("http://127.0.0.1:8081/scrap", json = {
"method": "GET",
"host": "https://example.com",
"params": [],
"tag": "a",
"element": "href",
"attrs": {},
"headers": {},
"data": {},
"range": [0, 10],
"ptag": "page"
})
```
## Options
- Method: `GET`, `POST`, `PUT`, `DELETE`, `PATCH`
- Host: `https://example.com`
- Params:
```JSON
[
["param1", "paramvalue1"]
]
```
- Tag: `img`, `a`, `p`, `...`
- Element: `href`, `src`, `class`, `...`
- Attrs:
```JSON
{
"class": "avatar",
"id": "main"
}
```
- Headers:
```JSON
{
"User-Agent": "Example",
"Token": "issou"
}
```
- Data:
```JSON
{
"EXAMPLE": "DATA"
}
```
- Range: `[0, 3]`
- Ptag: `page`, `p`
## Request
- Method: `POST`
- Route: `/scrap`
- Json:
```JSON
{
"method": "GET",
"host": "https://github.com/Neotoxic-off",
"params": [
["tab", "repositories"]
],
"tag": "img",
"element": "src",
"attrs": {
"class": "avatar avatar-user width-full border color-bg-default"
},
"headers": {},
"data": {},
"range": [0, 3],
"ptag": "page"
}
```