https://github.com/promptapi/scraper-cli
CLI Tool for Prompt API's Scraper API, works for macOS, linux and windows :)
https://github.com/promptapi/scraper-cli
api-marketplace cli css-selector css-selector-parser data-extraction golang-cli image-scraper image-scraping linux-shell macos-shell promptapi scraper scraper-api web-scraper web-scraping windows-shell
Last synced: 27 days ago
JSON representation
CLI Tool for Prompt API's Scraper API, works for macOS, linux and windows :)
- Host: GitHub
- URL: https://github.com/promptapi/scraper-cli
- Owner: promptapi
- License: mit
- Created: 2020-09-09T04:13:41.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2020-10-06T08:20:49.000Z (over 5 years ago)
- Last Synced: 2024-06-20T03:42:15.488Z (over 1 year ago)
- Topics: api-marketplace, cli, css-selector, css-selector-parser, data-extraction, golang-cli, image-scraper, image-scraping, linux-shell, macos-shell, promptapi, scraper, scraper-api, web-scraper, web-scraping, windows-shell
- Language: Go
- Homepage: https://promptapi.com/marketplace/description/scraper-api
- Size: 27.3 KB
- Stars: 1
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
[](https://pkg.go.dev/github.com/promptapi/scraper-cli)
[](https://goreportcard.com/report/github.com/promptapi/scraper-cli)
[](https://travis-ci.org/promptapi/scraper-cli)
# Prompt API - Scraper - Command Line Interface
Do you like [Scraper API][scraper-api]? Don’t you like to access this beauty
from your terminal?
---
## Requirements
1. You need to signup for [Prompt API][promptapi-signup]
1. You need to subscribe [scraper api][scraper-api], test drive is **free!!!**
1. You need to set `PROMPTAPI_TOKEN` environment variable after subscription.
If you have `golang` environment installed, you can fetch via
```bash
$ go get -u github.com/promptapi/scraper-cli/cmd/scraper-cli
```
Or you can download binaries from [releases][releases] page
---
## Usage
```bash
$ scraper-cli -h
usage: scraper-cli [flags...]
scraper-cli is a command-line interface for Prompt API's Scraper API. Details
can be found:
https://promptapi.com/marketplace/description/scraper-api
you need to signup for Prompt API to get your PROMPTAPI_TOKEN. you can signup
from:
https://promptapi.com/#signup-form
application looks for PROMPTAPI_TOKEN environment variable. if you pass
"token" flag, this will override environment variable lookup.
required flag(s):
-url web url/address to scrape
optional flags:
-country 2 character country code
-token promptapi apikey instead of PROMPTAPI_TOKEN env-var
-username for HTTP Realm auth username
-password for HTTP Realm auth password
-cookie URL Encoded cookie header
-referer HTTP referer header
-selector CSS style selector path such as: a.btn div li
-version display version information
-header request header(s)
-help, -h display help
examples:
$ scraper-cli -help
$ scraper-cli -url "https://promptapi.com"
$ scraper-cli -url "https://promptapi.com" -country "EE"
$ scraper-cli -url "https://promptapi.com" -country "EE" -selector "a.btn div li"
$ PROMPTAPI_TOKEN="your-api-key" scraper-cli -url "https://promptapi.com"
$ scraper-cli -url "https://promptapi.com" -token "your-api-key"
$ scraper-cli -url "https://promptapi.com" -token "your-api-key" -header "X-Referer: https://www.google.com"
$ scraper-cli -url "https://promptapi.com" -token "your-api-key" -header "X-Referer: https://www.google.com" -header "X-Custom-Header: Hello"
```
---
## License
This project is licensed under MIT
---
## Contributer(s)
* [Prompt API](https://github.com/promptapi) - Creator, maintainer
---
## Contribute
All PR’s are welcome!
1. `fork` (https://github.com/promptapi/scraper-cli/fork)
1. Create your `branch` (`git checkout -b my-feature`)
1. `commit` yours (`git commit -am 'Add awesome features...'`)
1. `push` your `branch` (`git push origin my-feature`)
1. Than create a new **Pull Request**!
This project is intended to be a safe,
welcoming space for collaboration, and contributors are expected to adhere to
the [code of conduct][coc].
---
[scraper-api]: https://promptapi.com/marketplace/description/scraper-api
[promptapi-signup]: https://promptapi.com/#signup-form
[coc]: https://github.com/promptapi/scraper-cli/blob/main/CODE_OF_CONDUCT.md
[releases]: https://github.com/promptapi/scraper-cli/releases