https://github.com/backdevs/scrapeninja-go
A Go client for ScrapeNinja - https://scrapeninja.net
https://github.com/backdevs/scrapeninja-go
go go-module golang scrape scrapeninja scraper scraping
Last synced: about 1 month ago
JSON representation
A Go client for ScrapeNinja - https://scrapeninja.net
- Host: GitHub
- URL: https://github.com/backdevs/scrapeninja-go
- Owner: backdevs
- License: mit
- Archived: true
- Created: 2022-09-05T21:21:25.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-09-09T12:33:21.000Z (over 3 years ago)
- Last Synced: 2024-06-20T11:13:31.891Z (over 1 year ago)
- Topics: go, go-module, golang, scrape, scrapeninja, scraper, scraping
- Language: Go
- Homepage:
- Size: 6.84 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# [ScrapeNinja](https://scrapeninja.net) Go API Client
A slim Go client for interacting with the ScrapeNinja API
## More info about [ScrapeNinja](https://scrapeninja.net)
* Official website - https://scrapeninja.net
* Rapid API documentation - https://rapidapi.com/restyler/api/scrapeninja
# Installation
```shell
go get -u github.com/backdevs/scrapeninja-go
```
# Usage
```go
package main
import (
"encoding/json"
"fmt"
"github.com/backdevs/scrapeninja-go"
"log"
"net/http"
"strings"
)
type Post struct {
ID string `json:"id"`
Title string `json:"title"`
Body string `json:"body"`
UserID int `json:"userId"`
Tags []string `json:"tags"`
Reactions int `json:"reactions"`
}
func main() {
ninja := scrapeninja.New("Your-API-Key")
postJson, err := json.Marshal(&Post{
ID: "1",
UserID: 12,
Title: "ScrapeNinja Web scraper Go API Client",
Body: "Lorem ipsum etc.",
})
if err != nil {
log.Fatal(err)
}
response, err := ninja.Scrape(&scrapeninja.ScrapeRequest{
Url: "https://dummyjson.com/posts/1",
Method: http.MethodPut,
Proxy: scrapeninja.ProxyEU,
Data: string(postJson),
Headers: map[string]string{
"Content-Type": "application/json",
"Accept": "application/json",
},
})
if err != nil {
log.Fatal(err)
}
var post Post
err = json.NewDecoder(strings.NewReader(response.Body)).Decode(&post)
if err != nil {
log.Fatal(err)
}
fmt.Println(post.Title)
}
```