https://github.com/virrages/scrape

A jquery like interface for Go website scrapping.
https://github.com/virrages/scrape

Last synced: 23 days ago
JSON representation

A jquery like interface for Go website scrapping.

• Host: GitHub
• URL: https://github.com/virrages/scrape
• Owner: VirrageS
• License: mit
• Created: 2016-08-21T13:54:24.000Z (about 8 years ago)
• Default Branch: master
• Last Pushed: 2016-08-31T16:31:42.000Z (about 8 years ago)
• Last Synced: 2024-06-21T09:07:28.157Z (5 months ago)
• Language: Go
• Size: 7.81 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# scrape

A jquery like interface for Go website scrapping.

## Usage

```go

import (

    "fmt"

    "net/http"

    "golang.org/x/net/html"

    "github.com/VirrageS/scrape"

)

func main() {

    response, err := http.Get("https://github.com/trending")

    if err != nil {

        return

    }

    root, err := html.Parse(response.Body)

    if err != nil {

        return

    }

    repos := scrape.Find(root, ".repo-list-item")

    for _, repo := range repos {

        // get url

        link := scrape.Find(repo, ".repo-list-name a")[0]

        url := "https://github.com" + scrape.Attr(link, "href")

        // get name

        name := scrape.Text(link)

        fmt.Printf("[REPO] name: %s; url: %s\n", name, url)

    }

}

```