https://github.com/syrusakbary/gdom

DOM Traversing and Scraping using GraphQL
https://github.com/syrusakbary/gdom

Last synced: about 2 months ago
JSON representation

DOM Traversing and Scraping using GraphQL

• Host: GitHub
• URL: https://github.com/syrusakbary/gdom
• Owner: syrusakbary
• License: bsd-3-clause
• Created: 2016-02-25T18:43:23.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2019-10-22T23:49:24.000Z (almost 5 years ago)
• Last Synced: 2024-06-05T13:36:09.276Z (4 months ago)
• Language: Python
• Homepage: http://gdom.graphene-python.org
• Size: 19.5 KB
• Stars: 1,245
• Watchers: 34
• Forks: 41
• Open Issues: 5
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-graphql - gdom - DOM Traversing and Scraping using GraphQL. (Tools / Julia Libraries)
• awesome-list - gdom

README

        
# GDOM

GDOM is the next generation of web-parsing, powered by `GraphQL`

syntax and the [Graphene framework](http://graphene-python.org).

Install it typing in your console:

```bash

pip install gdom

```

**DEMO**: [Try GDOM online](http://gdom.graphene-python.org/)

## Usage

You can either do `gdom --test` to start a test server for testing

queries or

```bash

gdom QUERY_FILE

```

This command will write in the standard output (or other output if specified

via `--output`) the resulting JSON.

Your `QUERY_FILE` could look similar to this:

```graphql

{

  page(url:"http://news.ycombinator.com") {

    items: query(selector:"tr.athing") {

      rank: text(selector:"td span.rank")

      title: text(selector:"td.title a")

      sitebit: text(selector:"span.comhead a")

      url: attr(selector:"td.title a", name:"href")

      attrs: next {

         score: text(selector:"span.score")

         user: text(selector:"a:eq(0)")

         comments: text(selector:"a:eq(2)")

      }

    }

  }

}

```

## Advanced usage

If you want to generalize your gdom query to any page, just rewrite your

query file adding the `$page` var. So should look to something like

this:

```graphql

query ($page: String) {

  page(url:$page) {

    # ...

  }

}

```

And then, query it like:

```bash

gdom QUERY_FILE http://news.ycombinator.com

```