https://github.com/itzg/restify

Enables REST-like access to HTML pages by scraping and parsing them into JSON.
https://github.com/itzg/restify

Last synced: 11 days ago
JSON representation

Enables REST-like access to HTML pages by scraping and parsing them into JSON.

• Host: GitHub
• URL: https://github.com/itzg/restify
• Owner: itzg
• License: apache-2.0
• Created: 2016-06-05T01:02:26.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2024-10-07T12:40:13.000Z (about 1 month ago)
• Last Synced: 2024-10-12T18:52:57.434Z (about 1 month ago)
• Language: Go
• Size: 93.8 KB
• Stars: 40
• Watchers: 4
• Forks: 7
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Funding: .github/FUNDING.yml
  • License: LICENSE.txt

Awesome Lists containing this project

README

        

A little utility/library (written in Go) that enables REST-like access to HTML pages by scraping and parsing them into JSON.

[![Test](https://github.com/itzg/restify/actions/workflows/test.yml/badge.svg)](https://github.com/itzg/restify/actions/workflows/test.yml)

```

usage: restify [] 

Flags:

  --help                      Show context-sensitive help (also try --help-long and --help-man).

  --class=CLASS               If specified, first-level elements encountered with this class will be extracted.

  --id=ID                     If specified, the element with this id will be extracted.

  --tag=TAGNAME               If specified, the first-level element with this tag name will be extracted.

  --attribute=ATTRIBUTE       If specified, as key=value, the element with the given attribute name set to the given value is extracted.

  --version                   Print version and exit

  --debug                     Enable debugging output

  --user-agent="restify/1.4.0"  user-agent header to provide with request

Args:

    A URL to RESTify into JSON

```

## Output Structure

When a successful URL retrieval and match occurs, the utility will output

the JSON conversion to stdout.

The top-level structure is an array of `jsonNode`, where each `jsonNode` is

structured as:

```

{

  "name":   "...element name...",

  "class":  "...class attribute, if present...",

  "id":     "...id attribute, if present...",

  "href":   "...href attribute, if present...",

  "text":   "...element text content, if present...",

  "elements": [

    ...jsonNodes, if present...

  ]

}

```

## Examples

Locate the latest Minecraft Bedrock server version by picking off the ``'s with `data-platform` set:


```bash

restify --attribute=data-platform https://www.minecraft.net/en-us/download/server/bedrock/

```

which produces:

```json

[

  {"name":"a","attributes":{"data-platform":"serverBedrockWindows","role":"button"},"class":"btn btn-disabled-outline mt-4 downloadlink","href":"https://minecraft.azureedge.net/bin-win/bedrock-server-1.12.0.28.zip","text":"Download"},

  {"name":"a","attributes":{"data-platform":"serverBedrockLinux","role":"button"},"class":"btn btn-disabled-outline mt-4 downloadlink","href":"https://minecraft.azureedge.net/bin-linux/bedrock-server-1.12.0.28.zip","text":"Download"}

]

```

or to grab just the Linux instance:

```bash

restify --attribute=data-platform=serverBedrockLinux https://www.minecraft.net/en-us/download/server/bedrock/

```

## Using as a library

The package `github.com/itzg/restify` provides the library functions used by the command-line utility.