Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/davemolk/urlbits
Pipe in a bunch of urls and see their bits.
https://github.com/davemolk/urlbits
go tools url-parser
Last synced: 16 days ago
JSON representation
Pipe in a bunch of urls and see their bits.
- Host: GitHub
- URL: https://github.com/davemolk/urlbits
- Owner: davemolk
- License: mit
- Created: 2022-11-28T20:31:51.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2022-12-13T02:12:08.000Z (about 2 years ago)
- Last Synced: 2024-06-20T10:22:22.118Z (6 months ago)
- Topics: go, tools, url-parser
- Language: Go
- Homepage:
- Size: 7.81 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# urlbits
Pipe in a bunch of urls and see their bits.
[](https://goreportcard.com/report/github.com/davemolk/urlbits)
## Overview
The default behavior is to parse the urls and print their components to Stdout as json. Use a flag to narrow the output to a specific component.
## Examples
```
$ cat urls.txt
https://www.google.com/search?name=golang&language=en&mascot=gopher
https://go.dev/play/
google.com
foo/bar
postgres://user:[email protected]:5432/path1?k=v#f
totaljunk
```
### Default Usage
```
$ cat urls.txt | urlbits
{
"scheme": "https",
"host": "www.google.com",
"path": "/search",
"raw_query": "name=golang\u0026language=en\u0026mascot=gopher"
}
{
"scheme": "https",
"host": "go.dev",
"path": "/play/"
}
{
"scheme": "postgres",
"user": "user:pass",
"host": "host.com:5432",
"path": "/path1",
"raw_query": "k=v#f"
}
```
### Keys
```
$ cat urls.txt | urlbits -keys
name
language
mascot
k
```
### Paths
```
$ cat urls.txt | urlbits -paths
/search
/play/
/path1
```
## Install
First, you'll need to [install go](https://golang.org/doc/install). Then, run the following command:
```
go install github.com/davemolk/urlbits@latest
```
## Command-line Options
```
Usage of urlbits:
-domains bool
Output the domains.
-keys bool
Output the keys.
-kv bool
Output keys and values.
-paths bool
Output the paths.
-save bool
Save output to a file.
-user bool
Output user information (username and password).
-values bool
Output the values.
-validate bool
Strip out URLs without a scheme and host.
-verbose bool
Verbose output.
```
### Why pipelines?
While they do work well in this scenario, I was trying to think of ways to practice using pipelines in Go that were a little more practical than what I was finding in tutorials and books.