https://github.com/vaayne/tap
CLI toolkit for structured web scraping, content extraction, and browser automation
https://github.com/vaayne/tap
Last synced: 2 months ago
JSON representation
CLI toolkit for structured web scraping, content extraction, and browser automation
- Host: GitHub
- URL: https://github.com/vaayne/tap
- Owner: vaayne
- License: mit
- Created: 2026-03-31T11:16:52.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2026-05-11T14:58:15.000Z (2 months ago)
- Last Synced: 2026-05-11T16:41:19.338Z (2 months ago)
- Language: Go
- Homepage: https://tap.vaayne.com
- Size: 545 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Agents: AGENTS.md
Awesome Lists containing this project
README
# 🚰 Tap
Tap into any website from your terminal.
## Install
```bash
curl -fsSL https://raw.githubusercontent.com/vaayne/tap/main/scripts/install.sh | sh
```
Or with Go:
```bash
go install github.com/vaayne/tap/cmd/tap@latest
```
Upgrade later with:
```bash
tap upgrade
```
Browser features use Chrome by default. Check dependencies with `tap doctor`.
## Quick start
### Structured data from site scripts
```bash
tap site list
tap site hackernews/top
tap site run github/repo repo=vaayne/tap
tap site search weather
```
Scripts auto-sync from [tap.vaayne.com](https://tap.vaayne.com) into the local cache. Local overrides at `~/.config/tap/sites/{site}/{script}.js` take precedence.
### Readable content from any URL
```bash
tap fetch https://example.com/article
tap fetch --json https://example.com/article
tap fetch -b https://example.com/app --wait-selector '.content'
```
### Use a visible browser for auth when needed
```bash
tap attach chrome
tap browser open https://github.com/login --show
tap site -b github/notifications
tap fetch -b https://github.com/notifications
```
### Reuse your existing Chrome
Chrome must already expose DevTools.
```bash
tap attach chrome
tap attach status
tap browser open https://example.com
tap browser snapshot --interactive
tap browser click '#submit'
tap browser click @e1
tap browser text
```
You can also attach explicitly:
```bash
tap attach chrome --browser-url http://127.0.0.1:9222
tap attach chrome --port-file ~/Library/Application\ Support/Google/Chrome/DevToolsActivePort
```
### Browser workflow
```bash
tap browser open https://news.ycombinator.com
tap browser open https://github.com --new-tab
tap browser tabs
tap browser switch tab-2
tap browser screenshot --output github.png
tap browser status
```
### Attached Chrome workflow
```bash
tap attach chrome
tap attach status --json
tap browser evaluate 'document.title'
tap browser screenshot
```
If the attached state becomes stale, rerun `tap attach chrome`.
## Command map
```text
tap
├── site structured extraction from known sites
├── fetch clean readable content from arbitrary URLs
├── browser open pages and automate the current browser context
├── attach connect tap to an existing Chrome browser
├── status show the active browser context and current tab
├── doctor dependency and environment checks
├── upgrade update tap
└── completion generate shell completion scripts
```
## Shell completion
Tap can generate shell completion scripts for bash, zsh, fish, and pwsh.
```bash
# Bash
source <(tap completion bash)
# Zsh
source <(tap completion zsh)
# Fish
mkdir -p ~/.config/fish/completions
tap completion fish > ~/.config/fish/completions/tap.fish
# PowerShell / pwsh
tap completion pwsh > ~/.config/powershell/tap.ps1
```
Persistent install paths commonly used by package managers and dotfiles:
```bash
# Bash
mkdir -p ~/.local/share/bash-completion/completions
tap completion bash > ~/.local/share/bash-completion/completions/tap
# Zsh
mkdir -p ~/.zfunc
tap completion zsh > ~/.zfunc/_tap
```
## Common browser-backed flags
These show up on the relevant commands instead of only in global help:
| Flag | Description |
| --- | --- |
| `--browser`, `-b` | Force browser execution and reuse the resolved browser context |
| `--show` | Run the browser visibly |
| `--wait` | Wait a fixed duration after navigation |
| `--wait-selector` | Wait for a CSS selector |
| `--wait-js` | Wait for a JS expression |
| `--timeout` | Set execution timeout |
| `--browser-url` | One-shot DevTools override |
| `--profile-dir` | One-shot profile override |
| `--lightpanda`, `--lp` | Use Lightpanda instead of Chrome |
Compatibility aliases still work:
- `--ws-url` -> `--browser-url`
- `--delay` -> `--wait`
- `--no-headless` -> `--show`
## Advanced browser commands
The browser command still includes lower-level tools when needed:
```bash
tap browser evaluate ...
tap browser snapshot
tap browser forms
tap browser cookies ...
tap browser network ...
```
## Lightpanda
| Backend | Platforms | Best for |
| --- | --- | --- |
| Chrome | macOS, Linux, Windows | Full browser automation, auth, network interception |
| Lightpanda | macOS, Linux | Fast headless rendering without auth-heavy flows |
Install or update Lightpanda with:
```bash
tap doctor --install
```
## Docs
- [docs/browser.md](docs/browser.md) — browser UX and reference
- [docs/network.md](docs/network.md) — network interception reference
- [web/README.md](web/README.md) — web app docs
## Agent skill
Tap ships with a built-in agent skill that gives coding agents web access, site scripts, and browser automation.
```bash
npx skills add vaayne/tap
```
## Go library
```bash
go get github.com/vaayne/tap
```
```go
client, _ := tap.New()
defer client.Close()
result, _ := client.RunScript(ctx, "hackernews/top", nil)
content, _ := client.Fetch(ctx, "https://example.com", &fetch.Options{Markdown: true})
fmt.Println(content.Markdown)
```
## License
MIT