{"id":23924584,"url":"https://github.com/moos/quget","last_synced_at":"2025-04-12T02:33:13.775Z","repository":{"id":21205568,"uuid":"24518375","full_name":"moos/quget","owner":"moos","description":"Get web snippets from the command-line.","archived":false,"fork":false,"pushed_at":"2022-12-11T04:28:59.000Z","size":110,"stargazers_count":3,"open_issues_count":4,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-04-14T22:13:40.908Z","etag":null,"topics":["cheerio","cli","css-selector","jquery","snippet","web-snippets"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-09-26T23:27:48.000Z","updated_at":"2020-05-19T06:38:18.000Z","dependencies_parsed_at":"2023-01-13T21:20:38.461Z","dependency_job_id":null,"html_url":"https://github.com/moos/quget","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moos%2Fquget","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moos%2Fquget/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moos%2Fquget/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moos%2Fquget/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moos","download_url":"https://codeload.github.com/moos/quget/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247682055,"owners_count":20978602,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cheerio","cli","css-selector","jquery","snippet","web-snippets"],"created_at":"2025-01-05T19:15:42.193Z","updated_at":"2025-04-12T02:33:13.744Z","avatar_url":"https://github.com/moos.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# qsel -  web snippets from the command-line\n\n[![NPM version](https://img.shields.io/npm/v/qsel.svg)](https://www.npmjs.com/package/qsel)\n[![Build Status](https://img.shields.io/travis/moos/quget/master.svg)](https://travis-ci.org/moos/quget)\n\n## Introduction\nqsel (formerly `quget`) brings together the power of [request](https://github.com/request/request), [cheerio](https://github.com/cheeriojs/cheerio), and jQuery-like CSS selectors to the command-line.\n\n\n```bash\n$ qsel http://news.ycombinator.com \".title \u003e a\" --limit 3\nBest things and stuff of 2015\nWhen coding style survives compilation: De-anonymizing programmers from binaries\nPostgres features and tips\n```\n\n```bash\n$ qsel http://www.google.com/search?q=the+price+of+gold \"td._dmh \u003c tr|yellow\"\nGold Price Per Ounce$1,075.20$3.90\nGold Price Per Gram$34.57$0.13\nGold Price Per Kilo$34,568.46$125.39\n```\n\n\n```bash\n$ qsel https://github.com/trending?since=monthly \".repo-list-name|pack\" --limit 5\napple / swift\nFreeCodeCamp / FreeCodeCamp\nMaximAbramchuck / awesome-interviews\noneuijs / You-Dont-Need-jQuery\nphanan / koel\n```\n\n\n## Installation\n`npm install -g qsel`\n\n## Usage\n\n```text\n\n  Usage: qsel [command] [options] \u003curl\u003e [selector] | -\n\n  Example: qsel http://news.ycombinator.com \".title \u003e a|bold|red\" --limit 5\n\n  Options:                                                                                                         \n    -T, --template \u003ctemplate\u003e            template \"node: {{name}}, text {{.|text}}\"                                \n    -l, --limit \u003ccount\u003e                  limit query to count matches (-count from bottom) (default: 0)            \n    -r, --rand                           select randomly from matched set (can be combined with --limit)           \n    -j, --json                           full results object as (pretty) JSON                                      \n    -c, --compact                        when used with --json, outputs compact format                             \n    -n, --line-number                    add line numbers to output\n    --request-options \u003crequest-options\u003e  options for \"request\" as relaxed JSON, \"{foo: bar}\"                       \n    -h, --help                           output usage information                                                  \n    -V, --version                        output the version number                                                 \n\nBatch processing options:\n    - , --stdin                          read \u003curl\u003e(s) from STDIN                                                  \n    -o, --outfile \u003cfile\u003e                 file to output to (default: stdout)                                       \n    -q, --quite                          quite the logging                                                         \n    -p, --pause \u003cseconds\u003e                pause between batch requests (default: 0.0 secs)\n    --sep \u003cseperator\u003e                    seperator for multiple matches (default: \"\\n\")                            \n    --tag \u003ctag\u003e                          enclose complete result in \u003ctag\u003e\u003c/tag\u003e\n\nCommands:                                                                                              \n   samples [N]                          show samples, or run sample N                                   \n   help [what]                          get extra help with: pipes, selector, request-options           \n```\n\n## Selectors\n\nqsel supports all CSS3, some CSS4 and custom jQuery selectors like `:contains()`.  For complete list see [css-selelct](https://github.com/moos/css-select#supported-selectors), or run `qsel help selector`.\n\nIf no `selector` is given, the complete HTML of the page is returned.\n\n## Attributes\n\nIn general qsel returns the `text()` of the matched nodes.  To select an attribute, add the [x-ray](https://github.com/lapwinglabs/x-ray)-like `@` to the selector (*before the pipes!*).\n\n- `selector@\u003cattr-name\u003e` - get an attribute by name, e.g., `selector@href`\n- `selector@text` - get text content of matched nodes recursively (default)\n- `selector@html` - get the innerHTML\n\nMultiple attributes are supported: `selector@id@class`.\n\n\n## Filters / Pipes\n\nqsel supports [Markup.js](https://github.com/adammark/Markup.js)-type pipes separated by `|`, for example, `selector|upcase`, `selector|pack|tease 7`. For complete list see Markup.js' [built-in pipes](https://github.com/adammark/Markup.js#built-in-pipes).\n\nNeed some emphasis or color? All [chalk.styles](https://github.com/chalk/chalk#styles) are available as pipes as well: e.g. `selector|red`, `selector|bold|bgBlue`.\n\nAdditional pipes are defined in (src/pipes/basic.js):\n\n- `|after text` - add text after each match\n- `|before text` - add text before each match\n- `|quote text` - add text before and after each match\n- `|tag name` - enclose match in \u0026lt;name\u0026gt; and \u0026lt;/name\u0026gt;\n- `|incr N` - increment the match value by N (default 1)  \n- `|decr N` - decrement the match value by N (default 1)\n- `|regex (foo.*)` - match by regex\n- `|colorize` - apply random chalk style to every line\n\nUse `\\n` to add a new line, e.g. `selector|after \\n\\n`. For complete list, run `qsel help pipes`.\n\n## Shell pipe\n\nqsel can be forced to read from STDIN, either interactively or in a shell pipe, by providing the single dash option `-`.  In this mode, each line of input is read as a url and executed in order.  Each line may also contain its own `selector`.  If none is given, the `selector` from the CLI is used.\n\n```shell\n$ qsel http://news.ycombinator.com \".title \u003e a@href\" -l 3 | qsel - \"title|pack\"\nPage not found | Docker Blog\nPermission to Fail - Michelle Wetzler of Keen IO\nThe Amazon Whisperer\n```\n\n## Examples\n\nFor other examples, run `qsel samples`. (Note: samples are run using Node's `child_process.exec()` which gobbles colors in output streams.  To see the colors, run the command directly from the shell.)\n\n#### Run samples interactively\n```bash\n$ qsel samples\nChoose a sample to run:\n1. Hacker News titles\n2. Hacker News titles and subtext\n3. Wikipedia's On This Day\n4. GitHub trending\n5. Markup filters\n6. Cheerio selectors\n7. Beijing Air Twitter feed\n8. Custom template 1\n9. Custom template 2\n10. Jeopardy!\n\u003e\n```\n\n#### Run a sample\n```bash\n$ qsel samples 1\nRunning:\nqsel http://news.ycombinator.com \".title \u003e a\" -l 7 -n\n Hacker News titles\n\n 1. Apple, Google ban location tracking in apps using their contact-tracing system\n 2. Extremely disillusioned with technology. Please help\n 3. Stealing your SMS messages with iOS 0day\n 4. Experts Doubt the Sun Is Burning Coal (1863)\n 5. I gave away my books and sales increased\n 6. Iloveyou\n 7. Backblaze B2 Cloud Storage Now Has S3 Compatible APIs\n ```\n\n#### Simple\n```bash\n$ qsel http://www.google.com/search?q=the+price+of+gold \"td._dmh \u003c tr\" --limit 1\nGold Price Per Ounce$1,075.20$3.90\n```\n\n#### With JSON output\n```bash\n$ qsel http://www.google.com/search?q=the+price+of+gold \"td._dmh \u003c tr\" --limit 1 --json\n[\n  {\n    \"type\": \"tag\",\n    \"name\": \"tr\",\n    \"attribs\": {},\n    \"children\": [\n      {\n        \"type\": \"tag\",\n        \"name\": \"td\",\n        \"attribs\": {\n          \"class\": \"_dmh\"\n        },\n        \"children\": [\n          {\n            \"data\": \"Gold Price Per Ounce\",\n            \"type\": \"text\"\n          }\n        ]\n      },\n      {\n        \"type\": \"tag\",\n        \"name\": \"td\",\n        \"attribs\": {\n          \"class\": \"_dmh\"\n        },\n        \"children\": [\n          {\n            \"data\": \"$1,075.20\",\n            \"type\": \"text\"\n          }\n        ]\n      },\n      {\n        \"type\": \"tag\",\n        \"name\": \"td\",\n        \"attribs\": {\n          \"class\": \"_dmh\"\n        },\n        \"children\": [\n          {\n            \"data\": \"$3.90\",\n            \"type\": \"text\"\n          }\n        ]\n      }\n    ],\n    \"selectorIndex\": 0\n  }\n]\n```\n\n#### With JSON compact\n```bash\n$ qsel http://www.google.com/search?q=the+price+of+gold \"td._dmh \u003c tr\" --limit 1 --json --compact\n[{\"type\":\"tag\",\"name\":\"tr\",\"attribs\":{},\"children\":[{\"type\":\"tag\",\"name\":\"td\",\"attribs\":{\"class\":\"_dmh\"},\"children\":[{\"data\"\n:\"Gold Price Per Ounce\",\"type\":\"text\"}]},{\"type\":\"tag\",\"name\":\"td\",\"attribs\":{\"class\":\"_dmh\"},\"children\":[{\"data\":\"$1,075.20\n\",\"type\":\"text\"}]},{\"type\":\"tag\",\"name\":\"td\",\"attribs\":{\"class\":\"_dmh\"},\"children\":[{\"data\":\"$3.90\",\"type\":\"text\"}]}],\"selec\ntorIndex\":0}]\n```\n\n#### With line numbers\n```bash\n$ qsel http://www.google.com/search?q=the+price+of+gold \"td._dmh \u003c tr\" -n\n1. Gold Price Per Ounce$1,075.20$3.90\n2. Gold Price Per Gram$34.57$0.13\n3. Gold Price Per Kilo$34,568.46$125.39\n```\n\n#### Select at random\n```text\n$ qsel \"https://www.reddit.com/r/oneliners/top/?sort=top\u0026t=year\" \"a.title|colorize\" --limit 3 --rand\n\"DO NOT TOUCH\" must be one of the most terrifying things to read in braille.\nI can't believe no one has managed to come up with a cure for anorexia yet, honestly, I thought it'd be a piece of.\nGravity is one of the most fundamental forces in the universe, but if you remove it, you get gravy.\n```\n\n#### Custom template\n```text\n$ qsel http://www.google.com/search?q=the+price+of+gold \"td._dmh \u003c tr|yellow\"  -T \"#{{index|incr}} {{ty\npe|upcase}} {{name}} has {{children.length}} children: {{.|text}}\"\n#1 TAG tr has 3 children: Gold Price Per Ounce$1,075.20$3.90\n#2 TAG tr has 3 children: Gold Price Per Gram$34.57$0.13\n#3 TAG tr has 3 children: Gold Price Per Kilo$34,568.46$125.39\n```\nSee [Markup.js](https://github.com/adammark/Markup.js) for template help.\n\n#### Custom HTTP headers\nAny options for [request](https://github.com/request/request#requestoptions-callback) can be entered in a *relaxed* [jsonic](https://github.com/rjrodger/jsonic) format using `--request-options`.\n\n```bash\n$ qsel https://api.github.com/users/moos\nRequest forbidden by administrative rules. Please make sure your request has a User-Agent header (http://developer.github.com/v3/#user-agent-required). Check https://developer.github.com for other possible causes.\n\n$ qsel https://api.github.com/users/moos  --request-options \"headers:{\\\"User-Agent\\\":foo}\"\n{\"login\":\"moos\",\"id\":233047,\"avatar_url\":\"https://avatars.githubusercontent.com/u/233047?v=3\", [snip]\n```\n\n#### Define an alias\n```bash\n$ alias def='function _blah(){ qsel https://www.bing.com/search?q=define+$@ \"#b_results ol:first-child|bold\"; };_blah'\n$ def foo\n  a term used as a universal substitute for something real, especially when discussing technological ideas and problems\n```\n\n## Package note\nqsel relies on a [fork of css-select](https://github.com/moos/css-select) which supplies the matched selector index in the list of matched elements.  Since css-select is a dependency of cheerio, it ues `npm shrinkwrap` to load the fork.  Any updates to cheerio will require manually updating the shrinkwrap json file.  Hopefully with upcoming npm 3's flat dependency tree, this shebang can be eliminated.\n\n## Change log\n\n- 0.7.1 - Add better logging for batch ops.\n- 0.7.0 - Add `--tag` option.  Write results out incrementally in batch ops.\n- 0.6.0 - Add `---pause` option for batch operations\n- 0.5.0 - Renamed package to **qsel** (query selector)\n- 0.4.1 - Add `|tag foo` pipe.\n- 0.4.0 - Add `--outfile` and `--quite` options\n- 0.3.3 - Update cheerio to 0.22.0 compatible with lodash 4.17\n- 0.3.2 - Use moos/cheerio to pick up moos/css-select.  \n- 0.3.1 - Add `npm-shrinkwrap.json` back in as it's needed to pick up the right `css-select` for `cheerio`\n- 0.3.0 - Fix reading multiple URLs from STDIN\n- 0.2.4 - Early version\n\n## License\n\n(The MIT License)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoos%2Fquget","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoos%2Fquget","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoos%2Fquget/lists"}