{"id":13448098,"url":"https://github.com/benibela/xidel","last_synced_at":"2025-05-15T11:04:27.284Z","repository":{"id":33634821,"uuid":"37287172","full_name":"benibela/xidel","owner":"benibela","description":"Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching.  It can also create new or transformed XML/HTML/JSON documents.","archived":false,"fork":false,"pushed_at":"2025-02-22T16:58:02.000Z","size":2187,"stargazers_count":711,"open_issues_count":20,"forks_count":42,"subscribers_count":28,"default_branch":"master","last_synced_at":"2025-03-27T03:08:38.166Z","etag":null,"topics":["cli","command-line","css-selector","curl","data-processing","datascraping","html","http","httpie","json","rest","scraper","web","webscraper","webscraping","wget","xml","xmlstarlet","xpath","xquery"],"latest_commit_sha":null,"homepage":"http://www.videlibri.de/xidel.html","language":"Pascal","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benibela.png","metadata":{"files":{"readme":"README.md","changelog":"changelog","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-06-11T21:29:20.000Z","updated_at":"2025-03-26T16:32:30.000Z","dependencies_parsed_at":"2023-02-16T00:45:41.151Z","dependency_job_id":"44f3af03-95a9-4711-a9ec-2fcf12489839","html_url":"https://github.com/benibela/xidel","commit_stats":{"total_commits":752,"total_committers":3,"mean_commits":"250.66666666666666","dds":0.003989361702127714,"last_synced_commit":"436b2754c37def5f543c8902f98eb837cd366f56"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benibela%2Fxidel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benibela%2Fxidel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benibela%2Fxidel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benibela%2Fxidel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benibela","download_url":"https://codeload.github.com/benibela/xidel/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246933381,"owners_count":20857055,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","command-line","css-selector","curl","data-processing","datascraping","html","http","httpie","json","rest","scraper","web","webscraper","webscraping","wget","xml","xmlstarlet","xpath","xquery"],"created_at":"2024-07-31T05:01:35.534Z","updated_at":"2025-04-03T04:10:31.847Z","avatar_url":"https://github.com/benibela.png","language":"Pascal","funding_links":[],"categories":["Pascal","Command Line"],"sub_categories":["Like jq"],"readme":"Xidel [![Build Status](https://api.travis-ci.com/benibela/xidel.svg?branch=master)](https://travis-ci.com/benibela/xidel)\n=============\nXidel is a command line tool to download and extract data from HTML/XML pages using CSS selectors, XPath/XQuery 3.0, as well as querying JSON files or APIs (e.g. REST) using JSONiq. \n\nThere are dependency-free [binaries for Windows, Linux and Mac](http://www.videlibri.de/xidel.html). \n\nIt is a wrapper around my Pascal Internet Tools (see repository internettools), so it supports XPath 2.0, XPath 3.0, XQuery 1.0, XQuery 3.0, JSONiq, CSS selectors and my own extensions/languages (e.g. pattern matching) and if you can compile that project, you can compile Xidel. \n\nA simple example to return the titles of all pages linked by some starting page:\n\n     xidel http://example.org --follow //a --extract //title\n     \nor simpler\n\n     xidel http://example.org -f //a -e //title\n     \n\nThe language can be explicitly chosen. For example\n\n     xidel input.html --css 'a'\n     xidel input.html --xpath '//a/@href'\n     xidel input.html --xquery 'for $var in //a order by $var return $var'\n\nreturns all links, the target URI of each link or the text of all links alphabetically.\n\nThere are more examples on the above page with binaries, the github wiki and in the directory examples.\n\n\nScreenshots\n-----------\n\n\u003cp align=\"center\"\u003e\n\n\u003cimg src=\"http://www.videlibri.de/img/xidel-linux.png\" alt=\"Xidel on Linux\"\u003e\n\n\u003cimg src=\"http://www.videlibri.de/img/xidel-windows-blue.png\" alt=\"Xidel on Windows\"\u003e\n\n\u003c/p\u003e\n\nCompilation and Installation\n-----------\n\nYou can compile it by calling `build.sh` and install it by calling `build.sh -t`. Alternatively you can compile it with the Lazarus IDE.\n\nYou can call the commands from the [.travis.yml](.travis.yml) script to download dependencies.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenibela%2Fxidel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenibela%2Fxidel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenibela%2Fxidel/lists"}